[LIVY-389][BUILD] Add maven plugin to generate license for all the dependencies

This change will generate a THIRD-PARTY file with license of all dependencies in `${execution.root}` folder, and will copy it to release assembly's root dir.

lresende ajbozarth can you please review the current solution, is it OK for an Apache way.

Here is the snippet of this THIRD-PARTY file:

```
===============================================================================

APACHE LIVY SUBCOMPONENTS:

The Apache Livy binary distribution includes a number of subcomponents
with separate copyright notices and license terms. Your use of the
code for the these subcomponents is subject to the terms and
conditions of the following licenses.

===============================================================================

We have listed all of these third party libraries and their licenses
below. This file can be regenerated at any time by simply running:

    mvn clean package

---------------------------------------------------
Third party Java libraries listed by License type.

PLEASE NOTE: Some dependencies may be listed under multiple licenses if they
are dual-licensed. This is especially true of anything listed as
"GNU General Public Library" below.
---------------------------------------------------

  ASL:

    * json4s-ast (org.json4s:json4s-ast_2.10:3.2.10 - https://github.com/json4s/json4s)
    * json4s-core (org.json4s:json4s-core_2.10:3.2.10 - https://github.com/json4s/json4s)
    * json4s-jackson (org.json4s:json4s-jackson_2.10:3.2.10 - https://github.com/json4s/json4s)

  Apache 2:

    * Apache Derby Database Engine and Embedded JDBC Driver (org.apache.derby:derby:10.10.2.0 - http://db.apache.org/derby/derby/)
    * JDO API (javax.jdo:jdo-api:3.0.1 - http://db.apache.org/jdo)
    * JavaEWAH (com.googlecode.javaewah:JavaEWAH:0.3.2 - http://code.google.com/p/javaewah/)
    * Joda-Convert (org.joda:joda-convert:1.6 - http://www.joda.org/joda-convert/)
    * Joda-Time (joda-time:joda-time:2.3 - http://www.joda.org/joda-time/)
    * Objenesis (org.objenesis:objenesis:1.2 - http://objenesis.googlecode.com/svn/docs/index.html)
    * RoaringBitmap (org.roaringbitmap:RoaringBitmap:0.5.11 - https://github.com/lemire/RoaringBitmap)
    * breeze (org.scalanlp:breeze_2.10:0.11.2 - http://scalanlp.org/)
    * breeze-macros (org.scalanlp:breeze-macros_2.10:0.11.2 - http://scalanlp.org/)
    * chill (com.twitter:chill_2.10:0.5.0 - https://github.com/twitter/chill)
    * chill-java (com.twitter:chill-java:0.5.0 - https://github.com/twitter/chill)
    * metrics-scala (nl.grons:metrics-scala_2.10:3.3.0_a2.3 - https://github.com/erikvanoosten/metrics-scala)
    * opencsv (net.sf.opencsv:opencsv:2.3 - http://opencsv.sf.net)
    * scala-ssh (com.decodified:scala-ssh_2.10:0.7.0 - https://github.com/sirthias/scala-ssh)
    * sshj (net.schmizz:sshj:0.10.0 - http://github.com/shikhar/sshj)

  Apache 2.0 License:

    * Spark Project Bagel (org.apache.spark:spark-bagel_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Catalyst (org.apache.spark:spark-catalyst_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Core (org.apache.spark:spark-core_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project GraphX (org.apache.spark:spark-graphx_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Hive (org.apache.spark:spark-hive_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Launcher (org.apache.spark:spark-launcher_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project ML Library (org.apache.spark:spark-mllib_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Networking (org.apache.spark:spark-network-common_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project REPL (org.apache.spark:spark-repl_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project SQL (org.apache.spark:spark-sql_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Shuffle Streaming Service (org.apache.spark:spark-network-shuffle_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Streaming (org.apache.spark:spark-streaming_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project Unsafe (org.apache.spark:spark-unsafe_2.10:1.6.2 - http://spark.apache.org/)
    * Spark Project YARN (org.apache.spark:spark-yarn_2.10:1.6.2 - http://spark.apache.org/spark-yarn_2.10/)

  Apache License:

    * HttpClient (commons-httpclient:commons-httpclient:3.1 - http://jakarta.apache.org/httpcomponents/httpclient-3.x/)
    * Tachyon Clients - Distribution (org.tachyonproject:tachyon-client:0.8.2 - http://tachyon-project.org/tachyon-clients/tachyon-client/)
    * Tachyon Under File System - HDFS (org.tachyonproject:tachyon-underfs-hdfs:0.8.2 - http://tachyon-project.org/tachyon-underfs/tachyon-underfs-hdfs/)
    * Tachyon Under File System - Local FS (org.tachyonproject:tachyon-underfs-local:0.8.2 - http://tachyon-project.org/tachyon-underfs/tachyon-underfs-local/)
    * Tachyon Under File System - S3 (org.tachyonproject:tachyon-underfs-s3:0.8.2 - http://tachyon-project.org/tachyon-underfs/tachyon-underfs-s3/)
```

Author: jerryshao <sshao@hortonworks.com>

Closes #34 from jerryshao/LIVY-389-third-party.
3 files changed
tree: c42a5245f4ef4fd491b8e7b8ad32a184ddc5e4a1
  1. api/
  2. assembly/
  3. bin/
  4. client-common/
  5. client-http/
  6. conf/
  7. core/
  8. coverage/
  9. dev/
  10. docs/
  11. examples/
  12. integration-test/
  13. python-api/
  14. repl/
  15. rsc/
  16. scala/
  17. scala-api/
  18. server/
  19. test-lib/
  20. .gitignore
  21. .rat-excludes
  22. .travis.yml
  23. checkstyle-suppressions.xml
  24. checkstyle.xml
  25. DISCLAIMER
  26. LICENSE
  27. NOTICE
  28. pom.xml
  29. README.md
  30. scalastyle.xml
README.md

Apache Livy

Build Status

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN.

  • Interactive Scala, Python and R shells
  • Batch submissions in Scala, Java, Python
  • Multiple users can share the same server (impersonation support)
  • Can be used for submitting jobs from anywhere with REST
  • Does not require any code change to your programs

Pull requests are welcomed! But before you begin, please check out the Contributing section on the Community page of our website.

Online Documentation

Guides and documentation on getting started using Livy, example code snippets, and Livy API documentation can be found at livy.incubator.apache.org.

Before Building Livy

To build Livy, you will need:

Debian/Ubuntu:

  • mvn (from maven package or maven3 tarball)
  • openjdk-7-jdk (or Oracle Java7 jdk)
  • Python 2.6+
  • R 3.x

Redhat/CentOS:

  • mvn (from maven package or maven3 tarball)
  • java-1.7.0-openjdk (or Oracle Java7 jdk)
  • Python 2.6+
  • R 3.x

MacOS:

  • Xcode command line tools
  • Oracle's JDK 1.7+
  • Maven (Homebrew)
  • Python 2.6+
  • R 3.x

Required python packages for building Livy:

  • cloudpickle
  • requests
  • requests-kerberos
  • flake8
  • flaky
  • pytest

To run Livy, you will also need a Spark installation. You can get Spark releases at https://spark.apache.org/downloads.html.

Livy requires at least Spark 1.6 and supports both Scala 2.10 and 2.11 builds of Spark, Livy will automatically pick repl dependencies through detecting the Scala version of Spark.

Livy also supports Spark 2.0+ for both interactive and batch submission, you could seamlessly switch to different versions of Spark through SPARK_HOME configuration, without needing to rebuild Livy.

Building Livy

Livy is built using Apache Maven. To check out and build Livy, run:

git clone https://github.com/apache/incubator-livy.git
cd livy
mvn package

By default Livy is built against Apache Spark 1.6.2, but the version of Spark used when running Livy does not need to match the version used to build Livy. Livy internally uses reflection to mitigate the gaps between different Spark versions, also Livy package itself does not contain a Spark distribution, so it will work with any supported version of Spark (Spark 1.6+) without needing to rebuild against specific version of Spark.