commit | 21380280677b2e284b1efc2c73654ec3fc538b42 | [log] [tgz] |
---|---|---|
author | jerryshao <sshao@hortonworks.com> | Tue Aug 15 13:40:54 2017 +0800 |
committer | jerryshao <sshao@hortonworks.com> | Tue Aug 15 13:40:54 2017 +0800 |
tree | c42a5245f4ef4fd491b8e7b8ad32a184ddc5e4a1 | |
parent | a4760c6683b4642fb388d2adbd4cd03ff557cb64 [diff] |
[LIVY-389][BUILD] Add maven plugin to generate license for all the dependencies This change will generate a THIRD-PARTY file with license of all dependencies in `${execution.root}` folder, and will copy it to release assembly's root dir. lresende ajbozarth can you please review the current solution, is it OK for an Apache way. Here is the snippet of this THIRD-PARTY file: ``` =============================================================================== APACHE LIVY SUBCOMPONENTS: The Apache Livy binary distribution includes a number of subcomponents with separate copyright notices and license terms. Your use of the code for the these subcomponents is subject to the terms and conditions of the following licenses. =============================================================================== We have listed all of these third party libraries and their licenses below. This file can be regenerated at any time by simply running: mvn clean package --------------------------------------------------- Third party Java libraries listed by License type. PLEASE NOTE: Some dependencies may be listed under multiple licenses if they are dual-licensed. This is especially true of anything listed as "GNU General Public Library" below. --------------------------------------------------- ASL: * json4s-ast (org.json4s:json4s-ast_2.10:3.2.10 - https://github.com/json4s/json4s) * json4s-core (org.json4s:json4s-core_2.10:3.2.10 - https://github.com/json4s/json4s) * json4s-jackson (org.json4s:json4s-jackson_2.10:3.2.10 - https://github.com/json4s/json4s) Apache 2: * Apache Derby Database Engine and Embedded JDBC Driver (org.apache.derby:derby:10.10.2.0 - http://db.apache.org/derby/derby/) * JDO API (javax.jdo:jdo-api:3.0.1 - http://db.apache.org/jdo) * JavaEWAH (com.googlecode.javaewah:JavaEWAH:0.3.2 - http://code.google.com/p/javaewah/) * Joda-Convert (org.joda:joda-convert:1.6 - http://www.joda.org/joda-convert/) * Joda-Time (joda-time:joda-time:2.3 - http://www.joda.org/joda-time/) * Objenesis (org.objenesis:objenesis:1.2 - http://objenesis.googlecode.com/svn/docs/index.html) * RoaringBitmap (org.roaringbitmap:RoaringBitmap:0.5.11 - https://github.com/lemire/RoaringBitmap) * breeze (org.scalanlp:breeze_2.10:0.11.2 - http://scalanlp.org/) * breeze-macros (org.scalanlp:breeze-macros_2.10:0.11.2 - http://scalanlp.org/) * chill (com.twitter:chill_2.10:0.5.0 - https://github.com/twitter/chill) * chill-java (com.twitter:chill-java:0.5.0 - https://github.com/twitter/chill) * metrics-scala (nl.grons:metrics-scala_2.10:3.3.0_a2.3 - https://github.com/erikvanoosten/metrics-scala) * opencsv (net.sf.opencsv:opencsv:2.3 - http://opencsv.sf.net) * scala-ssh (com.decodified:scala-ssh_2.10:0.7.0 - https://github.com/sirthias/scala-ssh) * sshj (net.schmizz:sshj:0.10.0 - http://github.com/shikhar/sshj) Apache 2.0 License: * Spark Project Bagel (org.apache.spark:spark-bagel_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Catalyst (org.apache.spark:spark-catalyst_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Core (org.apache.spark:spark-core_2.10:1.6.2 - http://spark.apache.org/) * Spark Project GraphX (org.apache.spark:spark-graphx_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Hive (org.apache.spark:spark-hive_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Launcher (org.apache.spark:spark-launcher_2.10:1.6.2 - http://spark.apache.org/) * Spark Project ML Library (org.apache.spark:spark-mllib_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Networking (org.apache.spark:spark-network-common_2.10:1.6.2 - http://spark.apache.org/) * Spark Project REPL (org.apache.spark:spark-repl_2.10:1.6.2 - http://spark.apache.org/) * Spark Project SQL (org.apache.spark:spark-sql_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Shuffle Streaming Service (org.apache.spark:spark-network-shuffle_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Streaming (org.apache.spark:spark-streaming_2.10:1.6.2 - http://spark.apache.org/) * Spark Project Unsafe (org.apache.spark:spark-unsafe_2.10:1.6.2 - http://spark.apache.org/) * Spark Project YARN (org.apache.spark:spark-yarn_2.10:1.6.2 - http://spark.apache.org/spark-yarn_2.10/) Apache License: * HttpClient (commons-httpclient:commons-httpclient:3.1 - http://jakarta.apache.org/httpcomponents/httpclient-3.x/) * Tachyon Clients - Distribution (org.tachyonproject:tachyon-client:0.8.2 - http://tachyon-project.org/tachyon-clients/tachyon-client/) * Tachyon Under File System - HDFS (org.tachyonproject:tachyon-underfs-hdfs:0.8.2 - http://tachyon-project.org/tachyon-underfs/tachyon-underfs-hdfs/) * Tachyon Under File System - Local FS (org.tachyonproject:tachyon-underfs-local:0.8.2 - http://tachyon-project.org/tachyon-underfs/tachyon-underfs-local/) * Tachyon Under File System - S3 (org.tachyonproject:tachyon-underfs-s3:0.8.2 - http://tachyon-project.org/tachyon-underfs/tachyon-underfs-s3/) ``` Author: jerryshao <sshao@hortonworks.com> Closes #34 from jerryshao/LIVY-389-third-party.
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in Apache Hadoop YARN.
Pull requests are welcomed! But before you begin, please check out the Contributing section on the Community page of our website.
Guides and documentation on getting started using Livy, example code snippets, and Livy API documentation can be found at livy.incubator.apache.org.
To build Livy, you will need:
Debian/Ubuntu:
maven
package or maven3 tarball)Redhat/CentOS:
maven
package or maven3 tarball)MacOS:
Required python packages for building Livy:
To run Livy, you will also need a Spark installation. You can get Spark releases at https://spark.apache.org/downloads.html.
Livy requires at least Spark 1.6 and supports both Scala 2.10 and 2.11 builds of Spark, Livy will automatically pick repl dependencies through detecting the Scala version of Spark.
Livy also supports Spark 2.0+ for both interactive and batch submission, you could seamlessly switch to different versions of Spark through SPARK_HOME
configuration, without needing to rebuild Livy.
Livy is built using Apache Maven. To check out and build Livy, run:
git clone https://github.com/apache/incubator-livy.git cd livy mvn package
By default Livy is built against Apache Spark 1.6.2, but the version of Spark used when running Livy does not need to match the version used to build Livy. Livy internally uses reflection to mitigate the gaps between different Spark versions, also Livy package itself does not contain a Spark distribution, so it will work with any supported version of Spark (Spark 1.6+) without needing to rebuild against specific version of Spark.