commit | 0af2ea047428d4e4c5fa4a623f9d4963e5c17a28 | [log] [tgz] |
---|---|---|
author | Lee moon soo <moon@apache.org> | Sun Nov 08 10:03:34 2015 +0900 |
committer | Lee moon soo <moon@apache.org> | Mon Nov 09 10:30:12 2015 +0900 |
tree | 5077f712673df44f82de1a98abd135802f196a8f | |
parent | 1e9729fff442750ac96af7f42d30a83382b77df1 [diff] |
ZEPPELIN-404 Certain project dependencies are pulled from 3rd parties repos instead of ASF or public Maven https://issues.apache.org/jira/browse/ZEPPELIN-404 This PR removes / move 3rd party repository into profile. cloudera repo is disabled by default and can be activated by -Pvendor-repo Author: Lee moon soo <moon@apache.org> Closes #404 from Leemoonsoo/ZEPPELIN-404 and squashes the following commits: fe403a9 [Lee moon soo] Fix typo. vender-repo -> vendor-repo 5b55ac0 [Lee moon soo] move cloudera repository into profile (cherry picked from commit 785ee3c8b46225da75698458acc5b730e8ead542) Signed-off-by: Lee moon soo <moon@apache.org>
#Zeppelin
Documentation: User Guide
Mailing List: User and Dev mailing list
Continuous Integration:
Contributing: Contribution Guide
License: Apache 2.0
Zeppelin, a web-based notebook that enables interactive data analytics. You can make beautiful data-driven, interactive and collaborative documents with SQL, Scala and more.
Core feature:
To know more about Zeppelin, visit our web site http://zeppelin.incubator.apache.org
If you don't have requirements prepared, install it. (The installation method may vary according to your environment, example is for Ubuntu.)
sudo apt-get update sudo apt-get install openjdk-7-jdk sudo apt-get install git sudo apt-get install maven sudo apt-get install npm sudo apt-get install libfontconfig
If you want to build Zeppelin from the source, please first clone this repository. And then:
mvn clean package -DskipTests
Build with specific Spark version
Spark 1.5.x
mvn clean package -Pspark-1.5 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests
Spark 1.4.x
mvn clean package -Pspark-1.4 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests
Spark 1.3.x
mvn clean package -Pspark-1.3 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests
Spark 1.2.x
mvn clean package -Pspark-1.2 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests
Spark 1.1.x
mvn clean package -Pspark-1.1 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests
CDH 5.X
mvn clean package -Pspark-1.2 -Dhadoop.version=2.5.0-cdh5.3.0 -Phadoop-2.4 -DskipTests
Yarn (Hadoop 2.7.x)
mvn clean package -Pspark-1.4 -Dspark.version=1.4.1 -Dhadoop.version=2.7.0 -Phadoop-2.6 -Pyarn -DskipTests
Yarn (Hadoop 2.6.x)
mvn clean package -Pspark-1.1 -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn -DskipTests
Yarn (Hadoop 2.4.x)
mvn clean package -Pspark-1.1 -Dhadoop.version=2.4.0 -Phadoop-2.4 -Pyarn -DskipTests
Yarn (Hadoop 2.3.x)
mvn clean package -Pspark-1.1 -Dhadoop.version=2.3.0 -Phadoop-2.3 -Pyarn -DskipTests
Yarn (Hadoop 2.2.x)
mvn clean package -Pspark-1.1 -Dhadoop.version=2.2.0 -Phadoop-2.2 -Pyarn -DskipTests
Ignite (1.1.0-incubating and later)
mvn clean package -Dignite.version=1.1.0-incubating -DskipTests
If you wish to configure Zeppelin option (like port number), configure the following files:
./conf/zeppelin-env.sh ./conf/zeppelin-site.xml
(You can copy ./conf/zeppelin-env.sh.template
into ./conf/zeppelin-env.sh
. Same for zeppelin-site.xml
.)
Without SPARK_HOME and HADOOP_HOME, Zeppelin uses embedded Spark and Hadoop binaries that you have specified with mvn build option. If you want to use system provided Spark and Hadoop, export SPARK_HOME and HADOOP_HOME in zeppelin-env.sh You can use any supported version of spark without rebuilding Zeppelin.
# ./conf/zeppelin-env.sh export SPARK_HOME=... export HADOOP_HOME=...
Mesos
# ./conf/zeppelin-env.sh export MASTER=mesos://... export ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=/path/to/spark-*.tgz" or SPARK_HOME="/path/to/spark_home" export MESOS_NATIVE_LIBRARY=/path/to/libmesos.so
If you set SPARK_HOME
, you should deploy spark binary on the same location to all worker nodes. And if you set spark.executor.uri
, every worker can read that file on its node.
Yarn
# ./conf/zeppelin-env.sh export SPARK_HOME=/path/to/spark_dir
./bin/zeppelin-daemon.sh start browse localhost:8080 in your browser.
For configuration details check ./conf subdirectory.
To package final distribution do:
mvn clean package -P build-distr
The archive is generated under zeppelin-distribution/target directory
###Run end-to-end tests Zeppelin comes with a set of end-to-end acceptance tests driving headless selenium browser
#assumes zeppelin-server running on localhost:8080 (use -Durl=.. to override) mvn verify #or take care of starting\stoping zeppelin-server from packaged _zeppelin-distribuion/target_ mvn verify -P using-packaged-distr