| # Licensed to the Apache Software Foundation (ASF) under one |
| # or more contributor license agreements. See the NOTICE file |
| # distributed with this work for additional information |
| # regarding copyright ownership. The ASF licenses this file |
| # to you under the Apache License, Version 2.0 (the |
| # "License"); you may not use this file except in compliance |
| # with the License. You may obtain a copy of the License at |
| # |
| # http://www.apache.org/licenses/LICENSE-2.0 |
| # |
| # Unless required by applicable law or agreed to in writing, software |
| # distributed under the License is distributed on an "AS IS" BASIS, |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| # See the License for the specific language governing permissions and |
| # limitations under the License. |
| |
| A. Building & Installing Falcon |
| =============================== |
| |
| 0. Prerequisites |
| ------------------ |
| |
| You would need the following installed: |
| |
| * JDK 1.7 |
| * Maven 3.x |
| |
| *Note:* To build Falcon on Ubuntu/Centos set umask to 022. To do so, add the following line in ~/.bashrc or |
| ~/.bash_profile. |
| umask 022 |
| |
| 1. Building Falcon |
| ------------------ |
| |
| a. Building falcon from the source release |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| * tar -xzvf apache-falcon-${project.version}-source.tar.gz |
| * cd falcon-sources-${project.version} |
| |
| * export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m -noverify" && mvn clean install |
| |
| [optionally -Dhadoop.version=<<hadoop.version>> can be appended to build for a specific version of hadoop] |
| *Note 1:* Falcon drops support for Hadoop-1 and only supports Hadoop-2 from Falcon 0.6 onwards |
| Falcon build with JDK 1.7 using -noverify option |
| *Note 2:* To compile Falcon with addon extensions, append additional profiles to build command using syntax -P<<profile1,profile2>> |
| For Hive Mirroring extension, use profile"hivedr". Hive >= 1.2.0 and Oozie >= 4.2.0 is required |
| For HDFS Snapshot mirroring extension, use profile "hdfs-snapshot-mirroring". Hadoop >= 2.7.0 is required |
| For ADF integration, use profile "adf" |
| |
| |
| b. Building falcon from the source repository |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| * git clone https://git-wip-us.apache.org/repos/asf/falcon.git falcon |
| * cd falcon |
| * export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m -noverify" && mvn clean install |
| |
| [optionally -Dhadoop.version=<<hadoop.version>> can be appended to build for a specific version of hadoop] |
| *Note 1:* Falcon drops support for Hadoop-1 and only supports Hadoop-2 from Falcon 0.6 onwards |
| Falcon build with JDK 1.7 using -noverify option |
| *Note 2:* To compile Falcon with addon extensions, append additional profiles to build command using syntax -P<<profile1,profile2>> |
| For Hive Mirroring extension, use profile"hivedr". Hive >= 1.2.0 and Oozie >= 4.2.0 is required |
| For HDFS Snapshot mirroring extension, use profile "hdfs-snapshot-mirroring". Hadoop >= 2.7.0 is required |
| For ADF integration, use profile "adf" |
| |
| |
| 2. Deploying Falcon |
| ------------------- |
| |
| Once the build successfully completes, artifacts can be packaged for deployment. |
| By default the falcon is built for embedded mode. |
| |
| * mvn clean assembly:assembly -DskipTests -DskipITs |
| |
| Tar can be found in {project dir}/target/apache-falcon-${project.version}-bin.tar.gz |
| |
| Tar is structured as follows |
| |
| |- bin |
| |- falcon |
| |- falcon-start |
| |- falcon-stop |
| |- falcon-config.sh |
| |- service-start.sh |
| |- service-stop.sh |
| |- conf |
| |- startup.properties |
| |- runtime.properties |
| |- client.properties |
| |- log4j.xml |
| |- falcon-env.sh |
| |- docs |
| |- client |
| |- lib (client support libs) |
| |- server |
| |- webapp |
| |- falcon.war |
| |- hadooplibs |
| |- README |
| |- NOTICE.txt |
| |- LICENSE.txt |
| |- DISCLAIMER.txt |
| |- CHANGES.txt |
| |
| |
| 3. Installing & running Falcon |
| ------------------------------ |
| |
| a. Installing falcon |
| ~~~~~~~~~~~~~~~~~~~~ |
| |
| * tar -xzvf apache-falcon-${project.version}-bin.tar.gz |
| * cd falcon-${project.version} |
| |
| b. Starting Falcon Server |
| ~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| * bin/falcon-start |
| |
| c. Using Falcon |
| ~~~~~~~~~~~~~~~ |
| |
| * bin/falcon admin -version |
| Falcon server build version: {"properties":[{"key":"Version","value":"0.10-rbe02edf0d5b10af27bbac694e536bef30885c00e"}, |
| {"key":"Mode","value":"embedded"},{"key":"authentication","value":"simple"},{"key":"safemode","value":"false"}]} |
| * bin/falcon help |
| (for more details about falcon cli usage) |
| |
| d. Stopping Falcon Server |
| ~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| * bin/falcon-stop |
| |
| 4. Preparing oozie bundle for use with Falcon |
| --------------------------------------------- |
| |
| * cd <<project home>> |
| * src/bin/pacakge.sh <<hadoop-version>> |
| >> ex. src/bin/pacakge.sh 2.5.0 |
| >> oozie bundle available in target/package/oozie-4.1.0/distro/target/oozie-4.1.0-distro.tar.gz |