| <!DOCTYPE html> |
| <!-- |
| | Generated by Apache Maven Doxia at 2018-03-12 |
| | Rendered using Apache Maven Fluido Skin 1.3.0 |
| --> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head> |
| <meta charset="UTF-8" /> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
| <meta name="Date-Revision-yyyymmdd" content="20180312" /> |
| <meta http-equiv="Content-Language" content="en" /> |
| <title>Falcon - HDFS DR Recipe</title> |
| <link rel="stylesheet" href="./css/apache-maven-fluido-1.3.0.min.css" /> |
| <link rel="stylesheet" href="./css/site.css" /> |
| <link rel="stylesheet" href="./css/print.css" media="print" /> |
| |
| |
| <script type="text/javascript" src="./js/apache-maven-fluido-1.3.0.min.js"></script> |
| |
| |
| |
| <script type="text/javascript">$( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } );</script> |
| |
| </head> |
| <body class="topBarDisabled"> |
| |
| |
| |
| <div class="container-fluid"> |
| <div id="banner"> |
| <div class="pull-left"> |
| <a href="../index.html" id="bannerLeft"> |
| <img src="images/falcon-logo.png" alt="Apache Falcon" width="200px" height="45px"/> |
| </a> |
| </div> |
| <div class="pull-right"> <a href="http://www.apache.org" id="bannerRight"> |
| <img src="images/apache-feather-tm.gif" alt="Falcon" height="45px"/> |
| </a> |
| </div> |
| <div class="clear"><hr/></div> |
| </div> |
| |
| <div id="breadcrumbs"> |
| <ul class="breadcrumb"> |
| |
| |
| <li class=""> |
| <a href="http://www.apache.org" class="externalLink" title="Apache"> |
| Apache</a> |
| </li> |
| <li class="divider ">/</li> |
| <li class=""> |
| <a href="index.html" title="Falcon"> |
| Falcon</a> |
| </li> |
| <li class="divider ">/</li> |
| <li class="">HDFS DR Recipe</li> |
| |
| |
| |
| <li id="publishDate" class="pull-right">Last Published: 2018-03-12</li> |
| |
| </ul> |
| </div> |
| |
| |
| <div class="row-fluid"> |
| <div id="leftColumn" class="span3"> |
| <div class="well sidebar-nav"> |
| |
| |
| <ul class="nav nav-list"> |
| <li class="nav-header">Falcon</li> |
| |
| <li> |
| |
| <a href="index.html" title="About"> |
| <i class="none"></i> |
| About</a> |
| </li> |
| |
| <li> |
| |
| <a href="slides/falcon-overview.html" title="Overview"> |
| <i class="none"></i> |
| Overview</a> |
| </li> |
| |
| <li> |
| |
| <a href="slides/falcon-user-guide.html" title="User Guide"> |
| <i class="none"></i> |
| User Guide</a> |
| </li> |
| |
| <li> |
| |
| <a href="GettingStarted.html" title="Getting Started"> |
| <i class="none"></i> |
| Getting Started</a> |
| </li> |
| |
| <li> |
| |
| <a href="FalconDocumentation.html" title="Architecture"> |
| <i class="none"></i> |
| Architecture</a> |
| </li> |
| |
| <li> |
| |
| <a href="InstallationSteps.html" title="Installation"> |
| <i class="none"></i> |
| Installation</a> |
| </li> |
| |
| <li> |
| |
| <a href="OnBoarding.html" title="On Boarding"> |
| <i class="none"></i> |
| On Boarding</a> |
| </li> |
| |
| <li> |
| |
| <a href="MigrationInstructions.html" title="Migrate to 0.10"> |
| <i class="none"></i> |
| Migrate to 0.10</a> |
| </li> |
| |
| <li> |
| |
| <a href="Operability.html" title="Operability"> |
| <i class="none"></i> |
| Operability</a> |
| </li> |
| |
| <li> |
| |
| <a href="EntitySpecification.html" title="Entity Specification"> |
| <i class="none"></i> |
| Entity Specification</a> |
| </li> |
| |
| <li> |
| |
| <a href="falconcli/FalconCLI.html" title="Client (Falcon CLI)"> |
| <i class="none"></i> |
| Client (Falcon CLI)</a> |
| </li> |
| |
| <li> |
| |
| <a href="restapi/ResourceList.html" title="Rest API"> |
| <i class="icon-chevron-right"></i> |
| Rest API</a> |
| </li> |
| |
| <li> |
| |
| <a href="HiveIntegration.html" title="Hive Integration"> |
| <i class="none"></i> |
| Hive Integration</a> |
| </li> |
| |
| <li> |
| |
| <a href="Extensions.html" title="Server side Extensions"> |
| <i class="none"></i> |
| Server side Extensions</a> |
| </li> |
| |
| <li> |
| |
| <a href="Security.html" title="Security"> |
| <i class="none"></i> |
| Security</a> |
| </li> |
| <li class="nav-header">Project Information</li> |
| |
| <li> |
| |
| <a href="project-info.html" title="Summary"> |
| <i class="none"></i> |
| Summary</a> |
| </li> |
| |
| <li> |
| |
| <a href="mail-lists.html" title="Mailing Lists"> |
| <i class="none"></i> |
| Mailing Lists</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://webchat.freenode.net?channels=apachefalcon&uio=d4" class="externalLink" title="IRC"> |
| <i class="none"></i> |
| IRC</a> |
| </li> |
| |
| <li> |
| |
| <a href="team-list.html" title="Team"> |
| <i class="none"></i> |
| Team</a> |
| </li> |
| |
| <li> |
| |
| <a href="issue-tracking.html" title="Issue Tracking"> |
| <i class="none"></i> |
| Issue Tracking</a> |
| </li> |
| |
| <li> |
| |
| <a href="source-repository.html" title="Source Repository"> |
| <i class="none"></i> |
| Source Repository</a> |
| </li> |
| |
| <li> |
| |
| <a href="https://cwiki.apache.org/confluence/display/FALCON/Index" class="externalLink" title="Wiki"> |
| <i class="none"></i> |
| Wiki</a> |
| </li> |
| |
| <li> |
| |
| <a href="license.html" title="License"> |
| <i class="none"></i> |
| License</a> |
| </li> |
| |
| <li> |
| |
| <a href="https://cwiki.apache.org/confluence/display/FALCON/News" class="externalLink" title="News"> |
| <i class="none"></i> |
| News</a> |
| </li> |
| |
| <li> |
| |
| <a href="https://cwiki.apache.org/confluence/display/FALCON/PoweredBy" class="externalLink" title="Powered by"> |
| <i class="none"></i> |
| Powered by</a> |
| </li> |
| |
| <li> |
| |
| <a href="https://cwiki.apache.org/confluence/display/FALCON/Acknowledgements" class="externalLink" title="Acknowledgements"> |
| <i class="none"></i> |
| Acknowledgements</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://blogs.apache.org/falcon/" class="externalLink" title="Blog"> |
| <i class="none"></i> |
| Blog</a> |
| </li> |
| <li class="nav-header">Releases</li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/dyn/closer.lua/falcon/0.11" class="externalLink" title="0.11"> |
| <i class="none"></i> |
| 0.11</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/dyn/closer.lua/falcon/0.10" class="externalLink" title="0.10"> |
| <i class="none"></i> |
| 0.10</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/dyn/closer.lua/falcon/0.9" class="externalLink" title="0.9"> |
| <i class="none"></i> |
| 0.9</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/dyn/closer.lua/falcon/0.8" class="externalLink" title="0.8"> |
| <i class="none"></i> |
| 0.8</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/dyn/closer.lua/falcon/0.7" class="externalLink" title="0.7"> |
| <i class="none"></i> |
| 0.7</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://archive.apache.org/dist/falcon/0.6.1" class="externalLink" title="0.6.1"> |
| <i class="none"></i> |
| 0.6.1</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://archive.apache.org/dist/incubator/falcon/0.6-incubating" class="externalLink" title="0.6-incubating"> |
| <i class="none"></i> |
| 0.6-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://archive.apache.org/dist/incubator/falcon/0.5-incubating" class="externalLink" title="0.5-incubating"> |
| <i class="none"></i> |
| 0.5-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://archive.apache.org/dist/incubator/falcon/0.4-incubating" class="externalLink" title="0.4-incubating"> |
| <i class="none"></i> |
| 0.4-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://archive.apache.org/dist/incubator/falcon/0.3-incubating" class="externalLink" title="0.3-incubating"> |
| <i class="none"></i> |
| 0.3-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="https://cwiki.apache.org/confluence/display/FALCON/Roadmap" class="externalLink" title="Coming soon"> |
| <i class="none"></i> |
| Coming soon</a> |
| </li> |
| <li class="nav-header">Documentation</li> |
| |
| <li> |
| |
| <a href="0.11/index.html" title="0.11 (Current)"> |
| <i class="none"></i> |
| 0.11 (Current)</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.10/index.html" title="0.10"> |
| <i class="none"></i> |
| 0.10</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.9/index.html" title="0.9"> |
| <i class="none"></i> |
| 0.9</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.8/index.html" title="0.8"> |
| <i class="none"></i> |
| 0.8</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.7/index.html" title="0.7"> |
| <i class="none"></i> |
| 0.7</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.6.1/index.html" title="0.6.1"> |
| <i class="none"></i> |
| 0.6.1</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.6-incubating/index.html" title="0.6-incubating"> |
| <i class="none"></i> |
| 0.6-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.5-incubating/index.html" title="0.5-incubating"> |
| <i class="none"></i> |
| 0.5-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.4-incubating/index.html" title="0.4-incubating"> |
| <i class="none"></i> |
| 0.4-incubating</a> |
| </li> |
| |
| <li> |
| |
| <a href="0.3-incubating/index.html" title="0.3-incubating"> |
| <i class="none"></i> |
| 0.3-incubating</a> |
| </li> |
| <li class="nav-header">ASF</li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/foundation/how-it-works.html" class="externalLink" title="How Apache Works"> |
| <i class="none"></i> |
| How Apache Works</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/foundation/" class="externalLink" title="Foundation"> |
| <i class="none"></i> |
| Foundation</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/foundation/sponsorship.html" class="externalLink" title="Sponsoring Apache"> |
| <i class="none"></i> |
| Sponsoring Apache</a> |
| </li> |
| |
| <li> |
| |
| <a href="http://www.apache.org/foundation/thanks.html" class="externalLink" title="Thanks"> |
| <i class="none"></i> |
| Thanks</a> |
| </li> |
| </ul> |
| |
| |
| |
| <hr class="divider" /> |
| |
| <div id="poweredBy"> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <div class="clear"></div> |
| <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"> |
| <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> |
| </a> |
| </div> |
| </div> |
| </div> |
| |
| |
| <div id="bodyColumn" class="span9" > |
| |
| <div class="section"> |
| <h2>HDFS DR Recipe<a name="HDFS_DR_Recipe"></a></h2></div> |
| <div class="section"> |
| <h3>Overview<a name="Overview"></a></h3> |
| <p>Falcon supports HDFS DR recipe to replicate data from source cluster to destination cluster.</p></div> |
| <div class="section"> |
| <h3>Usage<a name="Usage"></a></h3></div> |
| <div class="section"> |
| <h4>Setup cluster definition.<a name="Setup_cluster_definition."></a></h4> |
| <div class="source"> |
| <pre> |
| $FALCON_HOME/bin/falcon entity -submit -type cluster -file /cluster/definition.xml |
| |
| </pre></div></div> |
| <div class="section"> |
| <h4>Update recipes properties<a name="Update_recipes_properties"></a></h4> |
| <p>Copy HDFS replication recipe properties, workflow and template file from $FALCON_HOME/data-mirroring/hdfs-replication to the accessible directory path or to the recipe directory path (<b>falcon.recipe.path=<recipe directory path></b>). <b>"falcon.recipe.path"</b> must be specified in Falcon conf client.properties. Now update the copied recipe properties file with required attributes to replicate data from source cluster to destination cluster for HDFS DR.</p></div> |
| <div class="section"> |
| <h4>Submit HDFS DR recipe<a name="Submit_HDFS_DR_recipe"></a></h4> |
| <p>After updating the recipe properties file with required attributes in directory path or in falcon.recipe.path, there are two ways of submitting the HDFS DR recipe:</p> |
| <p></p> |
| <ul> |
| <li>1. Specify Falcon recipe properties file through recipe command line.</li></ul> |
| <div class="source"> |
| <pre> |
| $FALCON_HOME/bin/falcon recipe -name hdfs-replication -operation HDFS_REPLICATION |
| -properties /cluster/hdfs-replication.properties |
| |
| </pre></div> |
| <p></p> |
| <ul> |
| <li>2. Use Falcon recipe path specified in Falcon conf client.properties .</li></ul> |
| <div class="source"> |
| <pre> |
| $FALCON_HOME/bin/falcon recipe -name hdfs-replication -operation HDFS_REPLICATION |
| |
| </pre></div> |
| <p><b>Note:</b> Recipe properties file, workflow file and template file name must match to the recipe name, it must be unique and in the same directory.</p></div> |
| </div> |
| </div> |
| </div> |
| |
| <hr/> |
| |
| <footer> |
| <div class="container-fluid"> |
| <div class="row span12">Copyright © 2013-2018 |
| <a href="http://www.apache.org">Apache Software Foundation</a>. |
| All Rights Reserved. |
| |
| </div> |
| |
| |
| |
| </div> |
| </footer> |
| </body> |
| </html> |
