| <!DOCTYPE html> |
| <html lang="en"><head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"><!-- Begin Jekyll SEO tag v2.8.0 --> |
| <title>Starting Apache Falcon in distributed mode | Blogs Archive</title> |
| <meta name="generator" content="Jekyll v3.9.3" /> |
| <meta property="og:title" content="Starting Apache Falcon in distributed mode" /> |
| <meta property="og:locale" content="en_US" /> |
| <meta name="description" content="Apache Falcon can be deployed in two modes, Embedded mode and Distributed mode. Both the modes have there uses which are described below : 1. Embedded Mode : Embedded mode is also called standalone mode. This setup is useful when you have only one cluster from where you want data to be managed and processed. All the request like submit, delete, schedule etc go to the single server that is started. To start falcon in distributed mode the following steps are required : 1. Build falcon in distributed mode 2. Edit startup property for falcon server / servers 3. Edit runtime property for falcon prism 4. Example Step 1. Build falcon in distributed mode Please follow the steps here for building falcon in distributed mode. If above step is successful you should have 3 ".tar.gz" files at the end ,one each for falcon prism, server and client in the target dirs. Step 2. Edit startup property for falcon server For starting falcon in distributed we need to provide logical names to servers we are starting. These names need to be mentioned in startup.properties for the given server. Add the following property in the "conf/startup.properties" in the server package you have untar. *.current.colo=< colo name > Colo is a colocation or a physical data centre location where one or more clusters may be configured. If you are planning to start more than one server add the above in each of the conf/startup.properties . Remember the cole name you have mentioned here. Same name should be used in cluster.xml while submitted a cluster later via prism. Start the servers using the commands given here here Step 3. Edit runtime property for falcon prism Untar the falcon prism package. Once servers have been configured and started, prism need to be configured with information on where these servers are started. For that below two properties needs to be added to prism "conf/runtime.properties" . *.all.colos=< colo name 1 >, < colo name 2 > *.falcon.< colo name 1 >.endpoint=http://< hostname 1 >:15000 *.falcon.< colo name 2 >.endpoint=http://< hostname 2 >:15000 Start Falcon Prism using commands given here here Example : If you are configuring a falcon setup with 1 prism and 2 servers, with cluster name cluster1 and cluster2, below are the thing that needs to added : 1. edit server one startup.properties and add *.current.colo=colo1 2. edit server two startup.properties and add *.current.colo=colo2 3. Start the servers 4. edit prism runtime.properties and add *.all.colos=colo1, colo2 *.falcon.colo1.endpoint=http://< hostname 1 >:15000 *.falcon.colo2.endpoint=http://< hostname 2 >:15000 5. Start falcon prism. Note: By default falcon server start with its own activemq, if for any reason you dont want embedded activemq (like starting more than one server on same host) , it can be disabled. Step 1. Disable start of activemq in falcon server start script edit "bin/falcon-start" script and add FALCON_OPTS="-Dfalcon.embeddedmq=false" Step 2. In startup.properties of falcon server add your own activemq url *.broker.url=tcp://localhost:61616" /> |
| <meta property="og:description" content="Apache Falcon can be deployed in two modes, Embedded mode and Distributed mode. Both the modes have there uses which are described below : 1. Embedded Mode : Embedded mode is also called standalone mode. This setup is useful when you have only one cluster from where you want data to be managed and processed. All the request like submit, delete, schedule etc go to the single server that is started. To start falcon in distributed mode the following steps are required : 1. Build falcon in distributed mode 2. Edit startup property for falcon server / servers 3. Edit runtime property for falcon prism 4. Example Step 1. Build falcon in distributed mode Please follow the steps here for building falcon in distributed mode. If above step is successful you should have 3 ".tar.gz" files at the end ,one each for falcon prism, server and client in the target dirs. Step 2. Edit startup property for falcon server For starting falcon in distributed we need to provide logical names to servers we are starting. These names need to be mentioned in startup.properties for the given server. Add the following property in the "conf/startup.properties" in the server package you have untar. *.current.colo=< colo name > Colo is a colocation or a physical data centre location where one or more clusters may be configured. If you are planning to start more than one server add the above in each of the conf/startup.properties . Remember the cole name you have mentioned here. Same name should be used in cluster.xml while submitted a cluster later via prism. Start the servers using the commands given here here Step 3. Edit runtime property for falcon prism Untar the falcon prism package. Once servers have been configured and started, prism need to be configured with information on where these servers are started. For that below two properties needs to be added to prism "conf/runtime.properties" . *.all.colos=< colo name 1 >, < colo name 2 > *.falcon.< colo name 1 >.endpoint=http://< hostname 1 >:15000 *.falcon.< colo name 2 >.endpoint=http://< hostname 2 >:15000 Start Falcon Prism using commands given here here Example : If you are configuring a falcon setup with 1 prism and 2 servers, with cluster name cluster1 and cluster2, below are the thing that needs to added : 1. edit server one startup.properties and add *.current.colo=colo1 2. edit server two startup.properties and add *.current.colo=colo2 3. Start the servers 4. edit prism runtime.properties and add *.all.colos=colo1, colo2 *.falcon.colo1.endpoint=http://< hostname 1 >:15000 *.falcon.colo2.endpoint=http://< hostname 2 >:15000 5. Start falcon prism. Note: By default falcon server start with its own activemq, if for any reason you dont want embedded activemq (like starting more than one server on same host) , it can be disabled. Step 1. Disable start of activemq in falcon server start script edit "bin/falcon-start" script and add FALCON_OPTS="-Dfalcon.embeddedmq=false" Step 2. In startup.properties of falcon server add your own activemq url *.broker.url=tcp://localhost:61616" /> |
| <link rel="canonical" href="http://localhost:4000/falcon/entry/starting_falcon_in_distributed_mode" /> |
| <meta property="og:url" content="http://localhost:4000/falcon/entry/starting_falcon_in_distributed_mode" /> |
| <meta property="og:site_name" content="Blogs Archive" /> |
| <meta property="og:type" content="article" /> |
| <meta property="article:published_time" content="2014-03-20T15:25:44-04:00" /> |
| <meta name="twitter:card" content="summary" /> |
| <meta property="twitter:title" content="Starting Apache Falcon in distributed mode" /> |
| <script type="application/ld+json"> |
| {"@context":"https://schema.org","@type":"BlogPosting","dateModified":"2014-03-20T15:25:44-04:00","datePublished":"2014-03-20T15:25:44-04:00","description":"Apache Falcon can be deployed in two modes, Embedded mode and Distributed mode. Both the modes have there uses which are described below : 1. Embedded Mode : Embedded mode is also called standalone mode. This setup is useful when you have only one cluster from where you want data to be managed and processed. All the request like submit, delete, schedule etc go to the single server that is started. To start falcon in distributed mode the following steps are required : 1. Build falcon in distributed mode 2. Edit startup property for falcon server / servers 3. Edit runtime property for falcon prism 4. Example Step 1. Build falcon in distributed mode Please follow the steps here for building falcon in distributed mode. If above step is successful you should have 3 ".tar.gz" files at the end ,one each for falcon prism, server and client in the target dirs. Step 2. Edit startup property for falcon server For starting falcon in distributed we need to provide logical names to servers we are starting. These names need to be mentioned in startup.properties for the given server. Add the following property in the "conf/startup.properties" in the server package you have untar. *.current.colo=< colo name > Colo is a colocation or a physical data centre location where one or more clusters may be configured. If you are planning to start more than one server add the above in each of the conf/startup.properties . Remember the cole name you have mentioned here. Same name should be used in cluster.xml while submitted a cluster later via prism. Start the servers using the commands given here here Step 3. Edit runtime property for falcon prism Untar the falcon prism package. Once servers have been configured and started, prism need to be configured with information on where these servers are started. For that below two properties needs to be added to prism "conf/runtime.properties" . *.all.colos=< colo name 1 >, < colo name 2 > *.falcon.< colo name 1 >.endpoint=http://< hostname 1 >:15000 *.falcon.< colo name 2 >.endpoint=http://< hostname 2 >:15000 Start Falcon Prism using commands given here here Example : If you are configuring a falcon setup with 1 prism and 2 servers, with cluster name cluster1 and cluster2, below are the thing that needs to added : 1. edit server one startup.properties and add *.current.colo=colo1 2. edit server two startup.properties and add *.current.colo=colo2 3. Start the servers 4. edit prism runtime.properties and add *.all.colos=colo1, colo2 *.falcon.colo1.endpoint=http://< hostname 1 >:15000 *.falcon.colo2.endpoint=http://< hostname 2 >:15000 5. Start falcon prism. Note: By default falcon server start with its own activemq, if for any reason you dont want embedded activemq (like starting more than one server on same host) , it can be disabled. Step 1. Disable start of activemq in falcon server start script edit "bin/falcon-start" script and add FALCON_OPTS="-Dfalcon.embeddedmq=false" Step 2. In startup.properties of falcon server add your own activemq url *.broker.url=tcp://localhost:61616","headline":"Starting Apache Falcon in distributed mode","mainEntityOfPage":{"@type":"WebPage","@id":"http://localhost:4000/falcon/entry/starting_falcon_in_distributed_mode"},"url":"http://localhost:4000/falcon/entry/starting_falcon_in_distributed_mode"}</script> |
| <!-- End Jekyll SEO tag --> |
| <link rel="stylesheet" href="/assets/main.css"><link type="application/atom+xml" rel="alternate" href="http://localhost:4000/feed.xml" title="Blogs Archive" /></head> |
| <body><header class="site-header" role="banner"> |
| |
| <div class="wrapper"><a class="site-title" rel="author" href="/">Blogs Archive</a><nav class="site-nav"> |
| <input type="checkbox" id="nav-trigger" class="nav-trigger" /> |
| <label for="nav-trigger"> |
| <span class="menu-icon"> |
| <svg viewBox="0 0 18 15" width="18px" height="15px"> |
| <path d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.032C17.335,0,18,0.665,18,1.484L18,1.484z M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.032C17.335,6.031,18,6.696,18,7.516L18,7.516z M18,13.516C18,14.335,17.335,15,16.516,15H1.484 C0.665,15,0,14.335,0,13.516l0,0c0-0.82,0.665-1.483,1.484-1.483h15.032C17.335,12.031,18,12.695,18,13.516L18,13.516z"/> |
| </svg> |
| </span> |
| </label> |
| |
| <div class="trigger"><a class="page-link" href="/about/">About</a></div> |
| </nav></div> |
| </header> |
| <main class="page-content" aria-label="Content"> |
| <div class="wrapper"> |
| <article class="post h-entry" itemscope itemtype="http://schema.org/BlogPosting"> |
| |
| <header class="post-header"> |
| <h1 class="post-title p-name" itemprop="name headline">Starting Apache Falcon in distributed mode</h1> |
| <p class="post-meta"> |
| <time class="dt-published" datetime="2014-03-20T15:25:44-04:00" itemprop="datePublished">Mar 20, 2014 |
| </time>• <span itemprop="author" itemscope itemtype="http://schema.org/Person"><span class="p-author h-card" itemprop="name">{"display_name"=>"Samarth Gupta", "login"=>"samarthg", "email"=>"samarthg@apache.org"}</span></span></p> |
| </header> |
| |
| <div class="post-content e-content" itemprop="articleBody"> |
| <p>Apache Falcon can be deployed in two modes, Embedded mode and Distributed mode. Both the modes have there uses which are described below : </p> |
| <p>1. Embedded Mode : Embedded mode is also called standalone mode. This setup is useful when you have only one cluster from where you want data to be managed and processed. All the request like submit, delete, schedule etc go to the single server that is started. </p> |
| <h3>To start falcon in distributed mode the following steps are required : </h3> |
| <p>1. Build falcon in distributed mode<br /> |
| 2. Edit startup property for falcon server / servers<br /> |
| 3. Edit runtime property for falcon prism<br /> |
| 4. Example</p> |
| <h4>Step 1. Build falcon in distributed mode</h4> |
| <p>Please follow the steps <a href="http://falcon.incubator.apache.org/docs/InstallationSteps.html">here</a> for building falcon in distributed mode.<br /> |
| If above step is successful you should have 3 ".tar.gz" files at the end ,one each for falcon prism, server and client in the target dirs. </p> |
| <h4>Step 2. Edit startup property for falcon server</h4> |
| <p>For starting falcon in distributed we need to provide logical names to servers we are starting. These names need to be mentioned in startup.properties for the given server. </p> |
| <p>Add the following property in the "conf/startup.properties" in the server package you have untar. </p> |
| <pre class="code2"> *.current.colo=< colo name >
|
| </pre> |
| <p>Colo is a colocation or a physical data centre location where one or more clusters may be configured.</p> |
| <p>If you are planning to start more than one server add the above in each of the conf/startup.properties .<br /> |
| Remember the cole name you have mentioned here. Same name should be used in cluster.xml while submitted a cluster later via prism. </p> |
| <p>Start the servers using the commands given here <a href="http://falcon.incubator.apache.org/docs/InstallationSteps.html">here</a> </p> |
| <h4>Step 3. Edit runtime property for falcon prism</h4> |
| <p>Untar the falcon prism package. </p> |
| <p>Once servers have been configured and started, prism need to be configured with information on where these servers are started.<br /> |
| For that below two properties needs to be added to prism "conf/runtime.properties" . </p> |
| <pre class="code2">*.all.colos=< colo name 1 >, < colo name 2 >
|
| *.falcon.< colo name 1 >.endpoint=http://< hostname 1 >:15000
|
| *.falcon.< colo name 2 >.endpoint=http://< hostname 2 >:15000
|
| </pre> |
| <p>Start Falcon Prism using commands given here <a href="http://falcon.incubator.apache.org/docs/InstallationSteps.html">here</a> </p> |
| <h4>Example : </h4> |
| <p>If you are configuring a falcon setup with 1 prism and 2 servers, with cluster name cluster1 and cluster2, below are the thing that needs to added :</p> |
| <p>1. edit server one startup.properties and add *.current.colo=colo1<br /> |
| 2. edit server two startup.properties and add *.current.colo=colo2<br /> |
| 3. Start the servers<br /> |
| 4. edit prism runtime.properties and add </p> |
| <pre class="code2"> *.all.colos=colo1, colo2
|
| *.falcon.colo1.endpoint=http://< hostname 1 >:15000
|
| *.falcon.colo2.endpoint=http://< hostname 2 >:15000
|
| </pre> |
| <p>5. Start falcon prism. </p> |
| <h3>Note:</h3> |
| <p> By default falcon server start with its own activemq, if for any reason you dont want embedded activemq (like starting more than one server on same host) , it can be disabled. </p> |
| <p>Step 1. Disable start of activemq in falcon server start script<br /> |
| edit "bin/falcon-start" script and add<br /> |
| FALCON_OPTS="-Dfalcon.embeddedmq=false"</p> |
| <p>Step 2. In startup.properties of falcon server add your own activemq url </p> |
| <pre class="code2"> *.broker.url=tcp://localhost:61616
|
| </pre> |
| |
| </div><a class="u-url" href="/falcon/entry/starting_falcon_in_distributed_mode" hidden></a> |
| </article> |
| |
| </div> |
| </main><footer class="site-footer h-card"> |
| <data class="u-url" href="/"></data> |
| |
| <div class="wrapper"> |
| |
| <h2 class="footer-heading">Blogs Archive</h2> |
| |
| <div class="footer-col-wrapper"> |
| <div class="footer-col footer-col-1"> |
| <ul class="contact-list"> |
| <li class="p-name">Blogs Archive</li><li><a class="u-email" href="mailto:issues@infra.apache.org">issues@infra.apache.org</a></li></ul> |
| </div> |
| |
| <div class="footer-col footer-col-2"><ul class="social-media-list"><li><a href="https://github.com/jekyll"><svg class="svg-icon"><use xlink:href="/assets/minima-social-icons.svg#github"></use></svg> <span class="username">jekyll</span></a></li><li><a href="https://www.twitter.com/jekyllrb"><svg class="svg-icon"><use xlink:href="/assets/minima-social-icons.svg#twitter"></use></svg> <span class="username">jekyllrb</span></a></li></ul> |
| </div> |
| |
| <div class="footer-col footer-col-3"> |
| <p>This is an archive of the Roller blogs that were previously hosted on blogs.apache.org</p> |
| </div> |
| </div> |
| |
| </div> |
| |
| </footer> |
| </body> |
| |
| </html> |
