Google Git
Sign in
apache / activemq-web / refs/heads/main / . / shared-file-system-master-slave.html
blob: 1bbfaae5e0dea46f7c9976540bfc11210d4d3328 [file] [log] [blame]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<head>
<link href="http://activemq.apache.org/styles/site.css" rel="stylesheet" type="text/css"/>
<link href="http://activemq.apache.org/styles/type-settings.css" rel="stylesheet" type="text/css"/>
<script src="http://activemq.apache.org/styles/prototype.js" type="text/javascript"></script>
<script src="http://activemq.apache.org/styles/rico.js" type="text/javascript"></script>
<script src="http://activemq.apache.org/styles/site.js" type="text/javascript"></script>
<style type="text/css">
.maincontent { overflow:hidden; }
</style>
<!--[if IE]>
<style type="text/css">
.maincontent { width:100%; }
</style>
<![endif]-->
<link href='http://activemq.apache.org/styles/highlighter/styles/shCore.css' rel='stylesheet' type='text/css' />
<link href='http://activemq.apache.org/styles/highlighter/styles/shThemeEclipse.css' rel='stylesheet' type='text/css' />
<script src='http://activemq.apache.org/styles/highlighter/scripts/shCore.js' type='text/javascript'></script>
<script src='http://activemq.apache.org/styles/highlighter/scripts/shBrushJava.js' type='text/javascript'></script>
<script src='http://activemq.apache.org/styles/highlighter/scripts/shBrushXml.js' type='text/javascript'></script>
<script type="text/javascript">
SyntaxHighlighter.defaults['toolbar'] = false;
SyntaxHighlighter.all();
</script>
<title>
Apache ActiveMQ &#8482; -- Shared File System Master Slave
</title>
</head>
<body>
<div class="white_box">
<div class="header">
<div class="header_l">
<div class="header_r">
</div>
</div>
</div>
<div class="content">
<div class="content_l">
<div class="content_r">
<div>
<!-- Banner -->
<div id="asf_logo">
<div id="activemq_logo">
<a shape="rect" style="float:left; width:280px;display:block;text-indent:-5000px;text-decoration:none;line-height:60px; margin-top:10px; margin-left:100px;" href="http://activemq.apache.org" title="The most popular and powerful open source Message Broker">ActiveMQ</a>
<a shape="rect" style="float:right; width:210px;display:block;text-indent:-5000px;text-decoration:none;line-height:60px; margin-top:15px; margin-right:10px;" href="http://www.apache.org" title="The Apache Software Foundation">ASF</a>
</div>
</div>
<div class="top_red_bar">
<div id="site-breadcrumbs">
<a href="features.html">Features</a>&nbsp;&gt;&nbsp;<a href="clustering.html">Clustering</a>&nbsp;&gt;&nbsp;<a href="masterslave.html">MasterSlave</a>&nbsp;&gt;&nbsp;<a href="shared-file-system-master-slave.html">Shared File System Master Slave</a>
</div>
<div id="site-quicklinks">
<p><a shape="rect" href="download.html">Download</a> | <a shape="rect" class="external-link" href="http://activemq.apache.org/maven/apidocs/index.html">JavaDocs</a> <a shape="rect" href="javadocs.html">More...</a> | <a shape="rect" href="source.html">Source</a> | <a shape="rect" href="discussion-forums.html">Forums</a> | <a shape="rect" href="support.html">Support</a></p>
</div>
</div>
<table border="0">
<tbody>
<tr>
<td valign="top" width="100%">
<div class="wiki-content maincontent"><h2 id="SharedFileSystemMasterSlave-SharedFileSystemMasterSlave">Shared File System Master Slave</h2><p>If you have a SAN or shared file system it can be used to provide <em>high availability</em> such that if a broker is killed, another broker can take over immediately.</p><div class="confluence-information-macro confluence-information-macro-warning"><p class="title">Ensure your shared file locks work</p><span class="aui-icon aui-icon-small aui-iconfont-error confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>Note that the requirements of this failover system are a distributed file system like a SAN for which exclusive file locks work reliably. If you do not have such a thing available then consider using <a shape="rect" href="masterslave.html">MasterSlave</a> instead which implements something similar but working on commodity hardware using local file systems which ActiveMQ does the replication.</p><div class="confluence-information-macro confluence-information-macro-note"><p class="title">OCFS2 Warning</p><span class="aui-icon aui-icon-small aui-iconfont-warning confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>Was testing using OCFS2 and both brokers thought they had the master lock - this is because "OCFS2 only supports locking with 'fcntl' and not 'lockf and flock', therefore mutex file locking from Java isn't supported."</p><p>From <a shape="rect" class="external-link" href="http://sources.redhat.com/cluster/faq.html#gfs_vs_ocfs2" rel="nofollow">http://sources.redhat.com/cluster/faq.html#gfs_vs_ocfs2</a> :<br clear="none"> OCFS2: No cluster-aware flock or POSIX locks<br clear="none"> GFS: fully supports Cluster-wide flocks and POSIX locks and is supported.<br clear="none"> See this JIRA for more discussion: <a shape="rect" class="external-link" href="https://issues.apache.org/jira/browse/AMQ-4378">https://issues.apache.org/jira/browse/AMQ-4378</a></p></div></div><div class="confluence-information-macro confluence-information-macro-note"><p class="title">NFSv3 Warning</p><span class="aui-icon aui-icon-small aui-iconfont-warning confluence-information-macro-icon"></span><div class="confluence-information-macro-body"><p>In the event of an abnormal NFSv3 client termination (i.e., the ActiveMQ master broker), the NFSv3 server will not timeout the lock that is held by that client. This effectively renders the ActiveMQ data directory inaccessible because the ActiveMQ slave broker can't acquire the lock and therefore cannot start up. The only solution to this predicament with NFSv3 is to reboot all ActiveMQ instances to reset everything.</p><p>Use of NFSv4 is another solution because its design includes timeouts for locks. When using NFSv4 and the client holding the lock experiences an abnormal termination, by design, the lock is released after 30 seconds, allowing another client to grab the lock. For more information about this, see <a shape="rect" class="external-link" href="http://blogs.netapp.com/eislers_nfs_blog/2008/07/part-i-since-nf.html" rel="nofollow">this blog entry</a>.</p></div></div></div></div><p>Basically you can run as many brokers as you wish from the same shared file system directory. The first broker to grab the exclusive lock on the file is the master broker. If that broker dies and releases the lock then another broker takes over. The slave brokers sit in a loop trying to grab the lock from the master broker.</p><p>The following example shows how to configure a broker for Shared File System Master Slave where <strong>/sharedFileSystem</strong> is some directory on a shared file system. It is just a case of configuring a file based store to use a shared directory.</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: java; gutter: false; theme: Default" style="font-size:12px;"> &lt;persistenceAdapter&gt;
&lt;kahaDB directory="/sharedFileSystem/sharedBrokerData"/&gt;
&lt;/persistenceAdapter&gt;
</pre>
</div></div><p>or:</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: java; gutter: false; theme: Default" style="font-size:12px;"> &lt;persistenceAdapter&gt;
&lt;levelDB directory="/sharedFileSystem/sharedBrokerData"/&gt;
&lt;/persistenceAdapter&gt;
</pre>
</div></div><p>or:</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: java; gutter: false; theme: Default" style="font-size:12px;"> &lt;persistenceAdapter&gt;
&lt;amqPersistenceAdapter directory="/sharedFileSystem/sharedBrokerData"/&gt;
&lt;/persistenceAdapter&gt;
</pre>
</div></div><h3 id="SharedFileSystemMasterSlave-Startup">Startup</h3><p>On startup one master grabs an exclusive lock on the broker file directory - all other brokers are slaves and pause waiting for the exclusive lock.</p><p><span class="confluence-embedded-file-wrapper"><img class="confluence-embedded-image" src="shared-file-system-master-slave.data/Startup.png"></span></p><p>Clients should be using the <a shape="rect" href="failover-transport-reference.html">Failover Transport</a> to connect to the available brokers. e.g. using a URL something like the following</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: java; gutter: false; theme: Default" style="font-size:12px;">failover:(tcp://broker1:61616,tcp://broker2:61616,tcp://broker3:61616)
</pre>
</div></div><p>Only the master broker starts up its transport connectors and so the clients can only connect to the master.</p><h3 id="SharedFileSystemMasterSlave-Masterfailure">Master failure</h3><p>If the master looses the exclusive lock then it immediately shuts down. If a master shuts down or fails, one of the other slaves will grab the lock and so the topology switches to the following diagram</p><p><span class="confluence-embedded-file-wrapper"><img class="confluence-embedded-image" src="shared-file-system-master-slave.data/MasterFailed.png"></span></p><p>One of the other other slaves immediately grabs the exclusive lock on the file system to them commences becoming the master, starting all of its transport connectors.</p><p>Clients loose connection to the stopped master and then the failover transport tries to connect to the available brokers - of which the only one available is the new master.</p><h3 id="SharedFileSystemMasterSlave-Masterrestart">Master restart</h3><p>At any time you can restart other brokers which join the cluster and start as slaves waiting to become a master if the master is shutdown or a failure occurs. So the following topology is created after a restart of an old master...</p><p><span class="confluence-embedded-file-wrapper"><img class="confluence-embedded-image" src="shared-file-system-master-slave.data/MasterRestarted.png"></span></p><h3 id="SharedFileSystemMasterSlave-SchedulerSupport">Scheduler Support</h3><p>ActiveMQ maintains information about schedules independent to the settings in the persistence adapter. With a shared file-system it is therefore important to tell ActiveMQ expressly where to store scheduler information. To do this, set the&#160;<code>dataDirectory</code> attribute on the&#160;<code>broker</code>, for example:</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: xml; gutter: false; theme: Default" style="font-size:12px;">&lt;broker xmlns="http://activemq.apache.org/schema/core"
dataDirectory="/some/location"
brokerName="mmuserb2" useJmx="true" advisorySupport="false"
persistent="true" deleteAllMessagesOnStartup="false"
useShutdownHook="false" schedulerSupport="true"&gt;</pre>
</div></div></div>
</td>
<td valign="top">
<div class="navigation">
<div class="navigation_top">
<div class="navigation_bottom">
<h3 id="Navigation-Overviewhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35985"><a shape="rect" href="overview.html">Overview</a></h3><ul class="alternate"><li><a shape="rect" href="index.html">Index</a></li><li><a shape="rect" href="news.html">News</a></li><li><a shape="rect" href="new-features.html">New Features</a></li><li><a shape="rect" href="getting-started.html">Getting Started</a></li><li><a shape="rect" href="faq.html">FAQ</a></li><li><a shape="rect" href="articles.html">Articles</a></li><li><a shape="rect" href="books.html">Books</a></li><li><a shape="rect" href="download.html">Download</a></li><li><a shape="rect" class="external-link" href="http://www.apache.org/licenses/">License</a></li></ul><h3 id="Navigation-Search">Search</h3><div>
<form enctype="application/x-www-form-urlencoded" method="get" action="http://www.google.com/search" style="font-size: 10px;">
<input type="hidden" name="ie" value="UTF-8">
<input type="hidden" name="oe" value="UTF-8">
<input maxlength="255" type="text" name="q" size="15" value="value"><br clear="none">
<input type="submit" name="btnG" value="Search">
<input type="hidden" name="domains" value="activemq.apache.org">
<input type="hidden" name="sitesearch" value="activemq.apache.org">
</form>
</div>
<h3 id="Navigation-SubProjects">Sub Projects</h3><ul class="alternate"><li><a shape="rect" class="external-link" href="http://activemq.apache.org/artemis/">Artemis</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/apollo" title="ActiveMQ Apollo">Apollo</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/cms/">CMS</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/" title="NMS is the .Net Messaging API">NMS</a></li></ul><h3 id="Navigation-Communityhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=36130"><a shape="rect" href="community.html">Community</a></h3><ul class="alternate"><li><a shape="rect" href="support.html">Support</a></li><li><a shape="rect" href="contributing.html">Contributing</a></li><li><a shape="rect" href="discussion-forums.html">Discussion Forums</a></li><li><a shape="rect" href="mailing-lists.html">Mailing Lists</a></li><li><a shape="rect" href="irc.html">IRC</a></li><li><a shape="rect" class="external-link" href="http://javabot.evanchooly.com/logs/%23apache-activemq/today" rel="nofollow">IRC Log</a></li><li><a shape="rect" href="security-advisories.html">Security Advisories</a></li><li><a shape="rect" href="site.html">Site</a></li><li><a shape="rect" class="external-link" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li><li><a shape="rect" href="projects-using-activemq.html">Projects Using ActiveMQ</a></li><li><a shape="rect" href="users.html">Users</a></li><li><a shape="rect" href="team.html">Team</a></li><li><a shape="rect" href="thanks.html">Thanks</a></li></ul><h3 id="Navigation-Featureshttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35883"><a shape="rect" href="features.html">Features</a></h3><ul class="alternate"><li><a shape="rect" href="advisory-message.html">Advisory Message</a></li><li><a shape="rect" href="clustering.html">Clustering</a></li><li><a shape="rect" href="cross-language-clients.html">Cross Language Clients</a></li><li><a shape="rect" href="enterprise-integration-patterns.html">Enterprise Integration Patterns</a></li><li><a shape="rect" href="jmx.html">JMX</a></li><li><a shape="rect" href="jms-to-jms-bridge.html">JMS to JMS Bridge</a></li><li><a shape="rect" href="masterslave.html">MasterSlave</a></li><li><a shape="rect" href="message-groups.html">Message Groups</a></li><li><a shape="rect" href="networks-of-brokers.html">Networks of Brokers</a></li><li><a shape="rect" href="performance.html">Performance</a></li><li><a shape="rect" href="persistence.html">Persistence</a></li><li><a shape="rect" href="security.html">Security</a></li><li><a shape="rect" href="virtual-destinations.html">Virtual Destinations</a></li><li><a shape="rect" href="visualisation.html">Visualisation</a></li><li><a shape="rect" href="features.html">More ...</a></li></ul><h3 id="Navigation-Connectivityhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=36167"><a shape="rect" href="connectivity.html">Connectivity</a></h3><ul class="alternate"><li><a shape="rect" href="ajax.html">Ajax</a></li><li><a shape="rect" href="amqp.html">AMQP</a></li><li><a shape="rect" href="axis-and-cxf-support.html">Axis and CXF Support</a></li><li><a shape="rect" href="c-integration.html">C Integration</a></li><li><a shape="rect" href="activemq-c-clients.html">C++</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/">C# and .Net Integration</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/cms/">CMS</a></li><li><a shape="rect" href="j2ee.html">J2EE</a></li><li><a shape="rect" href="jboss-integration.html">JBoss Integration</a></li><li><a shape="rect" class="external-link" href="http://docs.codehaus.org/display/JETTY/Integrating+with+ActiveMQ" rel="nofollow">Jetty</a></li><li><a shape="rect" href="jndi-support.html">JNDI Support</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/" title="NMS is the .Net Messaging API">NMS</a></li><li><a shape="rect" href="rest.html">REST</a></li><li><a shape="rect" href="rss-and-atom.html">RSS and Atom</a></li><li><a shape="rect" href="spring-support.html">Spring Support</a></li><li><a shape="rect" href="stomp.html">Stomp</a></li><li><a shape="rect" href="tomcat.html">Tomcat</a></li><li><a shape="rect" href="unix-service.html">Unix Service</a></li><li><a shape="rect" href="weblogic-integration.html">WebLogic Integration</a></li><li><a shape="rect" href="xmpp.html">XMPP</a></li><li><a shape="rect" href="connectivity.html">More ...</a></li></ul><h3 id="Navigation-UsingActiveMQ5https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71176"><a shape="rect" href="using-activemq-5.html">Using ActiveMQ 5</a></h3><ul class="alternate"><li><a shape="rect" href="version-5-getting-started.html">Getting Started</a></li><li><a shape="rect" href="version-5-initial-configuration.html">Initial Configuration</a></li><li><a shape="rect" href="version-5-run-broker.html">Running a Broker</a></li><li><a shape="rect" href="how-do-i-embed-a-broker-inside-a-connection.html">Embedded Brokers</a></li><li><a shape="rect" href="activemq-command-line-tools-reference.html">Command Line Tools</a></li><li><a shape="rect" href="configuring-version-5-transports.html">Configuring Transports</a></li><li><a shape="rect" href="version-5-examples.html">Examples</a></li><li><a shape="rect" href="version-5-web-samples.html">Web Samples</a></li><li><a shape="rect" href="how-can-i-monitor-activemq.html">Monitoring the Broker</a></li><li><a shape="rect" href="version-5-xml-configuration.html">Xml Configuration</a></li><li><a shape="rect" href="xml-reference.html">Xml Reference</a></li><li><a shape="rect" href="using-activemq-5.html">More ...</a></li></ul><h3 id="Navigation-Toolshttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35912"><a shape="rect" href="tools.html">Tools</a></h3><ul class="alternate"><li><a shape="rect" href="web-console.html">Web Console</a></li><li><a shape="rect" href="activemq-performance-module-users-manual.html">Maven2 Performance Plugin</a></li></ul><h3 id="Navigation-Supporthttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35919"><a shape="rect" href="support.html">Support</a></h3><ul class="alternate"><li><a shape="rect" class="external-link" href="http://issues.apache.org/jira/browse/AMQ">Issues</a></li><li><a shape="rect" class="external-link" href="http://issues.apache.org/activemq/browse/AMQ?report=com.atlassian.jira.plugin.system.project:roadmap-panel">Roadmap</a></li><li><a shape="rect" class="external-link" href="http://issues.apache.org/activemq/browse/AMQ?report=com.atlassian.jira.plugin.system.project:changelog-panel">Change log</a></li></ul><h3 id="Navigation-Developershttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35903"><a shape="rect" href="developers.html">Developers</a></h3><ul class="alternate"><li><a shape="rect" href="source.html">Source</a></li><li><a shape="rect" href="building.html">Building</a></li><li><a shape="rect" href="developer-guide.html">Developer Guide</a></li><li><a shape="rect" href="becoming-a-committer.html">Becoming a committer</a></li><li><a shape="rect" href="code-overview.html">Code Overview</a></li><li><a shape="rect" href="wire-protocol.html">Wire Protocol</a></li><li><a shape="rect" href="release-guide.html">Release Guide</a></li></ul><h3 id="Navigation-Tests">Tests</h3><ul class="alternate"><li><a shape="rect" href="activemq-performance-module-users-manual.html">Maven2 Performance Plugin</a></li><li><a shape="rect" href="benchmark-tests.html">Benchmark Tests</a></li><li><a shape="rect" href="jmeter-system-tests.html">JMeter System Tests</a></li><li><a shape="rect" href="jmeter-performance-tests.html">JMeter Performance Tests</a></li><li><a shape="rect" href="integration-tests.html">Integration Tests</a></li></ul><h3 id="Navigation-ProjectReports">Project Reports</h3><ul class="alternate"><li><a shape="rect" href="junit-reports.html">JUnit Reports</a></li><li><a shape="rect" href="source-xref.html">Source XRef</a></li><li><a shape="rect" href="test-source-xref.html">Test Source XRef</a></li><li><a shape="rect" href="xml-reference.html">Xml Reference</a></li></ul>
</div>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<div class="bottom_red_bar"></div>
</div>
</div>
</div>
</div>
<div class="black_box">
<div class="footer">
<div class="footer_l">
<div class="footer_r">
<div>
<a href="http://activemq.apache.org/privacy-policy.html">Privacy Policy</a> -
(<a href="https://cwiki.apache.org/confluence/pages/editpage.action?pageId=35958">edit this page</a>)
</div>
</div>
</div>
</div>
</div>
</div>
<div class="design_attribution">
&copy; 2004-2011 The Apache Software Foundation.
<br/>
Apache ActiveMQ, ActiveMQ, Apache, the Apache feather logo, and the Apache ActiveMQ project logo are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.
<br/>
<a href="http://hiramchirino.com">Graphic Design By Hiram</a>
</div>
<!-- delay the loading of large javascript files to the end so that they don't interfere with the loading of page content -->
<span style="display: none">
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-1347593-1");
pageTracker._initData();
pageTracker._trackPageview();
</script>
</span>
</body>
</html>