Google Git
Sign in
apache / activemq-web / refs/heads/main / . / amq-message-store.html
blob: 88a834764fbbe41b2d60d79732152e4d109be134 [file] [log] [blame]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<head>
<link href="http://activemq.apache.org/styles/site.css" rel="stylesheet" type="text/css"/>
<link href="http://activemq.apache.org/styles/type-settings.css" rel="stylesheet" type="text/css"/>
<script src="http://activemq.apache.org/styles/prototype.js" type="text/javascript"></script>
<script src="http://activemq.apache.org/styles/rico.js" type="text/javascript"></script>
<script src="http://activemq.apache.org/styles/site.js" type="text/javascript"></script>
<style type="text/css">
.maincontent { overflow:hidden; }
</style>
<!--[if IE]>
<style type="text/css">
.maincontent { width:100%; }
</style>
<![endif]-->
<link href='http://activemq.apache.org/styles/highlighter/styles/shCore.css' rel='stylesheet' type='text/css' />
<link href='http://activemq.apache.org/styles/highlighter/styles/shThemeEclipse.css' rel='stylesheet' type='text/css' />
<script src='http://activemq.apache.org/styles/highlighter/scripts/shCore.js' type='text/javascript'></script>
<script src='http://activemq.apache.org/styles/highlighter/scripts/shBrushJava.js' type='text/javascript'></script>
<script type="text/javascript">
SyntaxHighlighter.defaults['toolbar'] = false;
SyntaxHighlighter.all();
</script>
<title>
Apache ActiveMQ &#8482; -- AMQ Message Store
</title>
</head>
<body>
<div class="white_box">
<div class="header">
<div class="header_l">
<div class="header_r">
</div>
</div>
</div>
<div class="content">
<div class="content_l">
<div class="content_r">
<div>
<!-- Banner -->
<div id="asf_logo">
<div id="activemq_logo">
<a shape="rect" style="float:left; width:280px;display:block;text-indent:-5000px;text-decoration:none;line-height:60px; margin-top:10px; margin-left:100px;" href="http://activemq.apache.org" title="The most popular and powerful open source Message Broker">ActiveMQ</a>
<a shape="rect" style="float:right; width:210px;display:block;text-indent:-5000px;text-decoration:none;line-height:60px; margin-top:15px; margin-right:10px;" href="http://www.apache.org" title="The Apache Software Foundation">ASF</a>
</div>
</div>
<div class="top_red_bar">
<div id="site-breadcrumbs">
<a href="features.html">Features</a>&nbsp;&gt;&nbsp;<a href="persistence.html">Persistence</a>&nbsp;&gt;&nbsp;<a href="amq-message-store.html">AMQ Message Store</a>
</div>
<div id="site-quicklinks">
<p><a shape="rect" href="download.html">Download</a> | <a shape="rect" class="external-link" href="http://activemq.apache.org/maven/apidocs/index.html">JavaDocs</a> <a shape="rect" href="javadocs.html">More...</a> | <a shape="rect" href="source.html">Source</a> | <a shape="rect" href="discussion-forums.html">Forums</a> | <a shape="rect" href="support.html">Support</a></p>
</div>
</div>
<table border="0">
<tbody>
<tr>
<td valign="top" width="100%">
<div class="wiki-content maincontent"><h2 id="AMQMessageStore-TheBasics">The Basics</h2>
<p>This is the default storage for AcitveMQ 5 and above. The AMQ Message Store is an embeddable transactional message storage solution that is extremely fast and reliable.<br clear="none">
message commands are written to a transactional journal - which consists of rolling data logs - which means writing is extremely fast and the state of the store is easily recoverable.</p>
<p>Messages themselves are persisted in the data logs of the journal - with references to their location being held by a reference store (by default Kaha) for fast retrevial.</p>
<div class="sectionColumnWrapper"><div class="sectionMacro">
<div class="sectionMacroRow"><div class="columnMacro" style="width:50%;min-width:50%;max-width:50%;">
<p><span class="confluence-embedded-file-wrapper image-left-wrapper"><img class="confluence-embedded-image image-left" src="amq-message-store.data/amqstore.png"></span></p></div>
<div class="columnMacro" style="width:50%;min-width:50%;max-width:50%;">
<p><br clear="none" class="atl-forced-newline">
<br clear="none" class="atl-forced-newline">
<br clear="none" class="atl-forced-newline">
References to messages are held in memory, and periodically inserted into the reference store to improve performance.
<br clear="none" class="atl-forced-newline">
<br clear="none" class="atl-forced-newline">
The messages are stored in data logs, which are individual files, typically 32mb in size (though this is configurable, they can be larger if the size of a message is large than the file size). When all the messages in a data log have been successfully consumed, the data log file is marked as being ready to be deleted - or archived - which will happen at the next clean up period.</p></div>
</div></div></div>
<h2 id="AMQMessageStore-Configuration">Configuration</h2>
<p>By default ActiveMQ will use the the AMQ Store - and its default settings. You can configure the properties of the AMQ Store however, by explictly defining its persistence adapter (amqPersistenceAdapter):</p>
<div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: java; gutter: false; theme: Default" style="font-size:12px;">
&lt;broker brokerName="broker" persistent="true" useShutdownHook="false"&gt;
&lt;persistenceAdapter&gt;
&lt;amqPersistenceAdapter directory="${activemq.base}/activemq-data" maxFileLength="32mb"/&gt;
&lt;/persistenceAdapter&gt;
&lt;transportConnectors&gt;
&lt;transportConnector uri="tcp://localhost:61616"/&gt;
&lt;/transportConnectors&gt;
&lt;/broker&gt;
</pre>
</div></div>
<p>The above shows the configuration required to set the AMQ Store through its amqPersistenceAdapter - and explicity setting the directory and maxFileLength properties.</p>
<h3 id="AMQMessageStore-AMQStoreProperties">AMQ Store Properties</h3>
<div class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1" rowspan="1" class="confluenceTh"><p>property name</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>default value</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Comments</p></th></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>directory</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>activemq-data</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>the path to the directory to use to store the message store data and log files</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>useNIO</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>true</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>use NIO to write messages to the data logs</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>syncOnWrite</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>false</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>sync every write to disk</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>maxFileLength</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>32mb</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>a hint to set the maximum size of the message data logs</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>persistentIndex</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>true</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>use a persistent index for the message logs. If this is false, an in-memory structure is maintained</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>maxCheckpointMessageAddSize</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>4kb</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>the maximum number of messages to keep in a transaction before automatically committing</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>cleanupInterval</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>30000</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>time (ms) before checking for a discarding/moving message data logs that are no longer used</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>indexBinSize</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>1024</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>default number of bins used by the index. The bigger the bin size - the better the relative performance of the index</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>indexKeySize</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>96</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>the size of the index key - the key is the message id </p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>indexPageSize</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>16kb</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>the size of the index page - the bigger the page - the better the write performance of the index </p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>directoryArchive</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>archive</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>the path to the directory to use to store discarded data logs</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>archiveDataLogs</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>false</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>if true data logs are moved to the archive directory instead of being deleted</p></td></tr></tbody></table></div>
<h2 id="AMQMessageStore-DataStructure">Data Structure</h2>
<p>In the data directory defined for the AMQ Store there is the following directory structure:</p>
<div class="sectionColumnWrapper"><div class="sectionMacro">
<div class="sectionMacroRow"><div class="columnMacro" style="width:50%;min-width:50%;max-width:50%;">
<p><span class="confluence-embedded-file-wrapper image-left-wrapper"><img class="confluence-embedded-image image-left" src="amq-message-store.data/amqdir.png"></span></p></div>
<div class="columnMacro" style="width:50%;min-width:50%;max-width:50%;">
<h3 id="AMQMessageStore-Toplevel">Top level</h3>
<p>the message broker's name is used to distinguish its directory of message data. By default, the broker name is local host.<br clear="none">
Below this top level directory are the following sub directories:</p>
<h3 id="AMQMessageStore-archive">archive</h3>
<p>message data logs are moved here when they are discarded.<br clear="none">
n.b. this directory only exists when the property archiveDataLogs is enabled</p>
<h3 id="AMQMessageStore-journal">journal</h3>
<p>Used to hold the message data logs</p>
<h3 id="AMQMessageStore-kr-store">kr-store</h3>
<p>The directory structure of the Kaha reference store (if used)</p>
<h5 id="AMQMessageStore-data">data</h5>
<p>The indexes used to reference the message data logs in the journal for fast retrieval</p>
<h5 id="AMQMessageStore-state">state</h5>
<p>The state of the store - i.e. names of durable subscribers - the reason for this is described in <a shape="rect" href="amq-message-store.html">Recovery</a></p>
<h3 id="AMQMessageStore-tmp-storage">tmp-storage</h3>
<p>use to hold data files for transient messages that may be stored on disk to alleviate memory consumption - e.g. non-persistent topic messages awaiting delivery to an active, but slow subscriber.</p></div></div></div></div>
<h2 id="AMQMessageStore-Recovery">Recovery</h2>
<p>If the message broker does not shutdown properly, then the reference store indexes are cleaned and the message data files (which contain messages/acknowledgements and transactional boundaries) are replayed to rebuild up the message store state. It is possbile to force automatic recovery if using the Kaha reference store (the default) by deleting the kr-store/state/ directory.</p></div>
</td>
<td valign="top">
<div class="navigation">
<div class="navigation_top">
<div class="navigation_bottom">
<h3 id="Navigation-Overviewhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35985"><a shape="rect" href="overview.html">Overview</a></h3><ul class="alternate"><li><a shape="rect" href="index.html">Index</a></li><li><a shape="rect" href="news.html">News</a></li><li><a shape="rect" href="new-features.html">New Features</a></li><li><a shape="rect" href="getting-started.html">Getting Started</a></li><li><a shape="rect" href="faq.html">FAQ</a></li><li><a shape="rect" href="articles.html">Articles</a></li><li><a shape="rect" href="books.html">Books</a></li><li><a shape="rect" href="download.html">Download</a></li><li><a shape="rect" class="external-link" href="http://www.apache.org/licenses/">License</a></li></ul><h3 id="Navigation-Search">Search</h3><div>
<form enctype="application/x-www-form-urlencoded" method="get" action="http://www.google.com/search" style="font-size: 10px;">
<input type="hidden" name="ie" value="UTF-8">
<input type="hidden" name="oe" value="UTF-8">
<input maxlength="255" type="text" name="q" size="15" value="value"><br clear="none">
<input type="submit" name="btnG" value="Search">
<input type="hidden" name="domains" value="activemq.apache.org">
<input type="hidden" name="sitesearch" value="activemq.apache.org">
</form>
</div>
<h3 id="Navigation-SubProjects">Sub Projects</h3><ul class="alternate"><li><a shape="rect" class="external-link" href="http://activemq.apache.org/artemis/">Artemis</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/apollo" title="ActiveMQ Apollo">Apollo</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/cms/">CMS</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/" title="NMS is the .Net Messaging API">NMS</a></li></ul><h3 id="Navigation-Communityhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=36130"><a shape="rect" href="community.html">Community</a></h3><ul class="alternate"><li><a shape="rect" href="support.html">Support</a></li><li><a shape="rect" href="contributing.html">Contributing</a></li><li><a shape="rect" href="discussion-forums.html">Discussion Forums</a></li><li><a shape="rect" href="mailing-lists.html">Mailing Lists</a></li><li><a shape="rect" href="irc.html">IRC</a></li><li><a shape="rect" class="external-link" href="http://javabot.evanchooly.com/logs/%23apache-activemq/today" rel="nofollow">IRC Log</a></li><li><a shape="rect" href="security-advisories.html">Security Advisories</a></li><li><a shape="rect" href="site.html">Site</a></li><li><a shape="rect" class="external-link" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li><li><a shape="rect" href="projects-using-activemq.html">Projects Using ActiveMQ</a></li><li><a shape="rect" href="users.html">Users</a></li><li><a shape="rect" href="team.html">Team</a></li><li><a shape="rect" href="thanks.html">Thanks</a></li></ul><h3 id="Navigation-Featureshttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35883"><a shape="rect" href="features.html">Features</a></h3><ul class="alternate"><li><a shape="rect" href="advisory-message.html">Advisory Message</a></li><li><a shape="rect" href="clustering.html">Clustering</a></li><li><a shape="rect" href="cross-language-clients.html">Cross Language Clients</a></li><li><a shape="rect" href="enterprise-integration-patterns.html">Enterprise Integration Patterns</a></li><li><a shape="rect" href="jmx.html">JMX</a></li><li><a shape="rect" href="jms-to-jms-bridge.html">JMS to JMS Bridge</a></li><li><a shape="rect" href="masterslave.html">MasterSlave</a></li><li><a shape="rect" href="message-groups.html">Message Groups</a></li><li><a shape="rect" href="networks-of-brokers.html">Networks of Brokers</a></li><li><a shape="rect" href="performance.html">Performance</a></li><li><a shape="rect" href="persistence.html">Persistence</a></li><li><a shape="rect" href="security.html">Security</a></li><li><a shape="rect" href="virtual-destinations.html">Virtual Destinations</a></li><li><a shape="rect" href="visualisation.html">Visualisation</a></li><li><a shape="rect" href="features.html">More ...</a></li></ul><h3 id="Navigation-Connectivityhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=36167"><a shape="rect" href="connectivity.html">Connectivity</a></h3><ul class="alternate"><li><a shape="rect" href="ajax.html">Ajax</a></li><li><a shape="rect" href="amqp.html">AMQP</a></li><li><a shape="rect" href="axis-and-cxf-support.html">Axis and CXF Support</a></li><li><a shape="rect" href="c-integration.html">C Integration</a></li><li><a shape="rect" href="activemq-c-clients.html">C++</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/">C# and .Net Integration</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/cms/">CMS</a></li><li><a shape="rect" href="j2ee.html">J2EE</a></li><li><a shape="rect" href="jboss-integration.html">JBoss Integration</a></li><li><a shape="rect" class="external-link" href="http://docs.codehaus.org/display/JETTY/Integrating+with+ActiveMQ" rel="nofollow">Jetty</a></li><li><a shape="rect" href="jndi-support.html">JNDI Support</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/" title="NMS is the .Net Messaging API">NMS</a></li><li><a shape="rect" href="rest.html">REST</a></li><li><a shape="rect" href="rss-and-atom.html">RSS and Atom</a></li><li><a shape="rect" href="spring-support.html">Spring Support</a></li><li><a shape="rect" href="stomp.html">Stomp</a></li><li><a shape="rect" href="tomcat.html">Tomcat</a></li><li><a shape="rect" href="unix-service.html">Unix Service</a></li><li><a shape="rect" href="weblogic-integration.html">WebLogic Integration</a></li><li><a shape="rect" href="xmpp.html">XMPP</a></li><li><a shape="rect" href="connectivity.html">More ...</a></li></ul><h3 id="Navigation-UsingActiveMQ5https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71176"><a shape="rect" href="using-activemq-5.html">Using ActiveMQ 5</a></h3><ul class="alternate"><li><a shape="rect" href="version-5-getting-started.html">Getting Started</a></li><li><a shape="rect" href="version-5-initial-configuration.html">Initial Configuration</a></li><li><a shape="rect" href="version-5-run-broker.html">Running a Broker</a></li><li><a shape="rect" href="how-do-i-embed-a-broker-inside-a-connection.html">Embedded Brokers</a></li><li><a shape="rect" href="activemq-command-line-tools-reference.html">Command Line Tools</a></li><li><a shape="rect" href="configuring-version-5-transports.html">Configuring Transports</a></li><li><a shape="rect" href="version-5-examples.html">Examples</a></li><li><a shape="rect" href="version-5-web-samples.html">Web Samples</a></li><li><a shape="rect" href="how-can-i-monitor-activemq.html">Monitoring the Broker</a></li><li><a shape="rect" href="version-5-xml-configuration.html">Xml Configuration</a></li><li><a shape="rect" href="xml-reference.html">Xml Reference</a></li><li><a shape="rect" href="using-activemq-5.html">More ...</a></li></ul><h3 id="Navigation-Toolshttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35912"><a shape="rect" href="tools.html">Tools</a></h3><ul class="alternate"><li><a shape="rect" href="web-console.html">Web Console</a></li><li><a shape="rect" href="activemq-performance-module-users-manual.html">Maven2 Performance Plugin</a></li></ul><h3 id="Navigation-Supporthttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35919"><a shape="rect" href="support.html">Support</a></h3><ul class="alternate"><li><a shape="rect" class="external-link" href="http://issues.apache.org/jira/browse/AMQ">Issues</a></li><li><a shape="rect" class="external-link" href="http://issues.apache.org/activemq/browse/AMQ?report=com.atlassian.jira.plugin.system.project:roadmap-panel">Roadmap</a></li><li><a shape="rect" class="external-link" href="http://issues.apache.org/activemq/browse/AMQ?report=com.atlassian.jira.plugin.system.project:changelog-panel">Change log</a></li></ul><h3 id="Navigation-Developershttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35903"><a shape="rect" href="developers.html">Developers</a></h3><ul class="alternate"><li><a shape="rect" href="source.html">Source</a></li><li><a shape="rect" href="building.html">Building</a></li><li><a shape="rect" href="developer-guide.html">Developer Guide</a></li><li><a shape="rect" href="becoming-a-committer.html">Becoming a committer</a></li><li><a shape="rect" href="code-overview.html">Code Overview</a></li><li><a shape="rect" href="wire-protocol.html">Wire Protocol</a></li><li><a shape="rect" href="release-guide.html">Release Guide</a></li></ul><h3 id="Navigation-Tests">Tests</h3><ul class="alternate"><li><a shape="rect" href="activemq-performance-module-users-manual.html">Maven2 Performance Plugin</a></li><li><a shape="rect" href="benchmark-tests.html">Benchmark Tests</a></li><li><a shape="rect" href="jmeter-system-tests.html">JMeter System Tests</a></li><li><a shape="rect" href="jmeter-performance-tests.html">JMeter Performance Tests</a></li><li><a shape="rect" href="integration-tests.html">Integration Tests</a></li></ul><h3 id="Navigation-ProjectReports">Project Reports</h3><ul class="alternate"><li><a shape="rect" href="junit-reports.html">JUnit Reports</a></li><li><a shape="rect" href="source-xref.html">Source XRef</a></li><li><a shape="rect" href="test-source-xref.html">Test Source XRef</a></li><li><a shape="rect" href="xml-reference.html">Xml Reference</a></li></ul>
</div>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<div class="bottom_red_bar"></div>
</div>
</div>
</div>
</div>
<div class="black_box">
<div class="footer">
<div class="footer_l">
<div class="footer_r">
<div>
<a href="http://activemq.apache.org/privacy-policy.html">Privacy Policy</a> -
(<a href="https://cwiki.apache.org/confluence/pages/editpage.action?pageId=69705">edit this page</a>)
</div>
</div>
</div>
</div>
</div>
</div>
<div class="design_attribution">
&copy; 2004-2011 The Apache Software Foundation.
<br/>
Apache ActiveMQ, ActiveMQ, Apache, the Apache feather logo, and the Apache ActiveMQ project logo are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.
<br/>
<a href="http://hiramchirino.com">Graphic Design By Hiram</a>
</div>
<!-- delay the loading of large javascript files to the end so that they don't interfere with the loading of page content -->
<span style="display: none">
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-1347593-1");
pageTracker._initData();
pageTracker._trackPageview();
</script>
</span>
</body>
</html>