blob: 2dd076a10560777d801d727ab14b6cf235251760 [file] [log] [blame]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<head>
<link href="http://activemq.apache.org/styles/site.css" rel="stylesheet" type="text/css"/>
<link href="http://activemq.apache.org/styles/type-settings.css" rel="stylesheet" type="text/css"/>
<script src="http://activemq.apache.org/styles/prototype.js" type="text/javascript"></script>
<script src="http://activemq.apache.org/styles/rico.js" type="text/javascript"></script>
<script src="http://activemq.apache.org/styles/site.js" type="text/javascript"></script>
<style type="text/css">
.maincontent { overflow:hidden; }
</style>
<!--[if IE]>
<style type="text/css">
.maincontent { width:100%; }
</style>
<![endif]-->
<title>
Apache ActiveMQ &#8482; -- Durable Queue Memory Management
</title>
</head>
<body>
<div class="white_box">
<div class="header">
<div class="header_l">
<div class="header_r">
</div>
</div>
</div>
<div class="content">
<div class="content_l">
<div class="content_r">
<div>
<!-- Banner -->
<div id="asf_logo">
<div id="activemq_logo">
<a shape="rect" style="float:left; width:280px;display:block;text-indent:-5000px;text-decoration:none;line-height:60px; margin-top:10px; margin-left:100px;" href="http://activemq.apache.org" title="The most popular and powerful open source Message Broker">ActiveMQ</a>
<a shape="rect" style="float:right; width:210px;display:block;text-indent:-5000px;text-decoration:none;line-height:60px; margin-top:15px; margin-right:10px;" href="http://www.apache.org" title="The Apache Software Foundation">ASF</a>
</div>
</div>
<div class="top_red_bar">
<div id="site-breadcrumbs">
<a href="developers.html">Developers</a>&nbsp;&gt;&nbsp;<a href="developer-guide.html">Developer Guide</a>&nbsp;&gt;&nbsp;<a href="design-documents.html">Design Documents</a>&nbsp;&gt;&nbsp;<a href="durable-queue-memory-management.html">Durable Queue Memory Management</a>
</div>
<div id="site-quicklinks">
<p><a shape="rect" href="download.html">Download</a> | <a shape="rect" class="external-link" href="http://activemq.apache.org/maven/apidocs/index.html">JavaDocs</a> <a shape="rect" href="javadocs.html">More...</a> | <a shape="rect" href="source.html">Source</a> | <a shape="rect" href="discussion-forums.html">Forums</a> | <a shape="rect" href="support.html">Support</a></p>
</div>
</div>
<table border="0">
<tbody>
<tr>
<td valign="top" width="100%">
<div class="wiki-content maincontent">
<p>The current 3.x code can have issues with massive queues with un-acknowledged messages; its complex due to consumption from multiple points in the queue, consumers coming &amp; going, selectors matching only certain points in the queue etc.</p>
<p>This page outlines design the design proposal.</p>
<h2 id="DurableQueueMemoryManagement-TargetArea">TargetArea</h2>
<p>Maintains TargetLists for each consumer (the messages that have matched one or more consumers's selectors and are gonna be used to try dispatch stuff to the consumer DispatchArea). TargetEntry objects are added to the TargetList as messages arrive from the Queue. </p>
<p>A TargetEntry can have either a direct Message reference or a CacheReference to an entry in a cache. </p>
<h3 id="DurableQueueMemoryManagement-Eagerloading">Eager loading</h3>
<p>Each TargetList has a watermark to indicate the first TargetEntry which is a CacheReference. The idea is that under ideal scenarios, each TargetList has the watermark equal to the high water mark (i.e. that there are enough direct message references in RAM).</p>
<p>So we can have a background thread increasing the watermark up to the ideal level - which means it starts at the current watermark and directly loads the objects; de-referencing the CacheReference and replacing with direct references. </p>
<h3 id="DurableQueueMemoryManagement-PendingCache">PendingCache</h3>
<p>TargetEntry objects added to the TargetList which are above the high watermark have their Message objects evicted from direct references into the PendingCache.</p>
<p>The Target eager loader will move objects from this cache to the TargetList when they are available. If they are not in the PendingCache, they are loaded from the QueueArea.</p>
<h3 id="DurableQueueMemoryManagement-QueueArea">QueueArea</h3>
<p>This represents the actual queue messages. There's also a matching HeadQueueCache which is a simple memory bound queue. FIFO so newer stuff is evicted from the end.</p>
<p>Back </p>
<h3 id="DurableQueueMemoryManagement-LinkingtheCaches">Linking the Caches</h3>
<p>The PendingCache and HeadQueueCache are linked together, along with the TargetArea/DispatchArea so that there is a fixed amount of RAM used in all the caches.</p>
<p>As the PendingCache grows, the HeadQueueCache shrinks. HeadQueueCache is only used for new consumers arriving. On startup this will stick around a while until lots of messages start being delivered to consumers where it'll shrink back to zero if there is sufficient dispatching.</p>
<p>Under heavy load, the HeadQueueCache will be pretty much empty unless new consumers come along.</p></div>
</td>
<td valign="top">
<div class="navigation">
<div class="navigation_top">
<div class="navigation_bottom">
<h3 id="Navigation-Overviewhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35985"><a shape="rect" href="overview.html">Overview</a></h3><ul class="alternate"><li><a shape="rect" href="index.html">Index</a></li><li><a shape="rect" href="news.html">News</a></li><li><a shape="rect" href="new-features.html">New Features</a></li><li><a shape="rect" href="getting-started.html">Getting Started</a></li><li><a shape="rect" href="faq.html">FAQ</a></li><li><a shape="rect" href="articles.html">Articles</a></li><li><a shape="rect" href="books.html">Books</a></li><li><a shape="rect" href="download.html">Download</a></li><li><a shape="rect" class="external-link" href="http://www.apache.org/licenses/">License</a></li></ul><h3 id="Navigation-Search">Search</h3><div>
<form enctype="application/x-www-form-urlencoded" method="get" action="http://www.google.com/search" style="font-size: 10px;">
<input type="hidden" name="ie" value="UTF-8">
<input type="hidden" name="oe" value="UTF-8">
<input maxlength="255" type="text" name="q" size="15" value="value"><br clear="none">
<input type="submit" name="btnG" value="Search">
<input type="hidden" name="domains" value="activemq.apache.org">
<input type="hidden" name="sitesearch" value="activemq.apache.org">
</form>
</div>
<h3 id="Navigation-SubProjects">Sub Projects</h3><ul class="alternate"><li><a shape="rect" class="external-link" href="http://activemq.apache.org/artemis/">Artemis</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/apollo" title="ActiveMQ Apollo">Apollo</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/cms/">CMS</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/" title="NMS is the .Net Messaging API">NMS</a></li></ul><h3 id="Navigation-Communityhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=36130"><a shape="rect" href="community.html">Community</a></h3><ul class="alternate"><li><a shape="rect" href="support.html">Support</a></li><li><a shape="rect" href="contributing.html">Contributing</a></li><li><a shape="rect" href="discussion-forums.html">Discussion Forums</a></li><li><a shape="rect" href="mailing-lists.html">Mailing Lists</a></li><li><a shape="rect" href="irc.html">IRC</a></li><li><a shape="rect" class="external-link" href="http://javabot.evanchooly.com/logs/%23apache-activemq/today" rel="nofollow">IRC Log</a></li><li><a shape="rect" href="security-advisories.html">Security Advisories</a></li><li><a shape="rect" href="site.html">Site</a></li><li><a shape="rect" class="external-link" href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li><li><a shape="rect" href="projects-using-activemq.html">Projects Using ActiveMQ</a></li><li><a shape="rect" href="users.html">Users</a></li><li><a shape="rect" href="team.html">Team</a></li><li><a shape="rect" href="thanks.html">Thanks</a></li></ul><h3 id="Navigation-Featureshttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35883"><a shape="rect" href="features.html">Features</a></h3><ul class="alternate"><li><a shape="rect" href="advisory-message.html">Advisory Message</a></li><li><a shape="rect" href="clustering.html">Clustering</a></li><li><a shape="rect" href="cross-language-clients.html">Cross Language Clients</a></li><li><a shape="rect" href="enterprise-integration-patterns.html">Enterprise Integration Patterns</a></li><li><a shape="rect" href="jmx.html">JMX</a></li><li><a shape="rect" href="jms-to-jms-bridge.html">JMS to JMS Bridge</a></li><li><a shape="rect" href="masterslave.html">MasterSlave</a></li><li><a shape="rect" href="message-groups.html">Message Groups</a></li><li><a shape="rect" href="networks-of-brokers.html">Networks of Brokers</a></li><li><a shape="rect" href="performance.html">Performance</a></li><li><a shape="rect" href="persistence.html">Persistence</a></li><li><a shape="rect" href="security.html">Security</a></li><li><a shape="rect" href="virtual-destinations.html">Virtual Destinations</a></li><li><a shape="rect" href="visualisation.html">Visualisation</a></li><li><a shape="rect" href="features.html">More ...</a></li></ul><h3 id="Navigation-Connectivityhttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=36167"><a shape="rect" href="connectivity.html">Connectivity</a></h3><ul class="alternate"><li><a shape="rect" href="ajax.html">Ajax</a></li><li><a shape="rect" href="amqp.html">AMQP</a></li><li><a shape="rect" href="axis-and-cxf-support.html">Axis and CXF Support</a></li><li><a shape="rect" href="c-integration.html">C Integration</a></li><li><a shape="rect" href="activemq-c-clients.html">C++</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/">C# and .Net Integration</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/cms/">CMS</a></li><li><a shape="rect" href="j2ee.html">J2EE</a></li><li><a shape="rect" href="jboss-integration.html">JBoss Integration</a></li><li><a shape="rect" class="external-link" href="http://docs.codehaus.org/display/JETTY/Integrating+with+ActiveMQ" rel="nofollow">Jetty</a></li><li><a shape="rect" href="jndi-support.html">JNDI Support</a></li><li><a shape="rect" class="external-link" href="http://activemq.apache.org/nms/" title="NMS is the .Net Messaging API">NMS</a></li><li><a shape="rect" href="rest.html">REST</a></li><li><a shape="rect" href="rss-and-atom.html">RSS and Atom</a></li><li><a shape="rect" href="spring-support.html">Spring Support</a></li><li><a shape="rect" href="stomp.html">Stomp</a></li><li><a shape="rect" href="tomcat.html">Tomcat</a></li><li><a shape="rect" href="unix-service.html">Unix Service</a></li><li><a shape="rect" href="weblogic-integration.html">WebLogic Integration</a></li><li><a shape="rect" href="xmpp.html">XMPP</a></li><li><a shape="rect" href="connectivity.html">More ...</a></li></ul><h3 id="Navigation-UsingActiveMQ5https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=71176"><a shape="rect" href="using-activemq-5.html">Using ActiveMQ 5</a></h3><ul class="alternate"><li><a shape="rect" href="version-5-getting-started.html">Getting Started</a></li><li><a shape="rect" href="version-5-initial-configuration.html">Initial Configuration</a></li><li><a shape="rect" href="version-5-run-broker.html">Running a Broker</a></li><li><a shape="rect" href="how-do-i-embed-a-broker-inside-a-connection.html">Embedded Brokers</a></li><li><a shape="rect" href="activemq-command-line-tools-reference.html">Command Line Tools</a></li><li><a shape="rect" href="configuring-version-5-transports.html">Configuring Transports</a></li><li><a shape="rect" href="version-5-examples.html">Examples</a></li><li><a shape="rect" href="version-5-web-samples.html">Web Samples</a></li><li><a shape="rect" href="how-can-i-monitor-activemq.html">Monitoring the Broker</a></li><li><a shape="rect" href="version-5-xml-configuration.html">Xml Configuration</a></li><li><a shape="rect" href="xml-reference.html">Xml Reference</a></li><li><a shape="rect" href="using-activemq-5.html">More ...</a></li></ul><h3 id="Navigation-Toolshttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35912"><a shape="rect" href="tools.html">Tools</a></h3><ul class="alternate"><li><a shape="rect" href="web-console.html">Web Console</a></li><li><a shape="rect" href="activemq-performance-module-users-manual.html">Maven2 Performance Plugin</a></li></ul><h3 id="Navigation-Supporthttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35919"><a shape="rect" href="support.html">Support</a></h3><ul class="alternate"><li><a shape="rect" class="external-link" href="http://issues.apache.org/jira/browse/AMQ">Issues</a></li><li><a shape="rect" class="external-link" href="http://issues.apache.org/activemq/browse/AMQ?report=com.atlassian.jira.plugin.system.project:roadmap-panel">Roadmap</a></li><li><a shape="rect" class="external-link" href="http://issues.apache.org/activemq/browse/AMQ?report=com.atlassian.jira.plugin.system.project:changelog-panel">Change log</a></li></ul><h3 id="Navigation-Developershttps://cwiki.apache.org/confluence/pages/viewpage.action?pageId=35903"><a shape="rect" href="developers.html">Developers</a></h3><ul class="alternate"><li><a shape="rect" href="source.html">Source</a></li><li><a shape="rect" href="building.html">Building</a></li><li><a shape="rect" href="developer-guide.html">Developer Guide</a></li><li><a shape="rect" href="becoming-a-committer.html">Becoming a committer</a></li><li><a shape="rect" href="code-overview.html">Code Overview</a></li><li><a shape="rect" href="wire-protocol.html">Wire Protocol</a></li><li><a shape="rect" href="release-guide.html">Release Guide</a></li></ul><h3 id="Navigation-Tests">Tests</h3><ul class="alternate"><li><a shape="rect" href="activemq-performance-module-users-manual.html">Maven2 Performance Plugin</a></li><li><a shape="rect" href="benchmark-tests.html">Benchmark Tests</a></li><li><a shape="rect" href="jmeter-system-tests.html">JMeter System Tests</a></li><li><a shape="rect" href="jmeter-performance-tests.html">JMeter Performance Tests</a></li><li><a shape="rect" href="integration-tests.html">Integration Tests</a></li></ul><h3 id="Navigation-ProjectReports">Project Reports</h3><ul class="alternate"><li><a shape="rect" href="junit-reports.html">JUnit Reports</a></li><li><a shape="rect" href="source-xref.html">Source XRef</a></li><li><a shape="rect" href="test-source-xref.html">Test Source XRef</a></li><li><a shape="rect" href="xml-reference.html">Xml Reference</a></li></ul>
</div>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<div class="bottom_red_bar"></div>
</div>
</div>
</div>
</div>
<div class="black_box">
<div class="footer">
<div class="footer_l">
<div class="footer_r">
<div>
<a href="http://activemq.apache.org/privacy-policy.html">Privacy Policy</a> -
(<a href="https://cwiki.apache.org/confluence/pages/editpage.action?pageId=35935">edit this page</a>)
</div>
</div>
</div>
</div>
</div>
</div>
<div class="design_attribution">
&copy; 2004-2011 The Apache Software Foundation.
<br/>
Apache ActiveMQ, ActiveMQ, Apache, the Apache feather logo, and the Apache ActiveMQ project logo are trademarks of The Apache Software Foundation. All other marks mentioned may be trademarks or registered trademarks of their respective owners.
<br/>
<a href="http://hiramchirino.com">Graphic Design By Hiram</a>
</div>
<!-- delay the loading of large javascript files to the end so that they don't interfere with the loading of page content -->
<span style="display: none">
<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-1347593-1");
pageTracker._initData();
pageTracker._trackPageview();
</script>
</span>
</body>
</html>