how-do-i-configure-10s-of-1000s-of-queues-in-a-single-broker.html

<div class="wiki-content maincontent"><p>Scaling to tens of thousands of Queues in a single broker is relatively straightforward - but requires some configuration changes from the default.</p><h2 id="HowdoIconfigure10sof1000sofQueuesinasinglebroker-ReducingThreads">Reducing Threads</h2><p>With the default configuration, ActiveMQ is configured to use a dispatch thread per Queue - you can use set the optimizedDispatch property on the destination policy entry - see <a shape="rect" href="per-destination-policies.html">configuring Queues</a>.</p><p>ActiveMQ can optionally use internally a thread pool to control dispatching of messages - but as a lot of deployment operating systems are good at handling a large number of threads, this is off by default. To enable this option, either set the ACTIVEMQ_OPTS to disable dedicated task runners in the start up script, INSTALL_DIR/bin/activemq -e.g.</p><div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent pdl">
<pre class="brush: java; gutter: false; theme: Default" style="font-size:12px;">ACTIVEMQ_OPTS="-Xmx512M -Dorg.apache.activemq.UseDedicatedTaskRunner=false"  
</pre>
</div></div><p>or you can set ACTIVEMQ_OPTS in /etc/activemq.conf.</p><p><strong>Note:</strong> From ActiveMQ 5.6 onwards the dedicated task runner is disabled by default (see <style>
    .jira-issue {
        padding: 0 0 0 2px;
        line-height: 20px;
    }

    .jira-issue img {
        padding-right: 5px;
    }
    .jira-issue .aui-lozenge {
        line-height: 18px;
        vertical-align: top;
    }

    .jira-issue .icon {
        background-position: left center;
        background-repeat: no-repeat;
        display: inline-block;
        font-size: 0;
        max-height: 16px;
        text-align: left;
        text-indent: -9999em;
        vertical-align: text-bottom;
    }
</style>

    <span class="jira-issue AMQ-3667"><a shape="rect" class="issue-link" href="https://issues.apache.org/jira/browse/AMQ-3667?src=confmacro">AMQ-3667</a></span>
).</p><p>To reduce the number of threads used for the transport - take a look at using the NIO transport - see <a shape="rect" href="configuring-transports.html">Configuring Transports</a></p><p>Here is an <a shape="rect" class="external-link" href="http://svn.apache.org/repos/asf/activemq/trunk/assembly/src/sample-conf/activemq-scalability.xml">example</a> of this in use in one of the provided sample broker configuration files.</p><h2 id="HowdoIconfigure10sof1000sofQueuesinasinglebroker-ReducingMemoryConsumption">Reducing Memory Consumption</h2><p>Reduce the memory used per thread - see <a shape="rect" href="javalangoutofmemory.html">reducing memory consumption</a></p><h2 id="HowdoIconfigure10sof1000sofQueuesinasinglebroker-Reducenumberoffiledescriptors">Reduce number of file descriptors</h2><p>ActiveMQ uses the amqPersistenceAdapter by default for persistent messages. Unfortunately, this persistence adapter (as well as the kahaPersistenceAdapter) opens a file descriptor for each queue. When creating large numbers of queues, you'll quickly run into the limit for your OS.</p><p>You can either choose another <a shape="rect" href="persistence.html">persistence option</a></p><p>or - try out the new <a shape="rect" href="kahadb.html">KahaDB</a> in version 5.3 and higher</p><h2 id="HowdoIconfigure10sof1000sofQueuesinasinglebroker-Increasethelimitonfiledescriptorsperprocess">Increase the limit on file descriptors per process</h2><p>Try <a shape="rect" class="external-link" href="http://tinyurl.com/o9qs2f" rel="nofollow">googling for the OS you are using</a></p></div>