| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta http-equiv="X-UA-Compatible" content="ie=edge"> |
| <title>ActiveMQ</title> |
| <link rel="icon" type="image/png" href="/assets/img/favicon.png"> |
| |
| <link rel="stylesheet" href="/css/main.css"> |
| <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js" integrity="sha384-SlE991lGASHoBfWbelyBPLsUlwY1GwNDJo3jSJO04KZ33K2bwfV9YBauFfnzvynJ" crossorigin="anonymous"></script> |
| <script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script> |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script> |
| <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script> |
| </head> |
| |
| <body> |
| <nav class="navbar navbar-expand-lg navbar-light fixed-top"> |
| <div class="container"> |
| <!-- <a class="navbar-brand mr-auto" href="#"><img style="height: 50px" src="assets/img/apache-feather.png" /></a> --> |
| <a class="navbar-brand mr-auto" href="/"><img src="/assets/img/activemq_logo_black_small.png" style="height: 50px"/></a> |
| <button class="navbar-toggler ml-auto" type="button" data-toggle="collapse" data-target="#navbarContent" aria-controls="navbarContent" aria-expanded="false" aria-label="Toggle navigation"> |
| <span class="navbar-toggler-icon"></span> |
| </button> |
| |
| <div class="ml-auto collapse navbar-collapse" id="navbarContent"> |
| <ul class="navbar-nav ml-auto"> |
| <li class="nav-item"> |
| <a class="nav-link active" href="/index.html">Home</a> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link" id="navbarDropdownComponents" data-target="#" href="" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Components</a> |
| <ul class="dropdown-menu dropdown-menu-center" aria-labelledby="navbarDropdownComponents"> |
| <div class="row"> |
| <div class="col-12"> |
| <ul class="multi-column-dropdown"> |
| <li class="nav-item"><a class="dropdown-item" href="/components/classic">ActiveMQ 5</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/components/artemis/">ActiveMQ Artemis</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/components/nms">NMS Clients</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/components/cms">CMS Client</a></li> |
| </ul> |
| </div> |
| </div> |
| </ul> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link" id="navbarDropdownCommunity" data-target="#" href="" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Contact</a> |
| <ul class="dropdown-menu dropdown-menu-center multi-column columns-1" aria-labelledby="navbarDropdownCommunity"> |
| <div class="row"> |
| <div class="col-12"> |
| <ul class="multi-column-dropdown"> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#mailing">Mailing Lists</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#chat">Chat</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#issues">Report Issues</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#contributing">Contributing</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/security-advisories.html">Security</a></li> |
| </ul> |
| </div> |
| </div> |
| </ul> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link" id="navbarDropdownTeam" data-target="#" href="" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a> |
| <ul class="dropdown-menu dropdown-menu-center multi-column columns-1" aria-labelledby="navbarDropdownTeam"> |
| <div class="row"> |
| <div class="col-sm-12"> |
| <ul class="multi-column-dropdown"> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org">The Apache Software Foundation</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/licenses/">License</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/security-advisories.html">Security</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/events/current-event">Events</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://people.apache.org/phonebook.html?pmc=activemq">PMC & Committers</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/team/reports">Board Reports</a></li> |
| </ul> |
| </div> |
| </div> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="content"> |
| <div class="page-title-activemq5"> |
| <div class="container"> |
| <h1>Producer Flow Control</h1> |
| </div> |
| </div> |
| <div class="container" > |
| <div class="row" style="margin-top: 30px"> |
| <div class="col-12 activemq5"> |
| <p><a href="features">Features</a> > <a href="message-dispatching-features">Message Dispatching Features</a> > <a href="producer-flow-control">Producer Flow Control</a></p> |
| |
| <h2 id="producer-flow-control">Producer Flow Control</h2> |
| |
| <p>In ActiveMQ 4.x flow control was implemented using TCP flow control. The underlying network connection of throttled consumers was suspended to enforce flow control limits. This strategy is very efficient but can lead to deadlocks if there are multiple producers and consumers sharing the same connection.</p> |
| |
| <p>As of ActiveMQ 5.0, we can now individually flow control each producer on a shared connection without having to suspend the entire connection. By ‘flow control’ we mean that if the broker detects that the memory limit for the destination, or the temp- or file-store limits for the broker, have been exceeded, then the flow of messages can be slowed down. The producer will be either blocked until resources are available <em>or</em> will receive a JMSException: this behaviour is configurable and described in the section below on <code class="highlighter-rouge"><systemUsage></code>.</p> |
| |
| <p>It’s worth noting that the default <code class="highlighter-rouge"><systemUsage></code> settings will cause the producer to <em>block</em> when the <code class="highlighter-rouge">memoryLimit</code> or <code class="highlighter-rouge"><systemUsage></code> limits are reached: this blocking behaviour is sometimes misinterpreted as a ‘hung producer’, when in fact the producer is simply diligently waiting until space is available.</p> |
| |
| <ul> |
| <li> |
| <p>Messages that are sent synchronously will automatically use per producer flow control; this applies generally to persistent messages which are sent synchronously <em>unless</em> you enable the <code class="highlighter-rouge">useAsyncSend</code> flag.</p> |
| </li> |
| <li> |
| <p>Producers that use <a href="async-sends">Async Sends</a> - generally speaking, producers of non-persistent messages - don’t bother waiting for any acknowledgement from the broker; so, if a memory limit has been exceeded, you will <em>not</em> get notfied. If you do want to be aware of broker limits being exceeded, you will need to configure the ProducerWindowSize connection option so that even async messages are flow controlled per producer.</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ActiveMQConnectionFactory connctionFactory = connctionFactory.setProducerWindowSize(1024000); |
| </code></pre></div> </div> |
| <p>The ProducerWindowSize is the maximum number of bytes of data that a producer will transmit to a broker before waiting for acknowledgment messages from the broker that it has accepted the previously sent messages.</p> |
| </li> |
| </ul> |
| |
| <p>Alternatively, if you’re sending non-persisted messages (which are by default sent async), and want to be informed if the queue or topic’s memory limit has been breached, then you can simply configure the connection factory to ‘alwaysSyncSend’. While this is going to be slower, it will ensure that your message producer is informed immediately of memory issues.</p> |
| |
| <p>If you like, you can disable flow control for specific JMS queues and topics on the broker by setting the <code class="highlighter-rouge">producerFlowControl</code> flag to false on the appropriate destination policy in the Broker configuration - e.g.</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><destinationPolicy> |
| <policyMap> |
| <policyEntries> |
| <policyEntry topic="FOO.>" producerFlowControl="false"/> |
| </policyEntries> |
| </policyMap> |
| </destinationPolicy> |
| </code></pre></div></div> |
| <p>see <a href="xml-configuration">Broker Configuration</a>.</p> |
| |
| <p>Note that, since the introduction of the new file cursor in ActiveMQ 5.x, non-persisted messages are shunted into the temporary file store to reduce the amount of memory used for non-persistent messaging. As a result, you may find that a queue’s memoryLimit is never reached, as the cursor doesn’t use very much memory. If you really do want to keep all your non-persistent messages in memory, and stop producers when the limit is reached, you should configure the <code class="highlighter-rouge"><vmQueueCursor></code>.</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb"> |
| <pendingQueuePolicy> |
| <vmQueueCursor/> |
| </pendingQueuePolicy> |
| </policyEntry> |
| </code></pre></div></div> |
| <p>The fragment above will ensure that all non-persistent queue messages are kept in memory, with each queue having a limit of 1Mb.</p> |
| |
| <h3 id="how-producer-flow-control-works">How Producer Flow Control works</h3> |
| |
| <p>If you are sending a persistent message (so that a response of the <a href="openwire">OpenWire</a> Message is expected then the broker will send the producer a <a href="http://activemq.apache.org/maven/5.9.0/apidocs/index.html">ProducerAck</a> message. This informs the producer that the previous sending window has been processed, so that it can now send another window. Its kinda like consumer acks but in reverse.</p> |
| |
| <h4 id="advantage">Advantage</h4> |
| |
| <p>So a nice producer might wait for a producer ack before sending more data, to avoid flooding the broker (and forcing the broker to block the entire connection if a slow consumer occurs). To see how this works in source code, check out the <a href="http://activemq.apache.org/maven/5.9.0/apidocs/index.html">ActiveMQMessageProducer</a> code.</p> |
| |
| <p>Though a client can ignore the producer ACKs altogether and the broker should just stall the transport if it has to for slow consumer handling; though this does mean it’ll stall the entire connection.</p> |
| |
| <h3 id="configure-client-side-exceptions">Configure Client-Side Exceptions</h3> |
| |
| <p>An alternative to the indefinite blocking of the <code class="highlighter-rouge">send()</code> operation when no space is free on the broker is to instead configure that an exception to be thrown on the client-side. By configuring the <code class="highlighter-rouge">sendFailIfNoSpace</code> property to <code class="highlighter-rouge">true</code>, the broker will cause the <code class="highlighter-rouge">send()</code> operation to fail with a <code class="highlighter-rouge">javax.jms.ResourceAllocationException</code>, which will propagate to the client. Below is an example of this configuration:</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><systemUsage> |
| <systemUsage sendFailIfNoSpace="true"> |
| <memoryUsage> |
| <memoryUsage limit="20 mb"/> |
| </memoryUsage> |
| </systemUsage> |
| </systemUsage> |
| </code></pre></div></div> |
| <p>The advantage of this property is that the client can catch the <code class="highlighter-rouge">javax.jms.ResourceAllocationException</code>, wait a bit and retry the <code class="highlighter-rouge">send()</code> operation instead of just hanging indefinitely.</p> |
| |
| <p>Starting in version 5.3.1 the <code class="highlighter-rouge">sendFailIfNoSpaceAfterTimeout</code> property has been added. This property causes the <code class="highlighter-rouge">send()</code> operation to fail with an exception on the client-side, but only after waiting the given amount of time. If space on the broker is still not freed after the configured amount of time, only then does the <code class="highlighter-rouge">send()</code> operation fail with an exception to the client-side. Below is an example:</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><systemUsage> |
| <systemUsage sendFailIfNoSpaceAfterTimeout="3000"> |
| <memoryUsage> |
| <memoryUsage limit="20 mb"/> |
| </memoryUsage> |
| </systemUsage> |
| </systemUsage> |
| </code></pre></div></div> |
| <p>The timeout is defined in milliseconds so the example above waits for three seconds before failing the <code class="highlighter-rouge">send()</code> operation with an exception to the client-side. The advantage of this property is that it will block for the configured amount of time instead of failing immediately or blocking indefinitely. This property offers not only an improvement on the broker-side, but also an improvement for the client so it can catch the exception, wait a bit and retry the <code class="highlighter-rouge">send()</code> operation.</p> |
| |
| <p>Starting in version 5.16.0 the <code class="highlighter-rouge">sendFailIfNoSpace</code> and <code class="highlighter-rouge">sendFailIfNoSpaceAfterTimeout</code> can be configured on a per destination basis via <a href="per-destination-policies">destination policies</a>.</p> |
| |
| <h2 id="disabling-flow-control">Disabling Flow Control</h2> |
| |
| <p>A common requirement is to disable flow control so that message dispatching continues until all available disk is used up by pending messages (whether persistent or non persistent messaging is configured). To do this enable <a href="message-cursors">Message Cursors</a>.</p> |
| |
| <h2 id="system-usage">System usage</h2> |
| |
| <p>You can also slow down producers via some attributes on the <code class="highlighter-rouge"><systemUsage></code> element. Take a look at the following example:</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><systemUsage> |
| <systemUsage> |
| <memoryUsage> |
| <memoryUsage limit="64 mb" /> |
| </memoryUsage> |
| <storeUsage> |
| <storeUsage limit="100 gb" /> |
| </storeUsage> |
| <tempUsage> |
| <tempUsage limit="10 gb" /> |
| </tempUsage> |
| </systemUsage> |
| </systemUsage> |
| </code></pre></div></div> |
| <p>You can set limits of memory for <code class="highlighter-rouge">NON_PERSISTENT</code> messages, disk storage for <code class="highlighter-rouge">PERSITENT</code> messages and total usage for temporary messages, the broker will use before it slowdown producers. <em>Using the default settings shown above, the broker will block the <code class="highlighter-rouge">send()</code> call until some messages are consumed and space becomes available on the broker.</em> The default values are shown above, you will probably need to increase these values for your environment.</p> |
| |
| <p><strong>PercentUsage</strong></p> |
| |
| <p>Both StoreUsage and TempUsage support a percentLimit attribute where the limit is determined as a percentage of the total available. From version 5.15.x there is an additional related total attribute that can be used to explicitly set the total available such that the file system is not queried. This is useful in the case that only part of a disk partition is available to the broker or where the underlying file store reports > Long.MAX_VALUE available capacity (e.g: EFS) which will overflow the long return value of java.io.File#getTotalSpace. Note that when a total is specified, that actual data available will not be validated agains the file system, just the store usage relative to that absolute total.</p> |
| |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="row sitemap"> |
| <div class="col-sm-12"> |
| <div class="container"> |
| <div class="row"> |
| <div class="col-sm-12"> |
| <div class="row"> |
| <div class="col-sm-3"> |
| <div > |
| <img class="float-left" style="max-height: 100px" src="/assets/img/activemq_logo_white_vertical_small.png"/> |
| </div> |
| </div> |
| <div style="text-align: center; margin-bottom: 0px; margin-top: 30px; font-size: 65%" class="col-sm-6"> |
| <p>Apache ActiveMQ, ActiveMQ, ActiveMQ Artemis, Apache, the Apache feather logo, and the Apache ActiveMQ project logo are trademarks of The Apache Software Foundation. Copyright © 2019, The Apache Software Foundation. Licensed under <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License 2.0</a>.</p> |
| </div> |
| <div class="col-sm-3"> |
| <div > |
| <a href="https://www.apache.org"><img class="float-right" style="margin-top: 10px; max-height: 80px" src="/assets/img/apache-logo-small.png"/></a> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| </body> |
| </html> |
