<div class="docbook"><div class="navheader"><table summary="Navigation header" width="100%"><tr><th align="center" colspan="3">9.11. Memory</th></tr><tr><td align="left" width="20%"><a accesskey="p" href="Java-Broker-Runtime-Connection-Limit.html">Prev</a> </td><th align="center" width="60%">Chapter 9. Runtime</th><td align="right" width="20%"> <a accesskey="n" href="Java-Broker-High-Availability.html">Next</a></td></tr></table><hr /></div><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="Java-Broker-Runtime-Memory"></a>9.11. Memory</h2></div></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Introduction"></a>9.11.1. Introduction</h3></div></div></div><p> | |
Understanding how the Qpid broker uses memory is essential to running a high performing and reliable service. | |
A wrongly configured broker can exhibit poor performance or even crash with an <code class="literal">OutOfMemoryError</code>. | |
Unfortunately, memory usage is not a simple topic and thus requires some in depth explanations. | |
This page should give the required background information to make informed decisions on how to configure your broker. | |
</p><p> | |
<a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Types" title="9.11.2. Types of Memory">Section 9.11.2, “Types of Memory”</a> explains the two different kinds of Java memory most relevant to the broker. | |
<a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Usage" title="9.11.3. Memory Usage in the Broker">Section 9.11.3, “Memory Usage in the Broker”</a> goes on to explain which parts of the broker use what kind of memory. | |
<a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory" title="9.11.4. Low Memory Conditions">Section 9.11.4, “Low Memory Conditions”</a> explains what happens when the system runs low on memory. | |
<a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Defaults" title="9.11.5. Defaults">Section 9.11.5, “Defaults”</a> lays out the default settings of the Qpid broker. | |
Finally, <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Tuning" title="9.11.6. Memory Tuning the Broker">Section 9.11.6, “Memory Tuning the Broker”</a> gives some advice on tuning your broker. | |
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Types"></a>9.11.2. Types of Memory</h3></div></div></div><p> | |
While Java has a couple of different internal memory types we will focus on the two types that are relevant to the Qpid broker. | |
Both of these memory types are taken from the same physical memory (RAM). | |
</p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6308"></a>9.11.2.1. Heap</h4></div></div></div><p> | |
Normally, all objects are allocated from Java's heap memory. | |
Once, nothing references an object it is cleaned up by the Java Garbage Collector and it's memory returned to the heap. | |
This works fine for most use cases. | |
However, when interacting with other parts of the operating system using Java's heap is not ideal. | |
This is where the so called direct memory comes into play. | |
</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6313"></a>9.11.2.2. Direct</h4></div></div></div><p> | |
The world outside of the JVM, in particular the operating system (OS), does not know about Java heap memory and uses other structures like C arrays. | |
In order to interact with these systems Java needs to copy data between its own heap memory and these native structures. | |
This can become a bottle neck when there is a lot of exchange between Java and the OS like in I/O (both disk and network) heavy applications. | |
Java's solution to this is to allow programmers to request <code class="literal">ByteBuffer</code>s from so called direct memory. | |
This is an opaque structure that <span class="emphasis"><em>might</em></span> have an underlying implementation that makes it efficient to interact with the OS. | |
Unfortunately, the GC is not good at tracking direct memory and in general it is inadvisable to use direct memory for regular objects. | |
</p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Usage"></a>9.11.3. Memory Usage in the Broker</h3></div></div></div><p> | |
This section lists some note worthy users of memory within the broker and where possible lists their usage of heap and direct memory. | |
Note that to ensure smooth performance some heap memory should remain unused by the application and be reserved for the JVM to do house keeping and garbage collection. | |
<a class="link" href="https://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/util/DbCacheSize.html" target="_top">Some guides</a> advise to reserve up to 30% of heap memory for the JVM. | |
</p><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6332"></a>9.11.3.1. Broker</h4></div></div></div><p> | |
The broker itself uses a moderate amount of heap memory (≈15 MB). | |
However, each connection and session comes with a heap overhead of about 17 kB and 15 kB respectively. | |
In addition, each connection reserves 512 kB direct memory for network I/O. | |
</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6337"></a>9.11.3.2. Virtual Hosts</h4></div></div></div><p> | |
The amount of memory a Virtual Host uses depends on its type. | |
For a JSON Virtual Host Node with a BDB Virtual Host the heap memory usage is approximately 2 MB. | |
However, each BDB Virtual Hosts has a mandatory cache in heap memory which has an impact on performance. | |
See <a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Usage-BDB" title="9.11.3.4. Message Store">below</a> for more information. | |
</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6345"></a>9.11.3.3. Messages</h4></div></div></div><p> | |
Messages and their headers are kept in direct memory and have an additional overhead of approximately 1 kB heap memory each. | |
This means that most brokers will want to have more direct memory than heap memory. | |
When many small messages accumulate on the broker the 1 kB heap memory overhead can become a <a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory-Heap" title="9.11.4.1. Low on Heap Memory">limiting factor</a>. | |
</p><p> | |
When the broker is <a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory-Direct" title="9.11.4.2. Low on Direct Memory">running low on direct memory</a> | |
it will evict enqueued messages from memory and <a class="link" href="Java-Broker-Runtime-Flow-To-Disk.html" title="9.6. Flow to Disk">flow them to disk</a>. | |
For persistent messages this only means freeing the direct memory representation because they always have an on-disk representation to guard against unexpected failure (e.g., a power cut). | |
For transient messages this implies additional disk I/O. | |
After being flowed to disk messages need to be re-read from disk before delivery. | |
</p><p>Please, note that messages from uncommitted transactions are not | |
<a class="link" href="Java-Broker-Runtime-Flow-To-Disk.html" title="9.6. Flow to Disk">flowed to disk</a> as part of | |
<a class="link" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory-Direct" title="9.11.4.2. Low on Direct Memory">running into low direct memory conditions</a>, | |
as they are not enqueued yet. The <code class="literal">Connection</code> has its own threshold for | |
keeping messages from uncommitted transactions in memory. Only when <code class="literal">Connection</code> threshold | |
is breached, the uncommitted messages on the connection are | |
<a class="link" href="Java-Broker-Runtime-Flow-To-Disk.html" title="9.6. Flow to Disk">flowed to disk</a>.</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Memory-Usage-BDB"></a>9.11.3.4. Message Store</h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e6381"></a>Berkeley DB (BDB)</h5></div></div></div><p> | |
The broker can use Oracle's BDB JE (BDB) as a message store to persist messages by writing them to a database. | |
BDB uses a mandatory cache for navigating and organising its database structure. | |
Sizing and tuning this cache is a topic of its own and would go beyond the scope of this guide. | |
Suffice to say that by default Qpid uses 5% of heap memory for BDB caches (each Virtual Host uses a separate cache) or 10 MB per BDB store, whichever is greater. | |
See the <a class="link" href="http://www.oracle.com/us/products/database/berkeley-db/je" target="_top">official webpage</a> especially <a class="link" href="http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/util/DbCacheSize.html" target="_top">this page</a> for more information. | |
For those interested, Qpid uses <a class="link" href="http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/CacheMode.html#EVICT_LN" target="_top">EVICT_LN</a> as its default JE cacheMode. | |
</p><p> | |
Note that due to licensing concerns Qpid does not ship the BDB JE jar files. | |
</p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e6397"></a>Derby</h5></div></div></div><p> | |
TODO | |
</p></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6402"></a>9.11.3.5. HTTP Management</h4></div></div></div><p> | |
Qpid uses Jetty for the HTTP Management (both REST and Web Management Console). | |
When the management plugin is loaded it will allocate the memory it needs and should not require more memory during operation and can thus be largely ignored. | |
</p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Low-Memory"></a>9.11.4. Low Memory Conditions</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Memory-Low-Memory-Heap"></a>9.11.4.1. Low on Heap Memory</h4></div></div></div><p> | |
When the broker runs low on heap memory performance will degrade because the JVM will trigger full garbage collection (GC) events in a struggle to free memory. | |
These full GC events are also called stop-the-world events as they completely halt the execution of the Java application. | |
Stop-the-world-events may take any where from a couple of milliseconds up to several minutes. | |
Should the heap memory demands rise even further the JVM will eventually throw an OutOfMemoryError which will cause the broker to shut down. | |
</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="Java-Broker-Runtime-Memory-Low-Memory-Direct"></a>9.11.4.2. Low on Direct Memory</h4></div></div></div><p> | |
When the broker detects that it uses 75% of available direct memory it will start flowing incoming transient messages to disk and reading them back before delivery. | |
This will prevent the broker from running out of direct memory but may degrade performance by requiring disk I/O. | |
</p></div></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Defaults"></a>9.11.5. Defaults</h3></div></div></div><p> | |
By default Qpid uses these settings: | |
</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> | |
0.5 GB heap memory | |
</li><li class="listitem"> | |
1.5 GB direct memory | |
</li><li class="listitem"> | |
5% of heap reserved for the BDB JE cache. | |
</li><li class="listitem"> | |
Start flow-to-disk at 75% direct memory utilisation. | |
</li></ul></div><p> | |
As an example, this would accommodate a broker with 50 connections, each serving 5 sessions, and each session having 1000 messages of 1 kB on queues in the broker. | |
This means a total of 250 concurrent sessions and a total of 250000 messages without flowing messages to disk. | |
</p></div><div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="Java-Broker-Runtime-Memory-Tuning"></a>9.11.6. Memory Tuning the Broker</h3></div></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6438"></a>9.11.6.1. Java Tuning</h4></div></div></div><p> | |
Most of these options are implementation specific. It is assumed you are using Oracle Java 1.8. | |
</p><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> | |
Heap and direct memory can be configured through the <a class="link" href="Java-Broker-Appendix-Environment-Variables.html#Java-Broker-Appendix-Environment-Variables-Qpid-Java-Mem"><code class="literal">QPID_JAVA_MEM</code> environment variable</a>. | |
</li></ul></div><p> | |
</p></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6452"></a>9.11.6.2. Qpid Tuning</h4></div></div></div><div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; "><li class="listitem"> | |
The system property <code class="literal">qpid.broker.bdbTotalCacheSize</code> sets the total amount of heap memory (in bytes) allocated to BDB caches. | |
</li><li class="listitem"> | |
The system property <code class="literal">broker.flowToDiskThreshold</code> sets the threshold (in bytes) for flowing transient messages to disk. | |
Should the broker use more than direct memory it will flow incoming messages to disk. | |
Should utilisation fall beneath the threshold it will stop flowing messages to disk. | |
</li><li class="listitem"> | |
The system property <code class="literal">connection.maxUncommittedInMemorySize</code> sets the threshold (in bytes) | |
for total messages sizes (in bytes) from connection uncommitted transactions when messages are hold in memory. | |
If threshold is exceeded, all messages from connection in-flight transactions are flowed to disk including | |
those arriving after breaching the threshold. | |
</li></ul></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6471"></a>9.11.6.3. Formulae</h4></div></div></div><p> | |
We developed a simple formula which estimates the <span class="emphasis"><em>minimum</em></span> memory usage of the broker under certain usage. | |
These are rough estimate so we strongly recommend testing your configuration extensively. | |
Also, if your machine has more memory available by all means use more memory as it can only improve the performance and stability of your broker. | |
However, remember that both heap and direct memory are served from your computer's physical memory so their sum should never exceed the physically available RAM (minus what other processes use). | |
</p><p> | |
</p><div class="informalequation"><span class="mathphrase"> | |
memory<sub>heap</sub> = 15 MB + 20 kB * N<sub>sessions</sub> + (1.7 kB + (120 + averageSize<sub>headerNameAndValue</sub> ) * averageNumber<sub>headers</sub>)* N<sub>messages</sub> + 100 kB * N<sub>connections</sub> | |
</span></div><p> | |
</p><p> | |
</p><div class="informalequation"><span class="mathphrase"> | |
memory<sub>direct</sub> = 2 MB + (200 B + averageSize<sub>msg</sub> *2)* N<sub>messages</sub> + 1MB * N<sub>connections</sub> | |
</span></div><p> | |
</p><p> | |
Where <span class="mathphrase">N</span> denotes the total number of connections/sessions/messages on the broker. Furthermore, for direct memory only the messages that have not been flowed to disk are relevant. | |
</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>The formulae assume the worst case in terms of memory usage: persistent messages and TLS connections. Transient messages consume less heap memory than persistent and plain connections consume less direct memory than TLS | |
connections. | |
</p></div></div><div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="d0e6529"></a>9.11.6.4. Things to Consider</h4></div></div></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e6532"></a>Performance</h5></div></div></div><p> | |
Choosing a smaller direct memory size will lower the threshold for flowing transient messages to disk when messages accumulate on a queue. | |
This can have impact on performance in the transient case where otherwise no disk I/O would be involved. | |
</p><p> | |
Having too little heap memory will result in poor performance due to frequent garbage collection events. See <a class="xref" href="Java-Broker-Runtime-Memory.html#Java-Broker-Runtime-Memory-Low-Memory" title="9.11.4. Low Memory Conditions">Section 9.11.4, “Low Memory Conditions”</a> for more details. | |
</p></div><div class="section"><div class="titlepage"><div><div><h5 class="title"><a id="d0e6541"></a>OutOfMemoryError</h5></div></div></div><p> | |
Choosing too low heap memory can cause an OutOfMemoryError which will force the broker to shut down. | |
In this sense the available heap memory puts a hard limit on the number of messages you can have in the broker at the same time. | |
</p><p> | |
If the Java runs out of direct memory it also throws a OutOfMemoryError resulting the a broker shutdown. | |
Under normal circumstances this should not happen but needs to be considered when deviating from the default configuration, especially when changing the flowToDiskThreshold. | |
</p><p> | |
If you are sending very large messages you should accommodate for this by making sure you have enough direct memory. | |
</p></div></div></div></div><div class="navfooter"><hr /><table summary="Navigation footer" width="100%"><tr><td align="left" width="40%"><a accesskey="p" href="Java-Broker-Runtime-Connection-Limit.html">Prev</a> </td><td align="center" width="20%"><a accesskey="u" href="Java-Broker-Runtime.html">Up</a></td><td align="right" width="40%"> <a accesskey="n" href="Java-Broker-High-Availability.html">Next</a></td></tr><tr><td align="left" valign="top" width="40%">9.10. Connection Limits </td><td align="center" width="20%"><a accesskey="h" href="Apache-Qpid-Broker-J-Book.html">Home</a></td><td align="right" valign="top" width="40%"> Chapter 10. High Availability</td></tr></table></div></div> |