| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta name="generator" content="Asciidoctor 2.0.18"> |
| <link rel="icon" type="image/png" href="images/favicon.png"> |
| <title>Persistence</title> |
| <link rel="stylesheet" href="css/asciidoctor.css"> |
| <link rel="stylesheet" href="css/font-awesome.css"> |
| <link rel="stylesheet" href="css/rouge-github.css"> |
| </head> |
| <body class="book toc2 toc-left"> |
| <div id="header"> |
| <h1>Persistence</h1> |
| <div id="toc" class="toc2"> |
| <div id="toctitle"><a href="index.html">User Manual for 2.32.0</a></div> |
| <ul class="sectlevel1"> |
| <li><a href="#file-journal-default">1. File Journal (Default)</a> |
| <ul class="sectlevel2"> |
| <li><a href="#journal-and-data-retention">1.1. Journal and Data Retention</a></li> |
| <li><a href="#java-nio">1.2. Java NIO</a></li> |
| <li><a href="#linux-asynchronous-io">1.3. Linux Asynchronous IO</a></li> |
| <li><a href="#memory-mapped">1.4. Memory mapped</a></li> |
| <li><a href="#standard-files">1.5. Standard Files</a></li> |
| <li><a href="#installing-aio">1.6. Installing AIO</a></li> |
| </ul> |
| </li> |
| <li><a href="#jdbc-persistence">2. JDBC Persistence</a> |
| <ul class="sectlevel2"> |
| <li><a href="#configuring-jdbc-persistence">2.1. Configuring JDBC Persistence</a></li> |
| <li><a href="#configuring-jdbc-connection-pooling">2.2. Configuring JDBC connection pooling</a></li> |
| </ul> |
| </li> |
| <li><a href="#zero-persistence">3. Zero Persistence</a></li> |
| </ul> |
| </div> |
| </div> |
| <div id="content"> |
| <div id="preamble"> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>Apache ActiveMQ Artemis ships with two persistence options. |
| The file journal which is highly optimized for the messaging use case and gives great performance, and also the JDBC Store, which uses JDBC to connect to a database of your choice.</p> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="file-journal-default"><a class="anchor" href="#file-journal-default"></a><a class="link" href="#file-journal-default">1. File Journal (Default)</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>The file journal is an <em>append only</em> journal. |
| It consists of a set of files on disk. |
| Each file is pre-created to a fixed size and initially filled with padding. |
| As operations are performed on the server, e.g. add message, update message, delete message, records are appended to the journal. |
| When one journal file is full we move to the next one.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Because records are only appended, i.e. added to the end of the journal we minimise disk head movement, i.e. we minimise random access operations which is typically the slowest operation on a disk.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Making the file size configurable means that an optimal size can be chosen, i.e. making each file fit on a disk cylinder. |
| Modern disk topologies are complex and we are not in control over which cylinder(s) the file is mapped onto so this is not an exact science. |
| But by minimising the number of disk cylinders the file is using, we can minimise the amount of disk head movement, since an entire disk cylinder is accessible simply by the disk rotating - the head does not have to move.</p> |
| </div> |
| <div class="paragraph"> |
| <p>As delete records are added to the journal, Apache ActiveMQ Artemis has a sophisticated file garbage collection algorithm which can determine if a particular journal file is needed any more - i.e. has all its data been deleted in the same or other files. |
| If so, the file can be reclaimed and re-used.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Apache ActiveMQ Artemis also has a compaction algorithm which removes dead space from the journal and compresses up the data so it takes up less files on disk.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The journal also fully supports transactional operation if required, supporting both local and XA transactions.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The majority of the journal is written in Java, however we abstract out the interaction with the actual file system to allow different pluggable implementations. |
| Apache ActiveMQ Artemis ships with two implementations:</p> |
| </div> |
| <div class="sect2"> |
| <h3 id="journal-and-data-retention"><a class="anchor" href="#journal-and-data-retention"></a><a class="link" href="#journal-and-data-retention">1.1. Journal and Data Retention</a></h3> |
| <div class="paragraph"> |
| <p>ActiveMQ Artemis provides a way to store and replay historical data. Refer to <a href="data-retention.html">Data Retention</a> chapter for more information.</p> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="java-nio"><a class="anchor" href="#java-nio"></a><a class="link" href="#java-nio">1.2. Java <a href="https://en.wikipedia.org/wiki/New_I/O">NIO</a></a></h3> |
| <div class="paragraph"> |
| <p>The first implementation uses standard Java NIO to interface with the file system. |
| This provides extremely good performance and runs on any platform where there’s a Java 6+ runtime.</p> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="linux-asynchronous-io"><a class="anchor" href="#linux-asynchronous-io"></a><a class="link" href="#linux-asynchronous-io">1.3. Linux Asynchronous IO</a></h3> |
| <div class="paragraph"> |
| <p>The second implementation uses a thin native code wrapper to talk to the Linux asynchronous IO library (AIO). |
| With AIO, Apache ActiveMQ Artemis will be called back when the data has made it to disk, allowing us to avoid explicit syncs altogether and simply send back confirmation of completion when AIO informs us that the data has been persisted.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Using AIO will typically provide even better performance than using Java NIO.</p> |
| </div> |
| <div class="paragraph"> |
| <p>This journal option is only available when running Linux kernel 2.6 or later and after having installed libaio (if it’s not already installed). |
| For instructions on how to install libaio please see Installing AIO section.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Also, please note that AIO will only work with the following file systems: ext2, ext3, ext4, jfs, xfs and NFSV4.</p> |
| </div> |
| <div class="paragraph"> |
| <p>For more information on libaio please see <a href="libaio.html#libaio-native-libraries">lib AIO</a>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>libaio is part of the kernel project.</p> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="memory-mapped"><a class="anchor" href="#memory-mapped"></a><a class="link" href="#memory-mapped">1.4. Memory mapped</a></h3> |
| <div class="paragraph"> |
| <p>The third implementation uses a file-backed <a href="https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.MapMode.html#READ_WRITE">READ_WRITE</a> <a href="https://en.wikipedia.org/wiki/Memory-mapped_file">memory mapping</a> against the OS page cache to interface with the file system.</p> |
| </div> |
| <div class="paragraph"> |
| <p>This provides extremely good performance (especially under strictly process failure durability requirements), almost zero copy (actually <em>is</em> the kernel page cache) and zero garbage (from the Java HEAP perspective) operations and runs on any platform where there’s a Java 4+ runtime.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Under power failure durability requirements it will perform at least on par with the NIO journal with the only exception of Linux OS with kernel less or equals 2.6, in which the <a href="https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByteBuffer.html#force%28%29"><em>msync</em></a>) implementation necessary to ensure durable writes was different (and slower) from the <a href="https://docs.oracle.com/javase/8/docs/api/java/nio/channels/FileChannel.html#force%28boolean%29"><em>fsync</em></a> used is case of NIO journal.</p> |
| </div> |
| <div class="paragraph"> |
| <p>It benefits by the configuration of OS <a href="https://en.wikipedia.org/wiki/Page_%28computer_memory%29">huge pages</a>, in particular when is used a big number of journal files and sizing them as multiple of the OS page size in bytes.</p> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="standard-files"><a class="anchor" href="#standard-files"></a><a class="link" href="#standard-files">1.5. Standard Files</a></h3> |
| <div class="paragraph"> |
| <p>The standard Apache ActiveMQ Artemis core server uses two instances of the journal:</p> |
| </div> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>Bindings journal.</p> |
| <div class="paragraph"> |
| <p>This journal is used to store bindings related data. |
| That includes the set of queues that are deployed on the server and their attributes. |
| It also stores data such as id sequence counters.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The bindings journal is always a NIO journal as it is typically low throughput compared to the message journal.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The files on this journal are prefixed as <code>activemq-bindings</code>. |
| Each file has a <code>bindings</code> extension. |
| File size is <code>1048576</code>, and it is located at the bindings folder.</p> |
| </div> |
| </li> |
| <li> |
| <p>Message journal.</p> |
| <div class="paragraph"> |
| <p>This journal instance stores all message related data, including the message themselves and also duplicate-id caches.</p> |
| </div> |
| <div class="paragraph"> |
| <p>By default Apache ActiveMQ Artemis will try and use an AIO journal. |
| If AIO is not available, e.g. the platform is not Linux with the correct kernel version or AIO has not been installed then it will automatically fall back to using Java NIO which is available on any Java platform.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The files on this journal are prefixed as <code>activemq-data</code>. |
| Each file has an <code>amq</code> extension. |
| File size is by the default <code>10485760</code> (configurable), and it is located at the journal folder.</p> |
| </div> |
| </li> |
| </ul> |
| </div> |
| <div class="paragraph"> |
| <p>For large messages, Apache ActiveMQ Artemis persists them outside the message journal. |
| This is discussed in <a href="large-messages.html#large-messages">Large Messages</a>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Apache ActiveMQ Artemis can also be configured to page messages to disk in low memory situations. |
| This is discussed in <a href="paging.html#paging">Paging</a>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>If no persistence is required at all, Apache ActiveMQ Artemis can also be configured not to persist any data at all to storage as discussed in the Configuring the broker for Zero Persistence section.</p> |
| </div> |
| <div class="sect3"> |
| <h4 id="configuring-the-bindings-journal"><a class="anchor" href="#configuring-the-bindings-journal"></a><a class="link" href="#configuring-the-bindings-journal">1.5.1. Configuring the bindings journal</a></h4> |
| <div class="paragraph"> |
| <p>The bindings journal is configured using the following attributes in <code>broker.xml</code></p> |
| </div> |
| <div class="dlist"> |
| <dl> |
| <dt class="hdlist1">bindings-directory</dt> |
| <dd> |
| <p>This is the directory in which the bindings journal lives. |
| The default value is <code>data/bindings</code>.</p> |
| </dd> |
| <dt class="hdlist1">create-bindings-dir</dt> |
| <dd> |
| <p>If this is set to <code>true</code> then the bindings directory will be automatically created at the location specified in <code>bindings-directory</code> if it does not already exist. |
| The default value is <code>true</code></p> |
| </dd> |
| </dl> |
| </div> |
| </div> |
| <div class="sect3"> |
| <h4 id="configuring-the-jms-journal"><a class="anchor" href="#configuring-the-jms-journal"></a><a class="link" href="#configuring-the-jms-journal">1.5.2. Configuring the jms journal</a></h4> |
| <div class="paragraph"> |
| <p>The jms config shares its configuration with the bindings journal.</p> |
| </div> |
| </div> |
| <div class="sect3"> |
| <h4 id="configuring-the-message-journal"><a class="anchor" href="#configuring-the-message-journal"></a><a class="link" href="#configuring-the-message-journal">1.5.3. Configuring the message journal</a></h4> |
| <div class="paragraph"> |
| <p>The message journal is configured using the following attributes in <code>broker.xml</code></p> |
| </div> |
| <div class="dlist"> |
| <dl> |
| <dt class="hdlist1">journal-directory</dt> |
| <dd> |
| <p>This is the directory in which the message journal lives. |
| The default value is <code>data/journal</code>.</p> |
| <div class="paragraph"> |
| <p>For the best performance, we recommend the journal is located on its own physical volume in order to minimise disk head movement. |
| If the journal is on a volume which is shared with other processes which might be writing other files (e.g. bindings journal, database, or transaction coordinator) then the disk head may well be moving rapidly between these files as it writes them, thus drastically reducing performance.</p> |
| </div> |
| <div class="paragraph"> |
| <p>When the message journal is stored on a SAN we recommend each journal instance that is stored on the SAN is given its own LUN (logical unit).</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">node-manager-lock-directory</dt> |
| <dd> |
| <p>This is the directory in which the node manager file lock lives. |
| By default has the same value of <code>journal-directory</code>.</p> |
| <div class="paragraph"> |
| <p>This is useful when the message journal is on a SAN and is being used a <a href="ha.html#shared-store">Shared Store HA</a> policy with the broker instances on the same physical machine.</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">create-journal-dir</dt> |
| <dd> |
| <p>If this is set to <code>true</code> then the journal directory will be automatically created at the location specified in <code>journal-directory</code> if it does not already exist. |
| The default value is <code>true</code></p> |
| </dd> |
| <dt class="hdlist1">journal-type</dt> |
| <dd> |
| <p>Valid values are <code>NIO</code>, <code>ASYNCIO</code> or <code>MAPPED</code>.</p> |
| <div class="paragraph"> |
| <p>Choosing <code>NIO</code> chooses the Java NIO journal. |
| Choosing <code>ASYNCIO</code> chooses the Linux asynchronous IO journal. |
| If you choose <code>ASYNCIO</code> but are not running Linux or you do not have libaio installed then Apache ActiveMQ Artemis will detect this and automatically fall back to using <code>NIO</code>. |
| Choosing <code>MAPPED</code> chooses the Java Memory Mapped journal.</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-sync-transactional</dt> |
| <dd> |
| <p>If this is set to true then Apache ActiveMQ Artemis will make sure all transaction data is flushed to disk on transaction boundaries (commit, prepare and rollback). |
| The default value is <code>true</code>.</p> |
| </dd> |
| <dt class="hdlist1">journal-sync-non-transactional</dt> |
| <dd> |
| <p>If this is set to true then Apache ActiveMQ Artemis will make sure non transactional message data (sends and acknowledgements) are flushed to disk each time. |
| The default value for this is <code>true</code>.</p> |
| </dd> |
| <dt class="hdlist1">journal-file-size</dt> |
| <dd> |
| <p>The size of each journal file in bytes. |
| The default value for this is <code>10485760</code> bytes (10MiB).</p> |
| </dd> |
| <dt class="hdlist1">journal-min-files</dt> |
| <dd> |
| <p>The minimum number of files the journal will maintain. |
| When Apache ActiveMQ Artemis starts and there is no initial message data, Apache ActiveMQ Artemis will pre-create <code>journal-min-files</code> number of files.</p> |
| <div class="paragraph"> |
| <p>Creating journal files and filling them with padding is a fairly expensive operation and we want to minimise doing this at run-time as files get filled. |
| By pre-creating files, as one is filled the journal can immediately resume with the next one without pausing to create it.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Depending on how much data you expect your queues to contain at steady state you should tune this number of files to match that total amount of data.</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-pool-files</dt> |
| <dd> |
| <p>The system will create as many files as needed however when reclaiming files it will shrink back to the <code>journal-pool-files</code>.</p> |
| <div class="paragraph"> |
| <p>The default to this parameter is -1, meaning it will never delete files on the journal once created.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Notice that the system can’t grow infinitely as you are still required to use paging for destinations that can grow indefinitely.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Notice: in case you get too many files you can use <a href="data-tools.html#data-tools">compacting</a>.</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-max-io</dt> |
| <dd> |
| <p>Write requests are queued up before being submitted to the system for execution. |
| This parameter controls the maximum number of write requests that can be in the IO queue at any one time. |
| If the queue becomes full then writes will block until space is freed up.</p> |
| <div class="paragraph"> |
| <p>When using NIO, this value should always be equal to <code>1</code></p> |
| </div> |
| <div class="paragraph"> |
| <p>When using ASYNCIO, the default should be <code>500</code>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>The system maintains different defaults for this parameter depending on whether it’s NIO or ASYNCIO (default for NIO is 1, default for ASYNCIO is 500)</p> |
| </div> |
| <div class="paragraph"> |
| <p>There is a limit and the total max ASYNCIO can’t be higher than what is configured at the OS level (/proc/sys/fs/aio-max-nr) usually at 65536.</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-buffer-timeout</dt> |
| <dd> |
| <p>Instead of flushing on every write that requires a flush, we maintain an internal buffer, and flush the entire buffer either when it is full, or when a timeout expires, whichever is sooner. |
| This is used for both NIO and ASYNCIO and allows the system to scale better with many concurrent writes that require flushing.</p> |
| <div class="paragraph"> |
| <p>This parameter controls the timeout at which the buffer will be flushed if it hasn’t filled already. |
| ASYNCIO can typically cope with a higher flush rate than NIO, so the system maintains different defaults for both NIO and ASYNCIO (default for NIO is 3333333 nanoseconds - 300 times per second, default for ASYNCIO is 500000 nanoseconds - ie. |
| 2000 times per second).</p> |
| </div> |
| <div class="paragraph"> |
| <p>Setting this property to 0 will disable the internal buffer and writes will be directly written to the journal file immediately.</p> |
| </div> |
| <div class="admonitionblock note"> |
| <table> |
| <tr> |
| <td class="icon"> |
| <i class="fa icon-note" title="Note"></i> |
| </td> |
| <td class="content"> |
| <div class="paragraph"> |
| <p>By increasing the timeout, you may be able to increase system throughput at the expense of latency, the default parameters are chosen to give a reasonable balance between throughput and latency.</p> |
| </div> |
| </td> |
| </tr> |
| </table> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-buffer-size</dt> |
| <dd> |
| <p>The size of the timed buffer on ASYNCIO. |
| The default value is <code>490KiB</code>.</p> |
| </dd> |
| <dt class="hdlist1">journal-compact-min-files</dt> |
| <dd> |
| <p>The minimal number of files before we can consider compacting the journal. |
| The compacting algorithm won’t start until you have at least <code>journal-compact-min-files</code></p> |
| <div class="paragraph"> |
| <p>Setting this to 0 will disable the feature to compact completely. |
| This could be dangerous though as the journal could grow indefinitely. |
| Use it wisely!</p> |
| </div> |
| <div class="paragraph"> |
| <p>The default for this parameter is <code>10</code></p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-compact-percentage</dt> |
| <dd> |
| <p>The threshold to start compacting. |
| When less than this percentage of journal space is considered live data, we start compacting. |
| Note also that compacting won’t kick in until you have at least <code>journal-compact-min-files</code> data files on the journal</p> |
| <div class="paragraph"> |
| <p>The default for this parameter is <code>30</code></p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-lock-acquisition-timeout</dt> |
| <dd> |
| <p>How long to wait (in milliseconds) to acquire a file lock on the journal before giving up</p> |
| <div class="paragraph"> |
| <p>The default for this parameter is <code>-1</code> (i.e. indefinite))</p> |
| </div> |
| </dd> |
| <dt class="hdlist1">journal-datasync</dt> |
| <dd> |
| <p>This will disable the use of fdatasync on journal writes. |
| When enabled it ensures full power failure durability, otherwise process failure durability on journal writes (OS guaranteed). |
| This is particular effective for <code>NIO</code> and <code>MAPPED</code> journals, which rely on <em>fsync</em>/<em>msync</em> to force write changes to disk.</p> |
| <div class="paragraph"> |
| <p>Default is <code>true</code>.</p> |
| </div> |
| </dd> |
| </dl> |
| </div> |
| <div class="sidebarblock"> |
| <div class="content"> |
| <div class="title">Note on disabling <code>journal-datasync</code></div> |
| <div class="paragraph"> |
| <p>Any modern OS guarantees that on process failures (i.e. crash) all the uncommitted changes to the page cache will be flushed to the file system, maintaining coherence between subsequent operations against the same pages and ensuring that no data will be lost. |
| The predictability of the timing of such flushes, in case of a disabled <em>journal-datasync</em>, depends on the OS configuration, but without compromising (or relaxing) the process failure durability semantics as described above. |
| Rely on the OS page cache sacrifice the power failure protection, while increasing the effectiveness of the journal operations, capable of exploiting the read caching and write combining features provided by the OS’s kernel page cache subsystem.</p> |
| </div> |
| </div> |
| </div> |
| <div class="sidebarblock"> |
| <div class="content"> |
| <div class="title">Note on disabling disk write cache</div> |
| <div class="paragraph"> |
| <p>Most disks contain hardware write caches. |
| A write cache can increase the apparent performance of the disk because writes just go into the cache and are then lazily written to the disk later.</p> |
| </div> |
| <div class="paragraph"> |
| <p>This happens irrespective of whether you have executed a fsync() from the operating system or correctly synced data from inside a Java program!</p> |
| </div> |
| <div class="paragraph"> |
| <p>By default many systems ship with disk write cache enabled. |
| This means that even after syncing from the operating system there is no guarantee the data has actually made it to disk, so if a failure occurs, critical data can be lost.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Some more expensive disks have non volatile or battery backed write caches which won’t necessarily lose data on event of failure, but you need to test them!</p> |
| </div> |
| <div class="paragraph"> |
| <p>If your disk does not have an expensive non volatile or battery backed cache and it’s not part of some kind of redundant array (e.g. RAID), and you value your data integrity you need to make sure disk write cache is disabled.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Be aware that disabling disk write cache can give you a nasty shock performance wise. |
| If you’ve been used to using disks with write cache enabled in their default setting, unaware that your data integrity could be compromised, then disabling it will give you an idea of how fast your disk can perform when acting really reliably.</p> |
| </div> |
| <div class="paragraph"> |
| <p>On Linux you can inspect and/or change your disk’s write cache settings using the tools <code>hdparm</code> (for IDE disks) or <code>sdparm</code> or <code>sginfo</code> (for SDSI/SATA disks)</p> |
| </div> |
| <div class="paragraph"> |
| <p>On Windows you can check / change the setting by right clicking on the disk and clicking properties.</p> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="installing-aio"><a class="anchor" href="#installing-aio"></a><a class="link" href="#installing-aio">1.6. Installing AIO</a></h3> |
| <div class="paragraph"> |
| <p>The Java NIO journal gives great performance, but If you are running Apache ActiveMQ Artemis using Linux Kernel 2.6 or later, we highly recommend you use the <code>ASYNCIO</code> journal for the very best persistence performance.</p> |
| </div> |
| <div class="paragraph"> |
| <p>It’s not possible to use the ASYNCIO journal under other operating systems or earlier versions of the Linux kernel.</p> |
| </div> |
| <div class="paragraph"> |
| <p>If you are running Linux kernel 2.6 or later and don’t already have <code>libaio</code> installed, you can easily install it using the following steps:</p> |
| </div> |
| <div class="paragraph"> |
| <p>Using yum, (e.g. on Fedora or Red Hat Enterprise Linux):</p> |
| </div> |
| <div class="listingblock"> |
| <div class="content"> |
| <pre class="rouge highlight nowrap"><code data-lang="sh">yum <span class="nb">install </span>libaio</code></pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>Using aptitude, (e.g. on Ubuntu or Debian system):</p> |
| </div> |
| <div class="listingblock"> |
| <div class="content"> |
| <pre class="rouge highlight nowrap"><code data-lang="sh">apt-get <span class="nb">install </span>libaio</code></pre> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="jdbc-persistence"><a class="anchor" href="#jdbc-persistence"></a><a class="link" href="#jdbc-persistence">2. JDBC Persistence</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>The Apache ActiveMQ Artemis JDBC persistence layer offers the ability to store broker state (messages, address & queue definitions, etc.) using a database.</p> |
| </div> |
| <div class="admonitionblock note"> |
| <table> |
| <tr> |
| <td class="icon"> |
| <i class="fa icon-note" title="Note"></i> |
| </td> |
| <td class="content"> |
| <div class="paragraph"> |
| <p>Using the ActiveMQ Artemis File Journal is the <strong>recommended</strong> configuration as it offers higher levels of performance and is more mature. |
| Performance for both paging and large messages is especially diminished with JDBC. |
| The JDBC persistence layer is targeted to those users who <em>must</em> use a database e.g. due to internal company policy.</p> |
| </div> |
| </td> |
| </tr> |
| </table> |
| </div> |
| <div class="paragraph"> |
| <p>ActiveMQ Artemis currently has support for a limited number of database vendors:</p> |
| </div> |
| <div class="olist arabic"> |
| <ol class="arabic"> |
| <li> |
| <p>PostgreSQL</p> |
| </li> |
| <li> |
| <p>MySQL</p> |
| </li> |
| <li> |
| <p>Microsoft SQL Server</p> |
| </li> |
| <li> |
| <p>Oracle</p> |
| </li> |
| <li> |
| <p>DB2</p> |
| </li> |
| <li> |
| <p>Apache Derby</p> |
| </li> |
| </ol> |
| </div> |
| <div class="paragraph"> |
| <p>The JDBC store uses a JDBC connection to store messages and bindings data in records in database tables. |
| The data stored in the database tables is encoded using Apache ActiveMQ Artemis internal encodings.</p> |
| </div> |
| <div class="sect2"> |
| <h3 id="configuring-jdbc-persistence"><a class="anchor" href="#configuring-jdbc-persistence"></a><a class="link" href="#configuring-jdbc-persistence">2.1. Configuring JDBC Persistence</a></h3> |
| <div class="paragraph"> |
| <p>To configure Apache ActiveMQ Artemis to use a database for persisting messages and bindings data you must do two things.</p> |
| </div> |
| <div class="olist arabic"> |
| <ol class="arabic"> |
| <li> |
| <p>See the documentation on <a href="using-server.html#adding-runtime-dependencies">adding runtime dependencies</a> to understand how to make the JDBC driver available to the broker.</p> |
| </li> |
| <li> |
| <p>Create a store element in your broker.xml config file under the <code><core></code> element. |
| For example:</p> |
| </li> |
| </ol> |
| </div> |
| <div class="listingblock"> |
| <div class="content"> |
| <pre class="rouge highlight nowrap"><code data-lang="xml"><span class="nt"><store></span> |
| <span class="nt"><database-store></span> |
| <span class="nt"><jdbc-driver-class-name></span>org.apache.derby.jdbc.EmbeddedDriver<span class="nt"></jdbc-driver-class-name></span> |
| <span class="nt"><jdbc-connection-url></span>jdbc:derby:data/derby/database-store;create=true<span class="nt"></jdbc-connection-url></span> |
| <span class="nt"><bindings-table-name></span>BINDINGS_TABLE<span class="nt"></bindings-table-name></span> |
| <span class="nt"><message-table-name></span>MESSAGE_TABLE<span class="nt"></message-table-name></span> |
| <span class="nt"><page-store-table-name></span>MESSAGE_TABLE<span class="nt"></page-store-table-name></span> |
| <span class="nt"><large-message-table-name></span>LARGE_MESSAGES_TABLE<span class="nt"></large-message-table-name></span> |
| <span class="nt"><node-manager-store-table-name></span>NODE_MANAGER_TABLE<span class="nt"></node-manager-store-table-name></span> |
| <span class="nt"></database-store></span> |
| <span class="nt"></store></span></code></pre> |
| </div> |
| </div> |
| <div class="dlist"> |
| <dl> |
| <dt class="hdlist1">jdbc-connection-url</dt> |
| <dd> |
| <p>The full JDBC connection URL for your database server. |
| The connection url should include all configuration parameters and database name.</p> |
| <div class="admonitionblock note"> |
| <table> |
| <tr> |
| <td class="icon"> |
| <i class="fa icon-note" title="Note"></i> |
| </td> |
| <td class="content"> |
| When configuring the server using the XML configuration files please ensure to escape any illegal chars; |
| "&" for example, is typical in JDBC connection url and should be escaped to "&". |
| </td> |
| </tr> |
| </table> |
| </div> |
| </dd> |
| <dt class="hdlist1">bindings-table-name</dt> |
| <dd> |
| <p>The name of the table in which bindings data will be persisted for the ActiveMQ Artemis server. |
| Specifying table names allows users to share single database amongst multiple servers, without interference.</p> |
| </dd> |
| <dt class="hdlist1">message-table-name</dt> |
| <dd> |
| <p>The name of the table in which bindings data will be persisted for the ActiveMQ Artemis server. |
| Specifying table names allows users to share single database amongst multiple servers, without interference.</p> |
| </dd> |
| <dt class="hdlist1">large-message-table-name</dt> |
| <dd> |
| <p>The name of the table in which messages and related data will be persisted for the ActiveMQ Artemis server. |
| Specifying table names allows users to share single database amongst multiple servers, without interference.</p> |
| </dd> |
| <dt class="hdlist1">page-store-table-name</dt> |
| <dd> |
| <p>The name of the table to house the page store directory information. |
| Note that each address will have its own page table which will use this name appended with a unique id of up to 20 characters.</p> |
| </dd> |
| <dt class="hdlist1">node-manager-store-table-name</dt> |
| <dd> |
| <p>The name of the table in which the HA Shared Store locks (i.e. primary and backup) and HA related data will be persisted for the ActiveMQ Artemis server. |
| Specifying table names allows users to share single database amongst multiple servers, without interference. |
| Each Shared Store primary/backup pairs must use the same table name and isn’t supported to share the same table between multiple (and unrelated) primary/backup pairs.</p> |
| </dd> |
| <dt class="hdlist1">jdbc-driver-class-name</dt> |
| <dd> |
| <p>The fully qualified class name of the desired database Driver.</p> |
| </dd> |
| <dt class="hdlist1">jdbc-network-timeout</dt> |
| <dd> |
| <p>The JDBC network connection timeout in milliseconds. |
| The default value is 20000 milliseconds (ie 20 seconds). |
| When using a shared store it is recommended to set it less then or equal to <code>jdbc-lock-expiration</code>.</p> |
| </dd> |
| <dt class="hdlist1">jdbc-lock-renew-period</dt> |
| <dd> |
| <p>The period in milliseconds of the keep alive service of a JDBC lock. |
| The default value is 2000 milliseconds (ie 2 seconds).</p> |
| </dd> |
| <dt class="hdlist1">jdbc-lock-expiration</dt> |
| <dd> |
| <p>The time in milliseconds a JDBC lock is considered valid without keeping it alive. |
| The default value is 20000 milliseconds (ie 20 seconds).</p> |
| </dd> |
| <dt class="hdlist1">jdbc-journal-sync-period</dt> |
| <dd> |
| <p>The time in milliseconds the journal will be synced with JDBC. |
| The default value is 5 milliseconds.</p> |
| </dd> |
| <dt class="hdlist1">jdbc-allowed-time-diff</dt> |
| <dd> |
| <p>The maximal time offset between the broker and the database in milliseconds when requesting the current time of the database while updating and validating primary and backup locks. |
| Currently this value only affects the logging and will show a warning if the detected difference exceeds the limit. |
| The default value is 250 milliseconds.</p> |
| </dd> |
| <dt class="hdlist1">jdbc-max-page-size-bytes</dt> |
| <dd> |
| <p>The maximal size a page can use. The default and recommended maximum value is 100K bytes. |
| Using larger sizes will result in downloading large blobs that would affect performance when using paged messages.</p> |
| </dd> |
| </dl> |
| </div> |
| <div class="admonitionblock note"> |
| <table> |
| <tr> |
| <td class="icon"> |
| <i class="fa icon-note" title="Note"></i> |
| </td> |
| <td class="content"> |
| Some DBMS (e.g. Oracle, 30 chars) have restrictions on the size of table names, this should be taken into consideration when configuring table names for the Artemis database store, pay particular attention to the page store table name, which can be appended with a unique ID of up to 20 characters. |
| (for Oracle this would mean configuring a page-store-table-name of max size of 10 chars). |
| </td> |
| </tr> |
| </table> |
| </div> |
| <div class="paragraph"> |
| <p>It is also possible to explicitly add the user and password rather than in the JDBC url if you need to encode it, this would look like:</p> |
| </div> |
| <div class="listingblock"> |
| <div class="content"> |
| <pre class="rouge highlight nowrap"><code data-lang="xml"><span class="nt"><store></span> |
| <span class="nt"><database-store></span> |
| <span class="nt"><jdbc-driver-class-name></span>org.apache.derby.jdbc.EmbeddedDriver<span class="nt"></jdbc-driver-class-name></span> |
| <span class="nt"><jdbc-connection-url></span>jdbc:derby:data/derby/database-store;create=true<span class="nt"></jdbc-connection-url></span> |
| <span class="nt"><jdbc-user></span>ENC(dasfn353cewc)<span class="nt"></jdbc-user></span> |
| <span class="nt"><jdbc-password></span>ENC(ucwiurfjtew345)<span class="nt"></jdbc-password></span> |
| <span class="nt"><bindings-table-name></span>BINDINGS_TABLE<span class="nt"></bindings-table-name></span> |
| <span class="nt"><message-table-name></span>MESSAGE_TABLE<span class="nt"></message-table-name></span> |
| <span class="nt"><page-store-table-name></span>MESSAGE_TABLE<span class="nt"></page-store-table-name></span> |
| <span class="nt"><large-message-table-name></span>LARGE_MESSAGES_TABLE<span class="nt"></large-message-table-name></span> |
| <span class="nt"><node-manager-store-table-name></span>NODE_MANAGER_TABLE<span class="nt"></node-manager-store-table-name></span> |
| <span class="nt"><jdbc-page-max-size-bytes></span>100K<span class="nt"></jdbc-page-max-size-bytes></span> |
| <span class="nt"></database-store></span> |
| <span class="nt"></store></span></code></pre> |
| </div> |
| </div> |
| </div> |
| <div class="sect2"> |
| <h3 id="configuring-jdbc-connection-pooling"><a class="anchor" href="#configuring-jdbc-connection-pooling"></a><a class="link" href="#configuring-jdbc-connection-pooling">2.2. Configuring JDBC connection pooling</a></h3> |
| <div class="paragraph"> |
| <p>To configure Apache ActiveMQ Artemis to use a database with a JDBC connection pool you need to set the data source properties, for example:</p> |
| </div> |
| <div class="listingblock"> |
| <div class="content"> |
| <pre class="rouge highlight nowrap"><code data-lang="xml"><span class="nt"><store></span> |
| <span class="nt"><database-store></span> |
| <span class="nt"><data-source-properties></span> |
| <span class="nt"><data-source-property</span> <span class="na">key=</span><span class="s">"driverClassName"</span> <span class="na">value=</span><span class="s">"com.mysql.jdbc.Driver"</span> <span class="nt">/></span> |
| <span class="nt"><data-source-property</span> <span class="na">key=</span><span class="s">"url"</span> <span class="na">value=</span><span class="s">"jdbc:mysql://localhost:3306/artemis"</span> <span class="nt">/></span> |
| <span class="nt"><data-source-property</span> <span class="na">key=</span><span class="s">"username"</span> <span class="na">value=</span><span class="s">"artemis"</span> <span class="nt">/></span> |
| <span class="nt"><data-source-property</span> <span class="na">key=</span><span class="s">"password"</span> <span class="na">value=</span><span class="s">"artemis"</span> <span class="nt">/></span> |
| <span class="nt"><data-source-property</span> <span class="na">key=</span><span class="s">"poolPreparedStatements"</span> <span class="na">value=</span><span class="s">"true"</span> <span class="nt">/></span> |
| <span class="nt"></data-source-properties></span> |
| <span class="nt"><bindings-table-name></span>BINDINGS<span class="nt"></bindings-table-name></span> |
| <span class="nt"><message-table-name></span>MESSAGES<span class="nt"></message-table-name></span> |
| <span class="nt"><large-message-table-name></span>LARGE_MESSAGES<span class="nt"></large-message-table-name></span> |
| <span class="nt"><page-store-table-name></span>PAGE_STORE<span class="nt"></page-store-table-name></span> |
| <span class="nt"><node-manager-store-table-name></span>NODE_MANAGER_STORE<span class="nt"></node-manager-store-table-name></span> |
| <span class="nt"></database-store></span> |
| <span class="nt"></store></span></code></pre> |
| </div> |
| </div> |
| <div class="paragraph"> |
| <p>You can find the documentation of the data source properties at <a href="https://commons.apache.org/proper/commons-dbcp/configuration.html" class="bare">https://commons.apache.org/proper/commons-dbcp/configuration.html</a>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>To mask the value of a property you can use the same procedure used to <a href="masking-passwords.html#masking-passwords">mask passwords</a>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Please note that the reconnection works only if there is no client sending messages. |
| Instead, if there is an attempt to write to the journal’s tables during the reconnection, then the broker will fail fast and shutdown.</p> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="zero-persistence"><a class="anchor" href="#zero-persistence"></a><a class="link" href="#zero-persistence">3. Zero Persistence</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>In some situations, zero persistence is sometimes required for a messaging system. |
| Configuring Apache ActiveMQ Artemis to perform zero persistence is straightforward. |
| Simply set the parameter <code>persistence-enabled</code> in <code>broker.xml</code> to <code>false</code>.</p> |
| </div> |
| <div class="paragraph"> |
| <p>Please note that if you set this parameter to false, then <em>zero</em> persistence will occur. |
| That means no bindings data, message data, large message data, duplicate id caches or paging data will be persisted.</p> |
| </div> |
| </div> |
| </div> |
| </div> |
| </body> |
| </html> |
