| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <meta http-equiv="X-UA-Compatible" content="ie=edge"> |
| <title>ActiveMQ</title> |
| <link rel="icon" type="image/png" href="/assets/img/favicon.png"> |
| |
| <link rel="stylesheet" href="/css/main.css"> |
| <script defer src="https://use.fontawesome.com/releases/v5.0.8/js/all.js" integrity="sha384-SlE991lGASHoBfWbelyBPLsUlwY1GwNDJo3jSJO04KZ33K2bwfV9YBauFfnzvynJ" crossorigin="anonymous"></script> |
| <script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script> |
| <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script> |
| <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script> |
| </head> |
| |
| <body> |
| <nav class="navbar navbar-expand-lg navbar-light fixed-top"> |
| <div class="container"> |
| <!-- <a class="navbar-brand mr-auto" href="#"><img style="height: 50px" src="assets/img/apache-feather.png" /></a> --> |
| <a class="navbar-brand mr-auto" href="/"><img src="/assets/img/activemq_logo_black_small.png" style="height: 50px"/></a> |
| <button class="navbar-toggler ml-auto" type="button" data-toggle="collapse" data-target="#navbarContent" aria-controls="navbarContent" aria-expanded="false" aria-label="Toggle navigation"> |
| <span class="navbar-toggler-icon"></span> |
| </button> |
| |
| <div class="ml-auto collapse navbar-collapse" id="navbarContent"> |
| <ul class="navbar-nav ml-auto"> |
| <li class="nav-item"> |
| <a class="nav-link active" href="/index.html">Home</a> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link" id="navbarDropdownComponents" data-target="#" href="" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Components</a> |
| <ul class="dropdown-menu dropdown-menu-center" aria-labelledby="navbarDropdownComponents"> |
| <div class="row"> |
| <div class="col-12"> |
| <ul class="multi-column-dropdown"> |
| <li class="nav-item"><a class="dropdown-item" href="/components/classic">ActiveMQ 5</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/components/artemis/">ActiveMQ Artemis</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/components/nms">NMS Clients</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/components/cms">CMS Client</a></li> |
| </ul> |
| </div> |
| </div> |
| </ul> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link" id="navbarDropdownCommunity" data-target="#" href="" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Contact</a> |
| <ul class="dropdown-menu dropdown-menu-center multi-column columns-1" aria-labelledby="navbarDropdownCommunity"> |
| <div class="row"> |
| <div class="col-12"> |
| <ul class="multi-column-dropdown"> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#mailing">Mailing Lists</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#chat">Chat</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#issues">Report Issues</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/contact#contributing">Contributing</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/security-advisories.html">Security</a></li> |
| </ul> |
| </div> |
| </div> |
| </ul> |
| </li> |
| <li class="nav-item dropdown"> |
| <a class="nav-link" id="navbarDropdownTeam" data-target="#" href="" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">Apache</a> |
| <ul class="dropdown-menu dropdown-menu-center multi-column columns-1" aria-labelledby="navbarDropdownTeam"> |
| <div class="row"> |
| <div class="col-sm-12"> |
| <ul class="multi-column-dropdown"> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org">The Apache Software Foundation</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/licenses/">License</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/security-advisories.html">Security</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://www.apache.org/events/current-event">Events</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="https://people.apache.org/phonebook.html?pmc=activemq">PMC & Committers</a></li> |
| <li class="nav-item"><a class="dropdown-item" href="/team/reports">Board Reports</a></li> |
| </ul> |
| </div> |
| </div> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| <div class="content"> |
| <div class="page-title-activemq5"> |
| <div class="container"> |
| <h1>Replicated LevelDB Store</h1> |
| </div> |
| </div> |
| <div class="container" > |
| <div class="row" style="margin-top: 30px"> |
| <div class="col-12 activemq5"> |
| <blockquote> |
| <p><strong>Warning</strong></p> |
| |
| <p>The LevelDB store has been deprecated and is no longer supported or recommended for use. The recommended store is <a href="kahadb">KahaDB</a></p> |
| </blockquote> |
| |
| <h2 id="synopsis">Synopsis</h2> |
| |
| <p>The Replicated LevelDB Store uses Apache ZooKeeper to pick a master from a set of broker nodes configured to replicate a LevelDB Store. Then synchronizes all slave LevelDB Stores with the master keeps them up to date by replicating all updates from the master.</p> |
| |
| <p>The Replicated LevelDB Store uses the same data files as a LevelDB Store, so you can switch a broker configuration between replicated and non replicated whenever you want.</p> |
| |
| <blockquote> |
| <p><strong>Version Compatibility</strong></p> |
| |
| <p>Available as of ActiveMQ 5.9.0.</p> |
| </blockquote> |
| |
| <h2 id="how-it-works">How it works.</h2> |
| |
| <p><img src="assets/img/replicated-leveldb-store.png" alt="" /></p> |
| |
| <p>It uses <a href="http://zookeeper.apache.org/">Apache ZooKeeper</a> to coordinate which node in the cluster becomes the master. The elected master broker node starts and accepts client connections. The other nodes go into slave mode and connect the the master and synchronize their persistent state /w it. The slave nodes do not accept client connections. All persistent operations are replicated to the connected slaves. If the master dies, the slaves with the latest update gets promoted to become the master. The failed node can then be brought back online and it will go into slave mode.</p> |
| |
| <p>All messaging operations which require a sync to disk will wait for the update to be replicated to a quorum of the nodes before completing. So if you configure the store with <code class="highlighter-rouge">replicas="3"</code> then the quorum size is <code class="highlighter-rouge">(3/2+1)=2</code>. The master will store the update locally and wait for 1 other slave to store the update before reporting success. Another way to think about it is that store will do synchronous replication to a quorum of the replication nodes and asynchronous replication replication to any additional nodes.</p> |
| |
| <p>When a new master is elected, you also need at least a quorum of nodes online to be able to find a node with the lastest updates. The node with the lastest updates will become the new master. Therefore, it’s recommend that you run with at least 3 replica nodes so that you can take one down without suffering a service outage.</p> |
| |
| <h3 id="deployment-tips">Deployment Tips</h3> |
| |
| <p>Clients should be using the <a href="failover-transport-reference">Failover Transport</a> to connect to the broker nodes in the replication cluster. e.g. using a URL something like the following:</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>failover:(tcp://broker1:61616,tcp://broker2:61616,tcp://broker3:61616) |
| </code></pre></div></div> |
| <p>You should run at least 3 ZooKeeper server nodes so that the ZooKeeper service is highly available. Don’t overcommit your ZooKeeper servers. An overworked ZooKeeper might start thinking live replication nodes have gone offline due to delays in processing their ‘keep-alive’ messages.</p> |
| |
| <p>For best results, make sure you explicitly configure the hostname attribute with a hostname or ip address for the node that other cluster members to access the machine with. The automatically determined hostname is not always accessible by the other cluster members and results in slaves not being able to establish a replication session with the master.</p> |
| |
| <h2 id="configuration">Configuration</h2> |
| |
| <p>You can configure ActiveMQ to use LevelDB for its persistence adapter - like below :</p> |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code><broker brokerName="broker" ... > |
| ... |
| <persistenceAdapter> |
| <replicatedLevelDB |
| directory="activemq-data" |
| replicas="3" |
| bind="tcp://0.0.0.0:0" |
| zkAddress="zoo1.example.org:2181,zoo2.example.org:2181,zoo3.example.org:2181" |
| zkPassword="password" |
| zkPath="/activemq/leveldb-stores" |
| hostname="broker1.example.org" |
| /> |
| </persistenceAdapter> |
| ... |
| </broker> |
| </code></pre></div></div> |
| |
| <h3 id="replicated-leveldb-store-properties">Replicated LevelDB Store Properties</h3> |
| |
| <p>All the broker nodes that are part of the same replication set should have matching <code class="highlighter-rouge">brokerName</code> XML attributes. The following configuration properties should be the same on all the broker nodes that are part of the same replication set:</p> |
| |
| <table> |
| <thead> |
| <tr> |
| <th>property name</th> |
| <th>default value</th> |
| <th>Comments</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><code class="highlighter-rouge">replicas</code></td> |
| <td><code class="highlighter-rouge">3</code></td> |
| <td>The number of nodes that will exist in the cluster. At least (replicas/2)+1 nodes must be online to avoid service outage.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">securityToken</code></td> |
| <td> </td> |
| <td>A security token which must match on all replication nodes for them to accept each others replication requests.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">zkAddress</code></td> |
| <td><code class="highlighter-rouge">127.0.0.1:2181</code></td> |
| <td>A comma separated list of ZooKeeper servers.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">zkPassword</code></td> |
| <td> </td> |
| <td>The password to use when connecting to the ZooKeeper server.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">zkPath</code></td> |
| <td><code class="highlighter-rouge">/default</code></td> |
| <td>The path to the ZooKeeper directory where Master/Slave election information will be exchanged.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">zkSessionTimeout</code></td> |
| <td><code class="highlighter-rouge">2s</code></td> |
| <td>How quickly a node failure will be detected by ZooKeeper. (prior to 5.11 - this had a typo zkSessionTmeout)</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">sync</code></td> |
| <td><code class="highlighter-rouge">quorum_mem</code></td> |
| <td>Controls where updates are reside before being considered complete. This setting is a comma separated list of the following options: <code class="highlighter-rouge">local_mem</code>, <code class="highlighter-rouge">local_disk</code>, <code class="highlighter-rouge">remote_mem</code>, <code class="highlighter-rouge">remote_disk</code>, <code class="highlighter-rouge">quorum_mem</code>, <code class="highlighter-rouge">quorum_disk</code>. If you combine two settings for a target, the stronger guarantee is used. For example, configuring <code class="highlighter-rouge">local_mem, local_disk</code> is the same as just using <code class="highlighter-rouge">local_disk</code>. quorum_mem is the same as <code class="highlighter-rouge">local_mem, remote_mem</code> and <code class="highlighter-rouge">quorum_disk</code> is the same as <code class="highlighter-rouge">local_disk, remote_disk</code></td> |
| </tr> |
| </tbody> |
| </table> |
| |
| <p>Different replication sets can share the same <code class="highlighter-rouge">zkPath</code> as long they have different <code class="highlighter-rouge">brokerName</code>.</p> |
| |
| <p>The following configuration properties can be unique per node:</p> |
| |
| <table> |
| <thead> |
| <tr> |
| <th>property name</th> |
| <th> </th> |
| <th>default value</th> |
| <th>Comments</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><code class="highlighter-rouge">bind</code></td> |
| <td><code class="highlighter-rouge">tcp://0.0.0.0:61619</code></td> |
| <td>When this node becomes a master, it will bind the configured address and port to service the replication protocol. Using dynamic ports is also supported. Just configure with <code class="highlighter-rouge">tcp://0.0.0.0:0</code></td> |
| <td> </td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">hostname</code></td> |
| <td> </td> |
| <td>The host name used to advertise the replication service when this node becomes the master. If not set it will be automatically determined.</td> |
| <td> </td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">weight</code></td> |
| <td><code class="highlighter-rouge">1</code></td> |
| <td>The replication node that has the latest update with the highest weight will become the master. Used to give preference to some nodes towards becoming master.</td> |
| <td> </td> |
| </tr> |
| </tbody> |
| </table> |
| |
| <p>The store also supports the same configuration properties of a standard <a href="leveldb-store">LevelDB Store</a> but it does not support the pluggable storage lockers :</p> |
| |
| <h3 id="standard-leveldb-store-properties">Standard LevelDB Store Properties</h3> |
| |
| <table> |
| <thead> |
| <tr> |
| <th>property name</th> |
| <th>default value</th> |
| <th>Comments</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td><code class="highlighter-rouge">directory</code></td> |
| <td><code class="highlighter-rouge">LevelDB</code></td> |
| <td>The directory which the store will use to hold it’s data files. The store will create the directory if it does not already exist.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">readThreads</code></td> |
| <td><code class="highlighter-rouge">10</code></td> |
| <td>The number of concurrent IO read threads to allowed.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">logSize</code></td> |
| <td><code class="highlighter-rouge">104857600</code> (100 MB)</td> |
| <td>The max size (in bytes) of each data log file before log file rotation occurs.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">verifyChecksums</code></td> |
| <td><code class="highlighter-rouge">false</code></td> |
| <td>Set to true to force checksum verification of all data that is read from the file system.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">paranoidChecks</code></td> |
| <td><code class="highlighter-rouge">false</code></td> |
| <td>Make the store error out as soon as possible if it detects internal corruption.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexFactory</code></td> |
| <td><code class="highlighter-rouge">org.fusesource.leveldbjni.JniDBFactory, org.iq80.leveldb.impl.Iq80DBFactory</code></td> |
| <td>The factory classes to use when creating the LevelDB indexes</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexMaxOpenFiles</code></td> |
| <td><code class="highlighter-rouge">1000</code></td> |
| <td>Number of open files that can be used by the index.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexBlockRestartInterval</code></td> |
| <td><code class="highlighter-rouge">16</code></td> |
| <td>Number keys between restart points for delta encoding of keys.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexWriteBufferSize</code></td> |
| <td><code class="highlighter-rouge">6291456</code> (6 MB)</td> |
| <td>Amount of index data to build up in memory before converting to a sorted on-disk file.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexBlockSize</code></td> |
| <td><code class="highlighter-rouge">4096</code> (4 K)</td> |
| <td>The size of index data packed per block.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexCacheSize</code></td> |
| <td><code class="highlighter-rouge">268435456</code> (256 MB)</td> |
| <td>The maximum amount of off-heap memory to use to cache index blocks.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">indexCompression</code></td> |
| <td><code class="highlighter-rouge">snappy</code></td> |
| <td>The type of compression to apply to the index blocks. Can be snappy or none.</td> |
| </tr> |
| <tr> |
| <td><code class="highlighter-rouge">logCompression</code></td> |
| <td><code class="highlighter-rouge">none</code></td> |
| <td>The type of compression to apply to the log records. Can be snappy or none.</td> |
| </tr> |
| </tbody> |
| </table> |
| |
| <blockquote> |
| <p><strong>Caveats</strong></p> |
| |
| <p>The LevelDB store does not yet support storing data associated with <a href="delay-and-schedule-message-delivery">Delay and Schedule Message Delivery</a>. Those are are stored in a separate non-replicated KahaDB data files. Unexpected results will occur if you use <a href="delay-and-schedule-message-delivery">Delay and Schedule Message Delivery</a> with the replicated leveldb store since that data will be not be there when the master fails over to a slave.</p> |
| </blockquote> |
| |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="row sitemap"> |
| <div class="col-sm-12"> |
| <div class="container"> |
| <div class="row"> |
| <div class="col-sm-12"> |
| <div class="row"> |
| <div class="col-sm-3"> |
| <div > |
| <img class="float-left" style="max-height: 100px" src="/assets/img/activemq_logo_white_vertical_small.png"/> |
| </div> |
| </div> |
| <div style="text-align: center; margin-bottom: 0px; margin-top: 30px; font-size: 65%" class="col-sm-6"> |
| <p>Apache ActiveMQ, ActiveMQ, ActiveMQ Artemis, Apache, the Apache feather logo, and the Apache ActiveMQ project logo are trademarks of The Apache Software Foundation. Copyright © 2019, The Apache Software Foundation. Licensed under <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License 2.0</a>.</p> |
| </div> |
| <div class="col-sm-3"> |
| <div > |
| <a href="https://www.apache.org"><img class="float-right" style="margin-top: 10px; max-height: 80px" src="/assets/img/apache-logo-small.png"/></a> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| |
| </body> |
| </html> |
