| <!DOCTYPE html> |
| <!-- |
| |
| Licensed to the Apache Software Foundation (ASF) under one or more |
| contributor license agreements. See the NOTICE file distributed with |
| this work for additional information regarding copyright ownership. |
| The ASF licenses this file to You under the Apache License, Version 2.0 |
| (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE- 2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| <html> |
| <head> |
| <title>Apache BookKeeper - Metastore Interface</title> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <!-- Bootstrap --> |
| <link href="/archives/css/bootstrap.min.css" rel="stylesheet"> |
| <link href="/archives/css/bootstrap-responsive.min.css" rel="stylesheet"> |
| <link href="/archives/css/styles.css" rel="stylesheet"> |
| </head> |
| <body> |
| <header class="navbar navbar-inverse navbar-static-top" role="banner"> |
| <div class="container"> |
| <div class="navbar-header hidden-xs hidden-sm"> |
| <a class="navbar-brand navbar-logo" href="/archives/"><img class="img-responsive" src="/archives/img/bookkeeper_blk40.png" alt="Bookkeeper Logo" /></a> |
| </div> |
| <div class="navbar-header"> |
| <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target=".bs-navbar-collapse"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="/archives/">Apache BookKeeper</a> |
| </div> |
| <nav class="collapse navbar-collapse bs-navbar-collapse" role="navigation"> |
| <ul class="nav navbar-nav"> |
| <li><a href="/archives/releases.html">Download</a></li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Documentation<span class="caret"></span></a> |
| <ul class="dropdown-menu" role="menu"> |
| <li><a href="/archives/docs/master">Latest (master)</a></li> |
| <li><ul> |
| <li><a href="/archives/docs/master/apidocs">Java API docs</a></li> |
| <li><a href="/archives/docs/master/bookkeeperTutorial.html">Tutorial</a></li> |
| <li><a href="/archives/docs/master/bookkeeperConfig.html">Admin guide</a></li> |
| </ul><li> |
| <li><a href="/archives/docs/r4.4.0">Release 4.4.0</a></li> |
| <li class="divider"></li> |
| <li>Older releases</li> |
| <li><a href="/archives/docs/r4.3.2">Release 4.3.2</a></li> |
| <li><a href="/archives/docs/r4.3.1">Release 4.3.1</a></li> |
| <li><a href="/archives/docs/r4.3.0">Release 4.3.0</a></li> |
| <li><a href="/archives/docs/r4.2.4">Release 4.2.4</a></li> |
| <li><a href="/archives/docs/r4.2.3">Release 4.2.3</a></li> |
| <li><a href="/archives/docs/r4.2.2">Release 4.2.2</a></li> |
| <li><a href="/archives/docs/r4.2.1">Release 4.2.1</a></li> |
| <li><a href="/archives/docs/r4.2.0">Release 4.2.0</a></li> |
| <li><a href="/archives/docs/r4.1.0">Release 4.1.0</a></li> |
| <li><a href="/archives/docs/r4.0.0">Release 4.0.0</a></li> |
| </ul> |
| </li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Get Involved<span class="caret"></span></a> |
| <ul class="dropdown-menu" role="menu"> |
| <li><a href="/archives/lists.html">Mailing Lists</a></li> |
| <li><a href="/archives/irc.html">IRC</a></li> |
| <li><a href="/archives/svn.html">Version Control</a></li> |
| <li><a href="https://issues.apache.org/jira/browse/BOOKKEEPER">Issue Tracker</a></li> |
| </ul> |
| </li> |
| |
| <li><a href="https://cwiki.apache.org/confluence/display/BOOKKEEPER/Index">Wiki</a></li> |
| <!--<li><a href="#">Hedwig</a></li>//--> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Project Info<span class="caret"></span></a> |
| <ul class="dropdown-menu" role="menu"> |
| <li><a href="/archives/credits.html">Who are we?</a></li> |
| <li><a href="/archives/bylaws.html">Bylaws</a></li> |
| <li><a href="http://www.apache.org/licenses/">License</a></li> |
| <li class="divider"></li> |
| <li><a href="/archives/privacy.html">Privacy Policy</a></li> |
| <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsership</a></li> |
| <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| </ul> |
| </li> |
| </ul> |
| <script> |
| (function() { |
| var cx = '017580107654524087317:iqnsyimpydg'; |
| var gcse = document.createElement('script'); |
| gcse.type = 'text/javascript'; |
| gcse.async = true; |
| gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') + |
| '//www.google.com/cse/cse.js?cx=' + cx; |
| var s = document.getElementsByTagName('script')[0]; |
| s.parentNode.insertBefore(gcse, s); |
| })(); |
| </script> |
| |
| <div class="navbar-form navbar-right visible-lg" id="googlebox"> |
| <gcse:searchbox-only></gcse:searchbox-only> |
| </div> |
| </nav> |
| </div> |
| </header> |
| <div class="container"> |
| |
| <h1>Metastore Interface</h1> |
| |
| <p>Although Apache BookKeeper provides <a href="./bookkeeperMetadata.html">LedgerManager</a> and <a href="./hedwigMetadata.html">Hedwig Metadata Managers</a> for users to plugin different metadata storages for both BookKeeper and Hedwig, it is quite difficult to implement a correct and efficient manager version based on the knowledge for both projects. The <i>MetaStore</i> interface extracts the commonality of the metadata storage interfaces and is provided for users to focus on adapting the underlying storage itself w/o having to worry about the detailed logic for BookKeeper and Hedwig.</p> |
| |
| <h2>MetaStore</h2> |
| |
| <p>The <i>MetaStore</i> interface provide users with access to <i>MetastoreTable__s used for BookKeeper and Hedwig metadata management. There are two kinds of table defined in a <i>MetaStore</i>, <i>MetastoreTable</i> which provides basic <i><span class="caps">PUT</span></i>,__GET</i>,__REMOVE__,__SCAN__ operations and which does not assume any ordering requirements from the underlying storage; and <i>MetastoreScannableTable</i> which is derived from <i>MetastoreTable</i>, but <strong>does</strong> assume that data is stored in key order in the underlying storage.</p> |
| |
| <ul> |
| <li><code>getName</code>: Return the name of the <i>MetaStore</i>.</li> |
| <li><code>getVersion</code>: Return current <i>MetaStore</i> plugin version.</li> |
| <li><code>init</code>: Initialize the <i>MetaStore</i> library with the given configuration and its version.</li> |
| <li><code>close</code>: Close the <i>MetaStore</i>, freeing all resources. i.e. release all the open connections and occupied memory etc.</li> |
| <li><code>createTable</code>: Create a table instance to access the data stored in it. A table name is given to locate the table. An <i>MetastoreTable</i> object is returned.</li> |
| <li><code>createScannableTable</code>: Similar as <i>createTable</i>, but returns <i>MetastoreScannableTable</i> rather then <i>MetastoreTable</i> object. If the underlying table is not an ordered table, <i>MetastoreException</i> should be thrown.</li> |
| </ul> |
| |
| <h2>MetaStore Table</h2> |
| |
| <p><i>MetastoreTable</i> is a basic unit in a <i>MetaStore</i>, which is used to handle different types of metadata, i.e. A <i>MetastoreTable</i> is used to store metadata for ledgers, while the other <i>MetastoreTable</i> is used to store metadata for topic persistence info. The interface for a <i>MetastoreTable</i> is quite simple:</p> |
| |
| <ul> |
| <li><code>get</code>: Retrieve a entry by a given <i>key</i>. <i>OK</i> and its current version in metadata storage is returned when succeed. <i>NoKey</i> returned for a non-existent key. If <i>fields</i> are specified, return only the specified fields for the key.</li> |
| <li><code>put</code>: Put the given <i>value</i> associated with <i>key</i> with given <i>version</i>. The value is only updated when the given <i>version</i> equals the current version in metadata storage. A new <i>version</i> should be returned when updated successfully. <i>NoKey</i> is returned for a non-existent key, <i>BadVersion</i> is returned when an update is attempted with a <i>version</i> which does not match the one in the metadata store.</li> |
| <li><code>remove</code>: Remove the given <i>value</i> associated with <i>key</i>. The value is only removed when the given <i>version</i> equals its current version in metadata storage. <i>NoKey</i> is returned for a non-existent key, <i>BadVersion</i> is returned when remove is attempted with a <i>version</i> which does not match.</li> |
| <li><code>openCursor</code>: Open a <i>cursor</i> to iterate over all the entries of a table. The returned cursor doesn't need to guarantee any order and transaction.</li> |
| </ul> |
| |
| <h2>MetaStore Scannable Table</h2> |
| |
| <p><i>MetastoreScannableTable</i> is identical to a <i>MetastoreTable</i> except that it provides an addition interface to iterate over entries in the table in key order.</p> |
| |
| <ul> |
| <li><code>openCursor</code>: Open a <i>cursor</i> to iterate over all the entries of a table between the key range of <i>firstKey</i> and <i>lastKey</i>.</li> |
| </ul> |
| |
| <h2>How to organize your metadata.</h2> |
| |
| <p>Some metadata in Hedwig and BookKeeper does not need to be stored in the order of the ledger id or the topic. You could use kind of hash table to store metadata for them. These metadata are topic ownership and topic persistence info. Besides that, subscription state and ledger metadata must be stored in key order due to the current logic in Hedwig/BookKeeper.</p> |
| |
| </div> |
| <footer class="footer"> |
| <div class="container"> |
| <p class="text-muted">Copyright © 2014 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.<br/> |
| Apache BookKeeper, BookKeeper, Apache, Apache ZooKeeper, ZooKeeper, the Apache feather logo, and the Apache BookKeeper project logo are trademarks of The Apache Software Foundation.</p> |
| </div> |
| </footer> |
| |
| <script src="//code.jquery.com/jquery.js"></script> |
| <script src="/archives/js/bootstrap.min.js"></script> |
| </body> |
| </html> |
