| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| |
| |
| <title>Apache Jena - TDB - Store Parameters</title> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <link href="/css/bootstrap.min.css" rel="stylesheet" media="screen"> |
| <link href="/css/bootstrap-extension.css" rel="stylesheet" type="text/css"> |
| <link href="/css/jena.css" rel="stylesheet" type="text/css"> |
| <link rel="shortcut icon" href="/images/favicon.ico" /> |
| |
| <script src="https://code.jquery.com/jquery-2.2.4.min.js" |
| integrity="sha256-BbhdlvQf/xTY9gja0Dq3HiwQF8LaCRTXxZKRutelT44=" |
| crossorigin="anonymous"></script> |
| <script src="/js/jena-navigation.js" type="text/javascript"></script> |
| <script src="/js/bootstrap.min.js" type="text/javascript"></script> |
| |
| <script src="/js/improve.js" type="text/javascript"></script> |
| |
| |
| </head> |
| |
| <body> |
| |
| <nav class="navbar navbar-default" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-ex1-collapse"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="/index.html"> |
| <img class="logo-menu" src="/images/jena-logo/jena-logo-notext-small.png" alt="jena logo">Apache Jena</a> |
| </div> |
| |
| <div class="collapse navbar-collapse navbar-ex1-collapse"> |
| <ul class="nav navbar-nav"> |
| <li id="homepage"><a href="/index.html"><span class="glyphicon glyphicon-home"></span> Home</a></li> |
| <li id="download"><a href="/download/index.cgi"><span class="glyphicon glyphicon-download-alt"></span> Download</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown"><span class="glyphicon glyphicon-book"></span> Learn <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li class="dropdown-header">Tutorials</li> |
| <li><a href="/tutorials/index.html">Overview</a></li> |
| <li><a href="/documentation/fuseki2/index.html">Fuseki Triplestore</a></li> |
| <li><a href="/documentation/notes/index.html">How-To's</a></li> |
| <li><a href="/documentation/query/manipulating_sparql_using_arq.html">Manipulating SPARQL using ARQ</a></li> |
| <li><a href="/tutorials/rdf_api.html">RDF core API tutorial</a></li> |
| <li><a href="/tutorials/sparql.html">SPARQL tutorial</a></li> |
| <li><a href="/tutorials/using_jena_with_eclipse.html">Using Jena with Eclipse</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">References</li> |
| <li><a href="/documentation/index.html">Overview</a></li> |
| <li><a href="/documentation/query/index.html">ARQ (SPARQL)</a></li> |
| <li><a href="/documentation/assembler/index.html">Assembler</a></li> |
| <li><a href="/documentation/tools/index.html">Command-line tools</a></li> |
| <li><a href="/documentation/rdfs/">Data with RDFS Inferencing</a></li> |
| <li><a href="/documentation/geosparql/index.html">GeoSPARQL</a></li> |
| <li><a href="/documentation/inference/index.html">Inference API</a></li> |
| <li><a href="/documentation/javadoc.html">Javadoc</a></li> |
| <li><a href="/documentation/ontology/">Ontology API</a></li> |
| <li><a href="/documentation/permissions/index.html">Permissions</a></li> |
| <li><a href="/documentation/extras/querybuilder/index.html">Query Builder</a></li> |
| <li><a href="/documentation/rdf/index.html">RDF API</a></li> |
| <li><a href="/documentation/rdfconnection/">RDF Connection - SPARQL API</a></li> |
| <li><a href="/documentation/io/">RDF I/O</a></li> |
| <li><a href="/documentation/rdfstar/index.html">RDF-star</a></li> |
| <li><a href="/documentation/shacl/index.html">SHACL</a></li> |
| <li><a href="/documentation/shex/index.html">ShEx</a></li> |
| <li><a href="/documentation/jdbc/index.html">SPARQL over JDBC</a></li> |
| <li><a href="/documentation/tdb/index.html">TDB</a></li> |
| <li><a href="/documentation/tdb2/index.html">TDB2</a></li> |
| <li><a href="/documentation/query/text-query.html">Text Search</a></li> |
| </ul> |
| </li> |
| |
| <li class="drop down"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown"><span class="glyphicon glyphicon-book"></span> Javadoc <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/documentation/javadoc.html">All Javadoc</a></li> |
| <li><a href="/documentation/javadoc/arq/">ARQ</a></li> |
| <li><a href="/documentation/javadoc_elephas.html">Elephas</a></li> |
| <li><a href="/documentation/javadoc/fuseki2/">Fuseki</a></li> |
| <li><a href="/documentation/javadoc/geosparql/">GeoSPARQL</a></li> |
| <li><a href="/documentation/javadoc/jdbc/">JDBC</a></li> |
| <li><a href="/documentation/javadoc/jena/">Jena Core</a></li> |
| <li><a href="/documentation/javadoc/permissions/">Permissions</a></li> |
| <li><a href="/documentation/javadoc/extras/querybuilder/">Query Builder</a></li> |
| <li><a href="/documentation/javadoc/shacl/">SHACL</a></li> |
| <li><a href="/documentation/javadoc/tdb/">TDB</a></li> |
| <li><a href="/documentation/javadoc/text/">Text Search</a></li> |
| </ul> |
| </li> |
| |
| <li id="ask"><a href="/help_and_support/index.html"><span class="glyphicon glyphicon-question-sign"></span> Ask</a></li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown"><span class="glyphicon glyphicon-bullhorn"></span> Get involved <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/getting_involved/index.html">Contribute</a></li> |
| <li><a href="/help_and_support/bugs_and_suggestions.html">Report a bug</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">Project</li> |
| <li><a href="/about_jena/about.html">About Jena</a></li> |
| <li><a href="/about_jena/architecture.html">Architecture</a></li> |
| <li><a href="/about_jena/citing.html">Citing</a></li> |
| <li><a href="/about_jena/team.html">Project team</a></li> |
| <li><a href="/about_jena/contributions.html">Related projects</a></li> |
| <li><a href="/about_jena/roadmap.html">Roadmap</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">ASF</li> |
| <li><a href="http://www.apache.org/">Apache Software Foundation</a></li> |
| <li><a href="http://www.apache.org/foundation/sponsorship.html">Become a Sponsor</a></li> |
| <li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li> |
| <li><a href="http://www.apache.org/security/">Security</a></li> |
| <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| </ul> |
| </li> |
| |
| |
| |
| |
| <li id="edit"><a href="https://github.com/apache/jena-site/edit/main/source/documentation/tdb/store-parameters.md" title="Edit this page on GitHub"><span class="glyphicon glyphicon-pencil"></span> Edit this page</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-12"> |
| <div id="breadcrumbs"> |
| |
|
|
|
|
|
|
|
|
|
|
|
|
| <ol class="breadcrumb">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <li><a href='/documentation'>DOCUMENTATION</a></li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <li><a href='/documentation/tdb'>TDB</a></li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <li class="active">STORE PARAMETERS</li>
|
|
|
|
|
|
|
|
|
| </ol>
|
|
|
|
|
|
|
| |
| |
| </div> |
| <h1 class="title">TDB - Store Parameters</h1> |
| |
| <p>TDB (as of version Jena 3.0.0) supports configuration of the |
| databases when they are first created and each time an application connects |
| to an existing database. Databases using the default settings built-into |
| TDB continue to work exactly as before.</p> |
| <h2 id="setting-store-parameters">Setting Store Parameters</h2> |
| <p>In TDB, there is exactly one internal object for each dataset in the JVM |
| and this is shared between all application datasets for that location of |
| persistent storage.</p> |
| <p>Setting store parameters is done by setting the internal system state |
| before any other access to the disk area occurs. It is not possible to have |
| different setups for the same dataset on disk.</p> |
| <p><code>StoreParams</code> are set by populating the internal state with the setup |
| before a application level dataset is created.</p> |
| <pre><code>TDBFactory.setup(Location location, StoreParams params) |
| </code></pre> |
| <p>This must be called before any application calls to get a <code>Dataset</code> (or |
| <code>DatasetGraph</code>) object otherwise <code>IllegalStateException</code> is thrown by this |
| function.</p> |
| <pre><code>Location location = ... ; |
| StoreParams customParams = ... ; |
| |
| TDBFactory.setup(location, customParams) ; |
| |
| Dataset ds = TDBFactory.createDataset(location) ; |
| ... |
| </code></pre> |
| <p>It is only possible to change store parameters by expelling the managed |
| storage by calling <code>TDBFactory.release(Location)</code>. This drops all caching. |
| Access to the dataset is then a cold start.</p> |
| <h2 id="per-connect-options">Per-connect Options</h2> |
| <p>The per-connect options are the ones that can be changed after the database has |
| been created and can be different each time the application attaches to the |
| database. A database can have at most one JVM attached to it (see Fuseki |
| to share a database).</p> |
| <p>These options do not affect the on-disk structures.</p> |
| <table> |
| <thead> |
| <tr> |
| <th>JSON key name</th> |
| <th align="right">Default value</th> |
| <th>Notes</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td>tdb.file_mode</td> |
| <td align="right">See below</td> |
| <td></td> |
| </tr> |
| <tr> |
| <td>tdb.node2nodeid_cache_size</td> |
| <td align="right">100,000</td> |
| <td>50,000 on 32 bit java</td> |
| </tr> |
| <tr> |
| <td>tdb.nodeid2node_cache_size</td> |
| <td align="right">500,000</td> |
| <td>50,000 on 32 bit java</td> |
| </tr> |
| <tr> |
| <td>tdb.node_miss_cache_size</td> |
| <td align="right">100</td> |
| <td></td> |
| </tr> |
| <tr> |
| <td>tdb.block_read_cache_size</td> |
| <td align="right">10000</td> |
| <td>Only in direct mode</td> |
| </tr> |
| <tr> |
| <td>tdb.block_write_cache_size</td> |
| <td align="right">2000</td> |
| <td>Only in direct mode</td> |
| </tr> |
| </tbody> |
| </table> |
| <h3 id="file-access---mapped-and-direct-modes">File access - “mapped” and “direct” modes</h3> |
| <p>TDB has two modes of operation for accessing block files - “mapped” and |
| “direct”.</p> |
| <ul> |
| <li> |
| <p>“mapped” uses memory mapped files and so the operating system is managing |
| caching, flexing the amount of memory for file system cache to balance |
| demands from other programmes on the same hardware.</p> |
| </li> |
| <li> |
| <p>“direct” using TDB’s own in-heap block caching. It avoids the problem that |
| addressing is limited to a total of about 1.5Gbytes on 32 bit Java.</p> |
| </li> |
| </ul> |
| <p>By default, TDB uses memory mapped files on 64 bit Java and its own file |
| caching on 32 bit java.</p> |
| <p>On Microsoft Windows, “mapped” databases can not be deleted while the JVM is running on MS |
| Windows. This is a <a href="http://bugs.java.com/view_bug.do?bug_id=4715154">known issue with Java</a>.</p> |
| <p>TDB databases are compatible across these file modes. There is no |
| difference to the file layouts. Memory mapped files may appear larger |
| because they contain unused space. Some utilities report this in file |
| size, some do not.</p> |
| <h3 id="caching-options">Caching options.</h3> |
| <p>These are the useful tuning options. Only the <tt>node*</tt> choices have |
| any effect when running in “mapped” mode.</p> |
| <p>All these options effect the amount of heap used. The block read/write |
| cache sizes are tuned to 32 bit Java.</p> |
| <p>Increasing the Node/NodeId cache sizes on 64 bit machines may be |
| beneficial.</p> |
| <h2 id="static-options">Static Options</h2> |
| <p>While it is possible to customize a database, this is considered to be |
| experimental. It is possible to corrupt, unrecoverable, existing databases |
| and create nonsense databases with inappropriate settings. It will be |
| useful in very few real situations. Not all combinations of index choices |
| will work. Only the standard layout is supported; alternative schemes are |
| for experimentation only.</p> |
| <h3 id="block-size">Block Size</h3> |
| <p>The block size can not be changed once a database has been created.</p> |
| <p>While the code attempts to detect block size mismatches, in order to retain |
| compatibility with existing database, the testing can not be perfect. If |
| undetected, any update will permanently and irrecoverably damage the |
| database.</p> |
| <h2 id="store-parameters-file-format">Store Parameters File Format</h2> |
| <p>JSON is used for the on-disk record of store parameters, see the example |
| below. Unspecified options defaults to the for the running setup.</p> |
| <p>These are default settings for a 64 bit Java:</p> |
| <pre> |
| { |
| "tdb.file_mode" : "mapped" , |
| "tdb.block_size" : 8192 , |
| "tdb.block_read_cache_size" : 10000 , |
| "tdb.block_write_cache_size" : 2000 , |
| "tdb.node2nodeid_cache_size" : 100000 , |
| "tdb.nodeid2node_cache_size" : 500000 , |
| "tdb.node_miss_cache_size" : 100 , |
| "tdb.index_node2id" : "node2id" , |
| "tdb.index_id2node" : "nodes" , |
| "tdb.triple_index_primary" : "SPO" , |
| "tdb.triple_indexes" : [ "SPO" , "POS" , "OSP" ] , |
| "tdb.quad_index_primary" : "GSPO" , |
| "tdb.quad_indexes" : [ "GSPO" , "GPOS" , "GOSP" , "POSG" , "OSPG" , "SPOG" ] , |
| "tdb.prefix_index_primary" : "GPU" , |
| "tdb.prefix_indexes" : [ "GPU" ] , |
| "tdb.file_prefix_index" : "prefixIdx" , |
| "tdb.file_prefix_nodeid" : "prefix2id" , |
| "tdb.file_prefix_id2node" : "prefixes" |
| } |
| </pre> |
| <h2 id="choosing-the-store-parameters">Choosing the store parameters</h2> |
| <p>This is the policy applied when creating or reattaching to a database.</p> |
| <p>If the database location has a parameter file, <code>tdb.cfg</code> then use that. |
| This is modified by any dynamic options supplied by the application. So to |
| create a specialized database, one way to do that is to create an empty |
| directory and put a <code>tdb.cfg</code> in place.</p> |
| <p>If there is no parameter file and this is a new database, use the |
| application provided store parameters, or if there are no application |
| provided parameters, use the system default parameters. If application |
| supplied parameters are used, write a <code>tdb.cfg</code> file.</p> |
| <p>Finally, if this is an existing database, with no <code>tdb.cfg</code>, use the system |
| default modified by any application parameters.</p> |
| <p>In other words, if there is no <code>tdb.cfg</code> assume the system defaults, except |
| when creating a database.</p> |
| <p><em>Modification</em> involves taking one set of store parameters and applying any |
| dynamic parameters set in the second set. Only explicitly set dynamic |
| parameters modify the original.</p> |
| |
| |
| </div> |
| </div> |
| |
| </div> |
| |
| <footer class="footer"> |
| <div class="container" style="font-size:80%" > |
| <p> |
| Copyright © 2011–2022 The Apache Software Foundation, Licensed under the |
| <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>. |
| </p> |
| <p> |
| Apache Jena, Jena, the Apache Jena project logo, Apache and the Apache feather logos are trademarks of |
| The Apache Software Foundation. |
| <br/> |
| <a href="https://privacy.apache.org/policies/privacy-policy-public.html" |
| >Apache Software Foundation Privacy Policy</a>. |
| </p> |
| </div> |
| </footer> |
| |
| |
| <script type="text/javascript"> |
| var link = $('a[href="' + this.location.pathname + '"]'); |
| if (link != undefined) |
| link.parents('li,ul').addClass('active'); |
| </script> |
| |
| </body> |
| </html> |
