| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| |
| |
| <title>Apache Jena - TDB2 - Command Line Tools</title> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <link href="/css/bootstrap.min.css" rel="stylesheet" media="screen"> |
| <link href="/css/bootstrap-extension.css" rel="stylesheet" type="text/css"> |
| <link href="/css/jena.css" rel="stylesheet" type="text/css"> |
| <link rel="shortcut icon" href="/images/favicon.ico" /> |
| |
| <script src="https://code.jquery.com/jquery-2.2.4.min.js" |
| integrity="sha256-BbhdlvQf/xTY9gja0Dq3HiwQF8LaCRTXxZKRutelT44=" |
| crossorigin="anonymous"></script> |
| <script src="/js/jena-navigation.js" type="text/javascript"></script> |
| <script src="/js/bootstrap.min.js" type="text/javascript"></script> |
| |
| <script src="/js/improve.js" type="text/javascript"></script> |
| |
| |
| </head> |
| |
| <body> |
| |
| <nav class="navbar navbar-default" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-ex1-collapse"> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="/index.html"> |
| <img class="logo-menu" src="/images/jena-logo/jena-logo-notext-small.png" alt="jena logo">Apache Jena</a> |
| </div> |
| |
| <div class="collapse navbar-collapse navbar-ex1-collapse"> |
| <ul class="nav navbar-nav"> |
| <li id="homepage"><a href="/index.html"><span class="glyphicon glyphicon-home"></span> Home</a></li> |
| <li id="download"><a href="/download/index.cgi"><span class="glyphicon glyphicon-download-alt"></span> Download</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown"><span class="glyphicon glyphicon-book"></span> Learn <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li class="dropdown-header">Tutorials</li> |
| <li><a href="/tutorials/index.html">Overview</a></li> |
| <li><a href="/documentation/fuseki2/index.html">Fuseki Triplestore</a></li> |
| <li><a href="/documentation/notes/index.html">How-To's</a></li> |
| <li><a href="/documentation/query/manipulating_sparql_using_arq.html">Manipulating SPARQL using ARQ</a></li> |
| <li><a href="/tutorials/rdf_api.html">RDF core API tutorial</a></li> |
| <li><a href="/tutorials/sparql.html">SPARQL tutorial</a></li> |
| <li><a href="/tutorials/using_jena_with_eclipse.html">Using Jena with Eclipse</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">References</li> |
| <li><a href="/documentation/index.html">Overview</a></li> |
| <li><a href="/documentation/query/index.html">ARQ (SPARQL)</a></li> |
| <li><a href="/documentation/assembler/index.html">Assembler</a></li> |
| <li><a href="/documentation/tools/index.html">Command-line tools</a></li> |
| <li><a href="/documentation/rdfs/">Data with RDFS Inferencing</a></li> |
| <li><a href="/documentation/geosparql/index.html">GeoSPARQL</a></li> |
| <li><a href="/documentation/inference/index.html">Inference API</a></li> |
| <li><a href="/documentation/javadoc.html">Javadoc</a></li> |
| <li><a href="/documentation/ontology/">Ontology API</a></li> |
| <li><a href="/documentation/permissions/index.html">Permissions</a></li> |
| <li><a href="/documentation/extras/querybuilder/index.html">Query Builder</a></li> |
| <li><a href="/documentation/rdf/index.html">RDF API</a></li> |
| <li><a href="/documentation/rdfconnection/">RDF Connection - SPARQL API</a></li> |
| <li><a href="/documentation/io/">RDF I/O</a></li> |
| <li><a href="/documentation/rdfstar/index.html">RDF-star</a></li> |
| <li><a href="/documentation/shacl/index.html">SHACL</a></li> |
| <li><a href="/documentation/shex/index.html">ShEx</a></li> |
| <li><a href="/documentation/jdbc/index.html">SPARQL over JDBC</a></li> |
| <li><a href="/documentation/tdb/index.html">TDB</a></li> |
| <li><a href="/documentation/tdb2/index.html">TDB2</a></li> |
| <li><a href="/documentation/query/text-query.html">Text Search</a></li> |
| </ul> |
| </li> |
| |
| <li class="drop down"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown"><span class="glyphicon glyphicon-book"></span> Javadoc <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/documentation/javadoc.html">All Javadoc</a></li> |
| <li><a href="/documentation/javadoc/arq/">ARQ</a></li> |
| <li><a href="/documentation/javadoc_elephas.html">Elephas</a></li> |
| <li><a href="/documentation/javadoc/fuseki2/">Fuseki</a></li> |
| <li><a href="/documentation/javadoc/geosparql/">GeoSPARQL</a></li> |
| <li><a href="/documentation/javadoc/jdbc/">JDBC</a></li> |
| <li><a href="/documentation/javadoc/jena/">Jena Core</a></li> |
| <li><a href="/documentation/javadoc/permissions/">Permissions</a></li> |
| <li><a href="/documentation/javadoc/extras/querybuilder/">Query Builder</a></li> |
| <li><a href="/documentation/javadoc/shacl/">SHACL</a></li> |
| <li><a href="/documentation/javadoc/tdb/">TDB</a></li> |
| <li><a href="/documentation/javadoc/text/">Text Search</a></li> |
| </ul> |
| </li> |
| |
| <li id="ask"><a href="/help_and_support/index.html"><span class="glyphicon glyphicon-question-sign"></span> Ask</a></li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown"><span class="glyphicon glyphicon-bullhorn"></span> Get involved <b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="/getting_involved/index.html">Contribute</a></li> |
| <li><a href="/help_and_support/bugs_and_suggestions.html">Report a bug</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">Project</li> |
| <li><a href="/about_jena/about.html">About Jena</a></li> |
| <li><a href="/about_jena/architecture.html">Architecture</a></li> |
| <li><a href="/about_jena/citing.html">Citing</a></li> |
| <li><a href="/about_jena/team.html">Project team</a></li> |
| <li><a href="/about_jena/contributions.html">Related projects</a></li> |
| <li><a href="/about_jena/roadmap.html">Roadmap</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">ASF</li> |
| <li><a href="http://www.apache.org/">Apache Software Foundation</a></li> |
| <li><a href="http://www.apache.org/foundation/sponsorship.html">Become a Sponsor</a></li> |
| <li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li> |
| <li><a href="http://www.apache.org/security/">Security</a></li> |
| <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| </ul> |
| </li> |
| |
| |
| |
| |
| <li id="edit"><a href="https://github.com/apache/jena-site/edit/main/source/documentation/tdb2/tdb2_cmds.md" title="Edit this page on GitHub"><span class="glyphicon glyphicon-pencil"></span> Edit this page</a></li> |
| </ul> |
| </div> |
| </div> |
| </nav> |
| |
| |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-12"> |
| <div id="breadcrumbs"> |
| |
|
|
|
|
|
|
|
|
|
|
|
|
| <ol class="breadcrumb">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <li><a href='/documentation'>DOCUMENTATION</a></li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <li><a href='/documentation/tdb2'>TDB2</a></li>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| <li class="active">TDB2 CMDS</li>
|
|
|
|
|
|
|
|
|
| </ol>
|
|
|
|
|
|
|
| |
| |
| </div> |
| <h1 class="title">TDB2 - Command Line Tools</h1> |
| |
| <p><em>TDB2 is not compatible with TDB1</em></p> |
| <p>Do not run TDB1 tools on a TDB2 database, nor run TDB2 tools on a TDB1 database.</p> |
| <p>These scripts are available jena binary distribution.</p> |
| <ul> |
| <li><code>tdb2.tdbbackup</code></li> |
| <li><code>tdb2.tdbdump</code></li> |
| <li><code>tdb2.tdbcompact</code></li> |
| <li><code>tdb2.tdbloader</code></li> |
| <li><code>tdb2.tdbquery</code></li> |
| <li><code>tdb2.tdbupdate</code></li> |
| </ul> |
| <p>On MS Windows, these commands are called <code>tdb2_tdbquery</code> etc.</p> |
| <p>Example usage:</p> |
| <pre><code>tdb2.tdbloader --loc <DB location> file1 file2 ... |
| </code></pre><p>Note:</p> |
| <p><code>tdbloader2</code> is a TDB1 command tool.</p> |
| <h2 id="tdb2tdbloader"><code>tdb2.tdbloader</code></h2> |
| <p>Basic usage: load files into a database at location “DB”:</p> |
| <pre><code>tdb2.tdbloader --loc DB file1 file2 .... |
| </code></pre> |
| <p>To load the data into a named graph, use the <code>--graph=IRI</code> argument:</p> |
| <pre><code>tdb2.tdbloader --loc DB --graph=https://example.org/graph#name file1 |
| </code></pre> |
| <p>For the complete syntax and list of all arguments use <code>--help</code>:</p> |
| <pre><code>tdb2.tdbloader --help |
| </code></pre> |
| <p>All TDB2 loaders can update datasets and do not have to work on an empty |
| dataset. However, only the basic and sequential loader are fully |
| transactional in the presence of crashes. The other loaders, while |
| faster, work by manipulating the low-level datastructures, and are tuned |
| for large changes of data. They do not provide perfect transaction |
| isolation in case a load goes wrong for some reason. The multiphase |
| loading operations use partial transactions which can leave the database |
| in a strange state.</p> |
| <p>When working with large data to load, it is advisable to check it |
| completely first with <code>riot --validate</code>. Parse errors during loading can |
| lead to inconsistent indexing. Fixing bad data, even if legal RDF, such |
| as bad lexical forms of literals or bad URIs, is much easier before the |
| data is in the database.</p> |
| <p>Because loading in hardware dependent, the right choice for any |
| situation can only be found by trying each loader to see what works best |
| and the notes below are only initial guidance. The default choice is |
| a reasonable starting point. Closing all applications to release their |
| memory and not use CPU improves the loading process performance.</p> |
| <p>Loading very large datasets (like Wikidata) with tdb2.tdbloader may |
| sometimes on linux configurations fail with errors like:</p> |
| <pre><code>Native memory allocation (mmap) failed to map 65536 bytes for |
| committing reserved memory. |
| </code></pre> |
| <p>This can be avoided by adding a larger value to the <code>vm.max_map_count</code> |
| option. The command <code>sudo sysctl -w vm.max_map_count=262144</code> updates |
| the value for your current session, or you can persist the change by |
| editing the value in <code>/etc/sysctl.conf</code> or in <code>/etc/sysctl.d/*</code> override |
| files if available.</p> |
| <h3 id="loader-options">Loader options</h3> |
| <p>The choice of loader is given by the optional <code>--loader</code> argument.</p> |
| <p><code>--loader=basic</code></p> |
| <p>The basic loader loads data as a single transaction into the dataset on |
| a single thread. It is suitable for small data and also for |
| incrementally adding to a dataset safely, A machine crash while running |
| this loader will not invalidate the database; the load simply will not happen.</p> |
| <p><code>--loader=sequential</code></p> |
| <p>The sequential loader is a single threaded loader that loads the primary |
| index then each of the other indexes. It is suitable only for low resource |
| hardware, especially in a low I/O bandwidth situation.</p> |
| <p><code>--loader=phased</code> (default)</p> |
| <p>The phased loader, the default if no <code>--loader</code> argument is provided, |
| is balance between performance and hardware demands.</p> |
| <p>It used multiple threads for both the initial loading (3 worker threads) |
| and then 2 threads in parallel for building the other indexes.</p> |
| <p><code>--loader=parallel</code></p> |
| <p>The parallel loader runs all operations at once. It can deliver the best |
| performance providing enough RAM is available and the persistent storage |
| is SSD. It can consume all hardware resources, greatly impacting |
| any other applications running.</p> |
| |
| |
| </div> |
| </div> |
| |
| </div> |
| |
| <footer class="footer"> |
| <div class="container" style="font-size:80%" > |
| <p> |
| Copyright © 2011–2022 The Apache Software Foundation, Licensed under the |
| <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>. |
| </p> |
| <p> |
| Apache Jena, Jena, the Apache Jena project logo, Apache and the Apache feather logos are trademarks of |
| The Apache Software Foundation. |
| <br/> |
| <a href="https://privacy.apache.org/policies/privacy-policy-public.html" |
| >Apache Software Foundation Privacy Policy</a>. |
| </p> |
| </div> |
| </footer> |
| |
| |
| <script type="text/javascript"> |
| var link = $('a[href="' + this.location.pathname + '"]'); |
| if (link != undefined) |
| link.parents('li,ul').addClass('active'); |
| </script> |
| |
| </body> |
| </html> |
