blob: 128407474542314841612e35a9fe34b43f03ec37 [file] [log] [blame]
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="author" content="dev@gora.apache.org" />
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8" />
<meta name="Description" content="Apache Gora -- Gora Kudu Module" />
<meta name="Keywords" content="Apache Gora NoSQL Framework" />
<meta name="Owner" content="dev@gora.apache.org" />
<meta name="Robots" content="index, follow" />
<meta name="Security" content="Public" />
<meta name="Source" content="wiki template" />
<meta
name="DC.Rights"
content="Copyright 2010-2024, The Apache Software Foundation"
/>
<link href="/resources/css/bootstrap.min.css" rel="stylesheet" />
<!-- Fav and touch icons -->
<link
rel="apple-touch-icon-precomposed"
sizes="144x144"
href="http://twitter.github.com/bootstrap/assets/ico/apple-touch-icon-144-precomposed.png"
/>
<link
rel="apple-touch-icon-precomposed"
sizes="114x114"
href="http://twitter.github.com/bootstrap/assets/ico/apple-touch-icon-114-precomposed.png"
/>
<link
rel="apple-touch-icon-precomposed"
sizes="72x72"
href="http://twitter.github.com/bootstrap/assets/ico/apple-touch-icon-72-precomposed.png"
/>
<link
rel="apple-touch-icon-precomposed"
href="http://twitter.github.com/bootstrap/assets/ico/apple-touch-icon-57-precomposed.png"
/>
<link rel="shortcut icon" href="/resources/img/feather-small.png" />
<title>Apache Gora&trade; - Gora Kudu Module</title>
</head>
<body style="padding-top: 100px">
<nav class="navbar navbar-expand-lg navbar-dark bg-dark fixed-top shadow-lg">
<div class="container-fluid">
<a class="navbar-brand" href="/index.html"
><img
src="/resources/img/gora-logo.png"
alt="Apache Gora"
title="Apache Gora"
height="50px"
/></a>
<button
class="navbar-toggler"
type="button"
data-bs-toggle="collapse"
data-bs-target="#navbarNav"
aria-controls="navbarNav"
aria-expanded="false"
aria-label="Toggle navigation"
>
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarNav">
<ul class="navbar-nav me-auto">
<li class="nav-item">
<a class="nav-link" href="/downloads.html">Downloads</a>
</li>
<li class="nav-item dropdown">
<a
class="nav-link dropdown-toggle"
href="#"
id="navbarDropdown1"
role="button"
data-bs-toggle="dropdown"
aria-expanded="false"
>Community</a
>
<ul class="dropdown-menu" aria-labelledby="navbarDropdown1">
<li>
<a
class="dropdown-item"
href="https://whimsy.apache.org/board/minutes/Gora.html"
>Board Reporting</a
>
</li>
<li>
<a class="dropdown-item" href="/contribute.html"
>Contribute</a
>
</li>
<li>
<a class="dropdown-item" href="/mailing_lists.html"
>Mailing Lists</a
>
</li>
<li>
<a class="dropdown-item" href="/credits.html">People</a>
</li>
<li>
<a class="dropdown-item" href="/related.html"
>Related Projects</a
>
</li>
</ul>
</li>
<li class="nav-item dropdown">
<a
class="nav-link dropdown-toggle"
href="#"
id="navbarDropdown2"
role="button"
data-bs-toggle="dropdown"
aria-expanded="false"
>Documentation</a
>
<ul class="dropdown-menu" aria-labelledby="navbarDropdown2">
<li><a class="dropdown-item" href="/about.html">About</a></li>
<li>
<a class="dropdown-item" href="/current/index.html"
>Current Documentation</a
>
</li>
<li>
<a class="dropdown-item" href="/current/api/javadoc.html"
>JavaDoc Documentation</a
>
</li>
<li>
<a class="dropdown-item" href="/current/tutorial.html"
>Gora Tutorial</a
>
</li>
<li>
<a
class="dropdown-item"
href="https://cwiki.apache.org/confluence/display/GORA/"
>Gora Wiki</a
>
</li>
</ul>
</li>
<li class="nav-item dropdown">
<a
class="nav-link dropdown-toggle"
href="#"
id="navbarDropdown3"
role="button"
data-bs-toggle="dropdown"
aria-expanded="false"
>Development</a
>
<ul class="dropdown-menu" aria-labelledby="navbarDropdown3">
<li>
<a
class="dropdown-item"
href="https://issues.apache.org/jira/browse/GORA"
>Issue Tracking</a
>
</li>
<li>
<a class="dropdown-item" href="/mailing_lists.html"
>Mailing Lists</a
>
</li>
<li>
<a class="dropdown-item" href="/version_control.html"
>Version Control</a
>
</li>
<li>
<a class="dropdown-item" href="/roadmap.html">Roadmap</a>
</li>
</ul>
</li>
<li class="nav-item dropdown">
<a
class="nav-link dropdown-toggle"
href="#"
id="navbarDropdown4"
role="button"
data-bs-toggle="dropdown"
aria-expanded="false"
>
<img
src="/resources/img/feather-small.png"
alt="Apache"
title="Apache"
/>
</a>
<ul class="dropdown-menu" aria-labelledby="navbarDropdown4">
<li>
<a class="dropdown-item" href="http://www.apache.org"
>Apache Home</a
>
</li>
<li>
<a
class="dropdown-item"
href="http://www.apache.org/licenses/"
>Apache License</a
>
</li>
<li>
<a
class="dropdown-item"
href="http://www.apache.org/security/"
>Security</a
>
</li>
<li>
<a
class="dropdown-item"
href="http://www.apache.org/foundation/sponsorship.html"
>Support</a
>
</li>
<li>
<a
class="dropdown-item"
href="http://www.apache.org/foundation/thanks.html"
>Thanks</a
>
</li>
</ul>
</li>
</ul>
</div>
</div>
</nav>
<div class="container top-buffer" id="Gora_Gora Kudu Module">
<h2 id="overview">Overview<a class="headerlink" href="#overview" title="Permalink">&para;</a></h2>
<p>This is the main documentation for the gora-kudu module. <b>gora-kudu</b> module enables <a href="https://kudu.apache.org">Apache Kudu</a> backend support for Gora.</p>
<div id="toc"><ul><li><a class="toc-href" href="#gora-kudu-properties-goraproperties" title="Gora Kudu Properties - gora.properties">Gora Kudu Properties - gora.properties</a></li><li><a class="toc-href" href="#gora-kudu-mappings-gora-kudu-mappingxml" title="Gora Kudu mappings - gora-kudu-mapping.xml">Gora Kudu mappings - gora-kudu-mapping.xml</a><ul><li><a class="toc-href" href="#class" title="Class">Class</a></li><li><a class="toc-href" href="#table" title="Table">Table</a></li></ul></li><li><a class="toc-href" href="#supported-data-types" title="Supported Data types">Supported Data types</a></li></ul></div>
<h2 id="gora-kudu-properties-goraproperties">Gora Kudu Properties - gora.properties<a class="headerlink" href="#gora-kudu-properties-goraproperties" title="Permalink">&para;</a></h2>
<ul>
<li><code>gora.datastore.default=org.apache.gora.kudu.store.KuduStore</code> - Implementation of the persistent Java storage class for Kudu</li>
<li><code>gora.datastore.kudu.masterAddresses=localhost:7051</code> - Comma-separated list of "host:port" pairs of the Kudu masters</li>
<li><code>gora.datastore.kudu.flushMode=AUTO_FLUSH_SYNC</code> - Flush mode for the Kudu-client session. More details here: <a href="https://kudu.apache.org/apidocs/org/apache/kudu/client/SessionConfiguration.FlushMode.html">https://kudu.apache.org/apidocs/org/apache/kudu/client/SessionConfiguration.FlushMode.html</a>.</li>
<li><code>gora.datastore.kudu.flushInterval=1000</code> - Flush interval in ms, which will be used for the next scheduling.</li>
<li><code>gora.datastore.kudu.bossCount=1</code> - Maximum number of boss threads. Optional. If not provided, 1 is used.</li>
<li><code>gora.datastore.kudu.defaultAdminOperationTimeoutMs=1000</code> - Default timeout used for administrative operations (e.g. createTable, deleteTable, etc). Optional. If not provided, defaults to 30s. A value of 0 disables the timeout.</li>
<li><code>gora.datastore.kudu.defaultOperationTimeoutMs=1000</code> - Default timeout used for user operations (using sessions and scanners). Optional. If not provided, defaults to 30s. A value of 0 disables the timeout.</li>
<li><code>gora.datastore.kudu.defaultSocketReadTimeoutMs=10000</code> - Default timeout to use when waiting on data from a socket. Optional. If not provided, defaults to 10s. A value of 0 disables the timeout.</li>
<li><code>gora.datastore.kudu.clientStatistics=true</code> - Client's collection of statistics. Statistics are enabled by default.</li>
<li><code>gora.datastore.kudu.workerCount=2</code> - Maximum number of worker threads. Optional. If not provided, (2 * the number of available processors) is used.</li>
</ul>
<h2 id="gora-kudu-mappings-gora-kudu-mappingxml">Gora Kudu mappings - gora-kudu-mapping.xml<a class="headerlink" href="#gora-kudu-mappings-gora-kudu-mappingxml" title="Permalink">&para;</a></h2>
<p>You should then create a gora-kudu-mapping.xml which will describe how you want to store each of your Gora persistent objects and which primary keys you want to use:</p>
<pre><code>&lt;gora-otd&gt;
&lt;table name ="Employee"&gt;
&lt;primaryKey column="pkssn" type="STRING" /&gt;
&lt;hashPartition numBuckets="8"/&gt;
&lt;/table&gt;
&lt;class name="org.apache.gora.examples.generated.Employee" keyClass="java.lang.String" table="Employee"&gt;
&lt;field name="ssn" column="ssn" type="STRING"/&gt;
&lt;field name="value" column="value" type="STRING"/&gt;
&lt;field name="name" column="name" type="STRING"/&gt;
&lt;field name="dateOfBirth" column="dateOfBirth" type="INT64"/&gt;
&lt;field name="salary" column="salary" type="INT32"/&gt;
&lt;field name="boss" column="boss" type="BINARY"/&gt;
&lt;field name="webpage" column="webpage" type="BINARY"/&gt;
&lt;/class&gt;
&lt;/gora-otd&gt;
</code></pre>
<p>Here you can see that we require the definition of two child elements within the <code>gora-otd</code> mapping configuration: a class and a table.</p>
<h3 id="class">Class<a class="headerlink" href="#class" title="Permalink">&para;</a></h3>
<p>Each <b>class</b> element should contain the following elements;</p>
<ol>
<li>
<p>a parameter defining the Persistent class name e.g. <b>org.apache.gora.examples.generated.Employee</b>,</p>
</li>
<li>
<p>a parameter defining the keyClass e.g. <b>java.lang.String</b> which specifies the key which maps to the field values,</p>
</li>
<li>
<p>a parameter defining the table e.g. <b>Employee</b> which will be used to persist each Gora object</p>
</li>
</ol>
<p>In addition, within the class element we should define the fields to be mapped (field tag). The fields elements define the actual mapping between persistent object's attributes and the table's columns. These mapping have three customizable parameters: name which correspond to the object attribute's name. column which defines the column's name of the table to be associated with the attribute. And type which defines the data type of that column.</p>
<ul>
<li>
<p>The DECIMAL data type is parameterizable, so two additional attributes might be used in the XML mapping, scale and precision. Example:</p>
<pre><code> &lt;field name="value" column="value" type="DECIMAL" scale=&rdquo;1&rdquo; precision=&rdquo;1&rdquo;/&gt;
</code></pre>
</li>
</ul>
<h3 id="table">Table<a class="headerlink" href="#table" title="Permalink">&para;</a></h3>
<p>Each <b>table</b> element should contain the following elements;</p>
<ol>
<li>
<p>A <strong><code>primaryKey</code></strong> element defining which column is used by Kudu to identify the records stored in the DataStore. It has two attributes: <strong><code>column</code></strong> which defines the column's name of the table to be used as identifier for the records, and <strong><code>type</code></strong> which defines the data type of the aforementioned column.</p>
</li>
<li>
<p>A partition definition for the table, which could be either a hash partition or a range partition. For more information about Partitioning in Kudu refer to <a href="https://kudu.apache.org/docs/schema_design.html#partitioning">https://kudu.apache.org/docs/schema_design.html#partitioning</a></p>
</li>
</ol>
<p>asdhfjalkjhf</p>
<p>Example for Hash partition:</p>
<pre><code> &lt;hashPartition numBuckets="8"/&gt;
</code></pre>
<p>The numBuckets attribute specifies the number of hash buckets to be used as partitions.</p>
<p>Example for Range partition:</p>
<pre><code> &lt;rangePartition lower="" upper ="1000"/&gt;
&lt;rangePartition lower="1000" upper =""/&gt;
</code></pre>
<p>The lower and upper attributes define the boundaries of each partition. This partition is applied only on the primary key column of the table. Take into consideration that the lower bound is inclusive and the upper one is not.</p>
<h2 id="supported-data-types">Supported Data types<a class="headerlink" href="#supported-data-types" title="Permalink">&para;</a></h2>
<p>Description of supported <b>type</b> values:</p>
<table class="table">
<thead>
<tr>
<th>Kudu datatype</th>
<th>Java datatype</th>
</tr>
</thead>
<tbody>
<tr>
<td>BOOL</td>
<td>boolean</td>
</tr>
<tr>
<td>INT8</td>
<td>byte</td>
</tr>
<tr>
<td>INT32</td>
<td>short</td>
</tr>
<tr>
<td>INT16</td>
<td>int</td>
</tr>
<tr>
<td>INT64</td>
<td>long</td>
</tr>
<tr>
<td>UNIXTIME_MICROS</td>
<td>java.sql.Timestamp</td>
</tr>
<tr>
<td>FLOAT</td>
<td>float</td>
</tr>
<tr>
<td>DOUBLE</td>
<td>double</td>
</tr>
<tr>
<td>DECIMAL</td>
<td>BigDecimal</td>
</tr>
<tr>
<td>STRING</td>
<td>String</td>
</tr>
<tr>
<td>BINARY</td>
<td>byte[]</td>
</tr></tbody></table>
<p>For more details about supported data types refer to <a href="https://kudu.apache.org/docs/schema_design.html#column-design">https://kudu.apache.org/docs/schema_design.html#column-design</a></p>
</div>
<!-- /container (main block) -->
<hr />
<div class="container">
<footer>
<p>
Copyright © 2010-2024 The Apache Software Foundation.
Licensed under
<a href="http://www.apache.org/licenses/LICENSE-2.0"
>Apache License 2.0</a
>.
</p>
<p>
Apache Gora, Gora, Apache, the Apache feather logo, and the Apache
Gora project logo are trademarks of The Apache Software Foundation.
</p>
</footer>
</div>
<!-- /container -->
<script src="/resources/js/bootstrap.bundle.min.js"></script>
<script type="text/javascript">
stLight.options({
publisher: "4059fafd-3891-49f9-8c96-e4100290d8e6",
doNotHash: false,
doNotCopy: false,
hashAddressBar: false,
});
</script>
<script src="//cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.0.1/build/highlight.min.js"></script>
<script>
hljs.highlightAll();
</script>
</body>
</html>