| --- |
| title: Apache Kudu 1.3.0 Release Notes |
| layout: default |
| active_nav: docs |
| last_updated: 'Last updated 2017-03-10 12:47:33 PST' |
| --- |
| <!-- |
| |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| |
| |
| <div class="container"> |
| <div class="row"> |
| <div class="col-md-9"> |
| |
| <h1>Apache Kudu 1.3.0 Release Notes</h1> |
| <div class="sect1"> |
| <h2 id="rn_1.3.0_new_features"><a class="link" href="#rn_1.3.0_new_features">New features</a></h2> |
| <div class="sectionbody"> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>Kudu 1.3 adds support for strong authentication based on Kerberos. This optional feature |
| allows users to authenticate themselves using Kerberos tickets, and also provides |
| mutual authentication of servers using Kerberos credentials stored in keytabs. This |
| feature is optional, but recommended for deployments requiring security.</p> |
| </li> |
| <li> |
| <p>Kudu 1.3 adds support for encryption of data on the network using Transport Layer Security |
| (TLS). Kudu will now use TLS to encrypt all network traffic between clients and servers as |
| well as any internal traffic among servers, with the exception of traffic determined to |
| be within a localhost network connection. Encryption is enabled by default whenever it can |
| be determined that both the client and server support the feature.</p> |
| </li> |
| <li> |
| <p>Kudu 1.3 adds coarse-grained service-level authorization of access to the cluster. |
| The operator may set up lists of permitted users who may act as administrators and |
| as clients of the cluster. Combined with the strong authentication feature described |
| above, this can enable a secure environment for some use cases. Note that fine-grained |
| access control (e.g. table-level or column-level) is not yet supported.</p> |
| </li> |
| <li> |
| <p>Kudu 1.3 adds a background task to tablet servers which removes historical versions of |
| data which have fallen behind the configured data retention time. This reduces disk space |
| usage in all workloads, but particularly in those with a higher volume of updates or |
| upserts.</p> |
| </li> |
| <li> |
| <p>Kudu now incorporates Google Breakpad, a library which writes crash reports in |
| the case of a server crash. These reports can be found within the configured log directory, |
| and can be useful during bug diagnosis.</p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="_optimizations_and_improvements"><a class="link" href="#_optimizations_and_improvements">Optimizations and improvements</a></h2> |
| <div class="sectionbody"> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>Kudu servers will now change the file permissions of data directories and contained |
| data files based on a new configuration flag <code>--umask</code>. As a result, after upgrading, |
| permissions on disk may be more restrictive than in previous versions. The new default |
| configuration improves data security.</p> |
| </li> |
| <li> |
| <p>Kudu’s web UI will now redact strings which may include sensitive user data. For example, |
| the monitoring page which shows in-progress scans no longer includes the scanner predicate |
| values. The tracing and RPC diagnostics endpoints no longer include contents of RPCs which |
| may include table data.</p> |
| </li> |
| <li> |
| <p>By default, Kudu now reserves 1% of each configured data volume as free space. If a volume |
| is seen to have less than 1% of disk space free, Kudu will stop writing to that volume |
| to avoid completely filling up the disk.</p> |
| </li> |
| <li> |
| <p>The default encoding for numeric columns (int, float, and double) has been changed |
| to <code>BIT_SHUFFLE</code>. The default encoding for binary and string columns has been |
| changed to <code>DICT_ENCODING</code>. Dictionary encoding automatically falls back to the old |
| default (<code>PLAIN</code>) when cardinality is too high to be effectively encoded.</p> |
| <div class="paragraph"> |
| <p>These new defaults match the default behavior of other storage mechanisms such as |
| Apache Parquet and are likely to perform better out of the box.</p> |
| </div> |
| </li> |
| <li> |
| <p>Kudu now uses <code>LZ4</code> compression when writing its Write Ahead Log (WAL). This improves |
| write performance and stability for many use cases.</p> |
| </li> |
| <li> |
| <p>Kudu now uses <code>LZ4</code> compression when writing delta files. This can improve both |
| read and write performance as well as save substantial disk usage, especially |
| for workloads involving a high number of updates or upserts containing compressible |
| data.</p> |
| </li> |
| <li> |
| <p>The Kudu API now supports the ability to express <code>IS NULL</code> and <code>IS NOT NULL</code> predicates |
| on scanners. The Spark DataSource integration will take advantage of these new |
| predicates when possible.</p> |
| </li> |
| <li> |
| <p>Both C++ and Java clients have been optimized to prune partitions more effectively |
| when performing scans using the <code>IN (…​)</code> predicate.</p> |
| </li> |
| <li> |
| <p>The exception messages produced by the Java client are now truncated to a maximum length |
| of 32KB.</p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="rn_1.3.0_fixed_issues"><a class="link" href="#rn_1.3.0_fixed_issues">Fixed Issues</a></h2> |
| <div class="sectionbody"> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1893">KUDU-1893</a> |
| Fixed a critical bug in which wrong results would be returned when evaluating |
| predicates applied to columns added using the <code>ALTER TABLE</code> operation.</p> |
| </li> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1905">KUDU-1905</a> |
| Fixed a crash after inserting a row sharing a primary key with a recently-deleted |
| row in tables where the primary key is comprised of all of the columns.</p> |
| </li> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1899">KUDU-1899</a> |
| Fixed a crash after inserting a row with an empty string as the single-column |
| primary key.</p> |
| </li> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1904">KUDU-1904</a> |
| Fixed a potential crash when performing random reads against a column using RLE |
| encoding and containing long runs of NULL values.</p> |
| </li> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1853">KUDU-1853</a> |
| Fixed an issue where disk space could be leaked on servers which experienced an error |
| during the process of copying tablet data from another server.</p> |
| </li> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1856">KUDU-1856</a> |
| Fixed an issue in which disk space could be leaked by Kudu servers storing data on |
| partitions using the XFS file system. Any leaked disk space will be automatically |
| recovered upon upgrade.</p> |
| </li> |
| <li> |
| <p><a href="https://issues.apache.org/jira/browse/KUDU-1888">KUDU-1888</a>, |
| <a href="https://issues.apache.org/jira/browse/KUDU-1906">KUDU-1906</a> |
| Fixed multiple issues in the Java client where operation callbacks would never be |
| triggered, causing the client to hang.</p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="rn_1.3.0_wire_compatibility"><a class="link" href="#rn_1.3.0_wire_compatibility">Wire Protocol compatibility</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>Kudu 1.3.0 is wire-compatible with previous versions of Kudu:</p> |
| </div> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>Kudu 1.3 clients may connect to servers running Kudu 1.0. If the client uses features |
| that are not available on the target server, an error will be returned.</p> |
| </li> |
| <li> |
| <p>Kudu 1.0 clients may connect to servers running Kudu 1.3 with the exception of the |
| below-mentioned restrictions regarding secure clusters.</p> |
| </li> |
| <li> |
| <p>Rolling upgrade between Kudu 1.2 and Kudu 1.3 servers is believed to be possible |
| though has not been sufficiently tested. Users are encouraged to shut down all nodes |
| in the cluster, upgrade the software, and then restart the daemons on the new version.</p> |
| </li> |
| </ul> |
| </div> |
| <div class="paragraph"> |
| <p>The authentication features newly introduced in Kudu 1.3 place the following limitations |
| on wire compatibility with older versions:</p> |
| </div> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>If a Kudu 1.3 cluster is configured with authentication or encryption set to "required", |
| older clients will be unable to connect.</p> |
| </li> |
| <li> |
| <p>If a Kudu 1.3 cluster is configured with authentication and encryption set to "optional" |
| or "disabled", older clients will still be able to connect.</p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="rn_1.3.0_incompatible_changes"><a class="link" href="#rn_1.3.0_incompatible_changes">Incompatible Changes in Kudu 1.3.0</a></h2> |
| <div class="sectionbody"> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>Due to storage format changes in Kudu 1.3, downgrade from Kudu 1.3 to earlier versions |
| is not supported. After upgrading to Kudu 1.3, attempting to restart with an earlier |
| version will result in an error.</p> |
| </li> |
| <li> |
| <p>In order to support running MapReduce and Spark jobs on secure clusters, these |
| frameworks now connect to the cluster at job submission time to retrieve authentication |
| credentials which can later be used by the tasks to be spawned. This means that |
| the process submitting jobs to Kudu clusters must have direct access to that cluster.</p> |
| </li> |
| <li> |
| <p>The embedded web servers in Kudu processes now specify the <code>X-Frame-Options: DENY</code> HTTP |
| header which prevents embedding Kudu web pages in HTML <code>iframe</code> elements.</p> |
| </li> |
| </ul> |
| </div> |
| <div class="sect2"> |
| <h3 id="rn_1.3.0_client_compatibility"><a class="link" href="#rn_1.3.0_client_compatibility">Client Library Compatibility</a></h3> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p>The Kudu 1.3 Java client library is API- and ABI-compatible with Kudu 1.2. Applications |
| written against Kudu 1.2 will compile and run against the Kudu 1.3 client library and |
| vice-versa, unless one of the following newly added APIs is used:</p> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p><code>[Async]KuduClient.exportAuthenticationCredentials(…​)</code> (unstable API)</p> |
| </li> |
| <li> |
| <p><code>[Async]KuduClient.importAuthenticationCredentials(…​)</code> (unstable API)</p> |
| </li> |
| <li> |
| <p><code>[Async]KuduClient.getMasterAddressesAsString()</code></p> |
| </li> |
| <li> |
| <p><code>KuduPredicate.newIsNotNullPredicate()</code></p> |
| </li> |
| <li> |
| <p><code>KuduPredicate.newIsNullPredicate()</code></p> |
| </li> |
| </ul> |
| </div> |
| </li> |
| <li> |
| <p>The Kudu 1.3 C++ client is API- and ABI-forward-compatible with Kudu 1.2. |
| Applications written and compiled against the Kudu 1.2 client library will run without |
| modification against the Kudu 1.3 client library. Applications written and compiled |
| against the Kudu 1.3 client library will run without modification against the Kudu 1.2 |
| client library unless they use one of the following new APIs:</p> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p><code>kudu::DisableOpenSSLInitialization()</code></p> |
| </li> |
| <li> |
| <p><code>KuduClientBuilder::import_authentication_credentials(…​)</code></p> |
| </li> |
| <li> |
| <p><code>KuduClient::ExportAuthenticationCredentials(…​)</code></p> |
| </li> |
| <li> |
| <p><code>KuduClient::NewIsNotNullPredicate(…​)</code></p> |
| </li> |
| <li> |
| <p><code>KuduClient::NewIsNullPredicate(…​)</code></p> |
| </li> |
| </ul> |
| </div> |
| </li> |
| <li> |
| <p>The Kudu 1.3 Python client is API-compatible with Kudu 1.2. Applications |
| written against Kudu 1.2 will continue to run against the Kudu 1.3 client |
| and vice-versa.</p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="rn_1.3.0_known_issues"><a class="link" href="#rn_1.3.0_known_issues">Known Issues and Limitations</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>Please refer to the <a href="known_issues.html">Known Issues and Limitations</a> section of the |
| documentation.</p> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="resources_and_next_steps"><a class="link" href="#resources_and_next_steps">Resources</a></h2> |
| <div class="sectionbody"> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p><a href="http://kudu.apache.org">Kudu Website</a></p> |
| </li> |
| <li> |
| <p><a href="http://github.com/apache/kudu">Kudu GitHub Repository</a></p> |
| </li> |
| <li> |
| <p><a href="index.html">Kudu Documentation</a></p> |
| </li> |
| <li> |
| <p><a href="prior_release_notes.html">Release notes for older releases</a></p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="_installation_options"><a class="link" href="#_installation_options">Installation Options</a></h2> |
| <div class="sectionbody"> |
| <div class="paragraph"> |
| <p>For full installation details, see <a href="installation.html">Kudu Installation</a>.</p> |
| </div> |
| </div> |
| </div> |
| <div class="sect1"> |
| <h2 id="_next_steps"><a class="link" href="#_next_steps">Next Steps</a></h2> |
| <div class="sectionbody"> |
| <div class="ulist"> |
| <ul> |
| <li> |
| <p><a href="quickstart.html">Kudu Quickstart</a></p> |
| </li> |
| <li> |
| <p><a href="installation.html">Installing Kudu</a></p> |
| </li> |
| <li> |
| <p><a href="configuration.html">Configuring Kudu</a></p> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| </div> |
| <div class="col-md-3"> |
| |
| <div id="toc" data-spy="affix" data-offset-top="70"> |
| <ul> |
| |
| <li> |
| |
| <a href="index.html">Introducing Kudu</a> |
| </li> |
| <li> |
| <span class="active-toc">Kudu Release Notes</span> |
| <ul class="sectlevel1"> |
| <li><a href="#rn_1.3.0_new_features">New features</a></li> |
| <li><a href="#_optimizations_and_improvements">Optimizations and improvements</a></li> |
| <li><a href="#rn_1.3.0_fixed_issues">Fixed Issues</a></li> |
| <li><a href="#rn_1.3.0_wire_compatibility">Wire Protocol compatibility</a></li> |
| <li><a href="#rn_1.3.0_incompatible_changes">Incompatible Changes in Kudu 1.3.0</a> |
| <ul class="sectlevel2"> |
| <li><a href="#rn_1.3.0_client_compatibility">Client Library Compatibility</a></li> |
| </ul> |
| </li> |
| <li><a href="#rn_1.3.0_known_issues">Known Issues and Limitations</a></li> |
| <li><a href="#resources_and_next_steps">Resources</a></li> |
| <li><a href="#_installation_options">Installation Options</a></li> |
| <li><a href="#_next_steps">Next Steps</a></li> |
| </ul> |
| </li> |
| <li> |
| |
| <a href="quickstart.html">Getting Started with Kudu</a> |
| </li> |
| <li> |
| |
| <a href="installation.html">Installation Guide</a> |
| </li> |
| <li> |
| |
| <a href="configuration.html">Configuring Kudu</a> |
| </li> |
| <li> |
| |
| <a href="kudu_impala_integration.html">Using Impala with Kudu</a> |
| </li> |
| <li> |
| |
| <a href="administration.html">Administering Kudu</a> |
| </li> |
| <li> |
| |
| <a href="troubleshooting.html">Troubleshooting Kudu</a> |
| </li> |
| <li> |
| |
| <a href="developing.html">Developing Applications with Kudu</a> |
| </li> |
| <li> |
| |
| <a href="schema_design.html">Kudu Schema Design</a> |
| </li> |
| <li> |
| |
| <a href="transaction_semantics.html">Kudu Transaction Semantics</a> |
| </li> |
| <li> |
| |
| <a href="contributing.html">Contributing to Kudu</a> |
| </li> |
| <li> |
| |
| <a href="style_guide.html">Kudu Documentation Style Guide</a> |
| </li> |
| <li> |
| |
| <a href="configuration_reference.html">Kudu Configuration Reference</a> |
| </li> |
| <li> |
| |
| <a href="known_issues.html">Known Issues and Limitations</a> |
| </li> |
| <li> |
| |
| <a href="export_control.html">Export Control Notice</a> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| </div> |
