blob: 3f33fbce3b4f58a8bd2ec94de6ed33a46777f20b [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<meta name="description" content="A new open source Apache Hadoop ecosystem project, Apache Kudu completes Hadoop's storage layer to enable fast analytics on fast data" />
<meta name="author" content="Cloudera" />
<title>Apache Kudu - Known Issues and Limitations</title>
<!-- Bootstrap core CSS -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css"
integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7"
crossorigin="anonymous">
<!-- Custom styles for this template -->
<link href="/css/kudu.css" rel="stylesheet"/>
<link href="/css/asciidoc.css" rel="stylesheet"/>
<link rel="shortcut icon" href="/img/logo-favicon.ico" />
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.1/css/font-awesome.min.css" />
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>
<div class="kudu-site container-fluid">
<!-- Static navbar -->
<nav class="navbar navbar-default">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="logo" href="/"><img
src="//d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_80px.png"
srcset="//d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_80px.png 1x, //d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_160px.png 2x"
alt="Apache Kudu"/></a>
</div>
<div id="navbar" class="collapse navbar-collapse">
<ul class="nav navbar-nav navbar-right">
<li >
<a href="/">Home</a>
</li>
<li >
<a href="/overview.html">Overview</a>
</li>
<li class="active">
<a href="/docs/">Documentation</a>
</li>
<li >
<a href="/releases/">Releases</a>
</li>
<li >
<a href="/blog/">Blog</a>
</li>
<!-- NOTE: this dropdown menu does not appear on Mobile, so don't add anything here
that doesn't also appear elsewhere on the site. -->
<li class="dropdown">
<a href="/community.html" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
<ul class="dropdown-menu">
<li class="dropdown-header">GET IN TOUCH</li>
<li><a class="icon email" href="/community.html">Mailing Lists</a></li>
<li><a class="icon slack" href="https://getkudu-slack.herokuapp.com/">Slack Channel</a></li>
<li role="separator" class="divider"></li>
<li><a href="/community.html#meetups-user-groups-and-conference-presentations">Events and Meetups</a></li>
<li><a href="/committers.html">Project Committers</a></li>
<li><a href="/ecosystem.html">Ecosystem</a></li>
<!--<li><a href="/roadmap.html">Roadmap</a></li>-->
<li><a href="/community.html#contributions">How to Contribute</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">DEVELOPER RESOURCES</li>
<li><a class="icon github" href="https://github.com/apache/incubator-kudu">GitHub</a></li>
<li><a class="icon gerrit" href="http://gerrit.cloudera.org:8080/#/q/status:open+project:kudu">Gerrit Code Review</a></li>
<li><a class="icon jira" href="https://issues.apache.org/jira/browse/KUDU">JIRA Issue Tracker</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">SOCIAL MEDIA</li>
<li><a class="icon twitter" href="https://twitter.com/ApacheKudu">Twitter</a></li>
<li><a href="https://www.reddit.com/r/kudu/">Reddit</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">APACHE SOFTWARE FOUNDATION</li>
<li><a href="https://www.apache.org/security/" target="_blank">Security</a></li>
<li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
<li><a href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
<li><a href="https://www.apache.org/licenses/" target="_blank">License</a></li>
</ul>
</li>
<li >
<a href="/faq.html">FAQ</a>
</li>
</ul><!-- /.nav -->
</div><!-- /#navbar -->
</div><!-- /.container-fluid -->
</nav>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div class="container">
<div class="row">
<div class="col-md-9">
<h1>Known Issues and Limitations</h1>
<div class="sect1">
<h2 id="_schema"><a class="link" href="#_schema">Schema</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_primary_keys"><a class="link" href="#_primary_keys">Primary keys</a></h3>
<div class="ulist">
<ul>
<li>
<p>The primary key may not be changed after the table is created.
You must drop and recreate a table to select a new primary key.</p>
</li>
<li>
<p>The columns which make up the primary key must be listed first in the schema.</p>
</li>
<li>
<p>The primary key of a row may not be modified using the <code>UPDATE</code> functionality.
To modify a row&#8217;s primary key, the row must be deleted and re-inserted with
the modified key. Such a modification is non-atomic.</p>
</li>
<li>
<p>Columns with <code>DOUBLE</code>, <code>FLOAT</code>, or <code>BOOL</code> types are not allowed as part of a
primary key definition. Additionally, all columns that are part of a primary
key definition must be <code>NOT NULL</code>.</p>
</li>
<li>
<p>Auto-generated primary keys are not supported.</p>
</li>
<li>
<p>Cells making up a composite primary key are limited to a total of 16KB after the internal
composite-key encoding done by Kudu.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_columns"><a class="link" href="#_columns">Columns</a></h3>
<div class="ulist">
<ul>
<li>
<p>DECIMAL, CHAR, VARCHAR, DATE, and complex types like ARRAY are not supported.</p>
</li>
<li>
<p>Type and nullability of existing columns cannot be changed by altering the table.</p>
</li>
<li>
<p>Tables can have a maximum of 300 columns.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_tables"><a class="link" href="#_tables">Tables</a></h3>
<div class="ulist">
<ul>
<li>
<p>Tables must have an odd number of replicas, with a maximum of 7.</p>
</li>
<li>
<p>Replication factor (set at table creation time) cannot be changed.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_cells_individual_values"><a class="link" href="#_cells_individual_values">Cells (individual values)</a></h3>
<div class="ulist">
<ul>
<li>
<p>Cells cannot be larger than 64KB before encoding or compression.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_other_usage_limitations"><a class="link" href="#_other_usage_limitations">Other usage limitations</a></h3>
<div class="ulist">
<ul>
<li>
<p>Kudu is primarily designed for analytic use cases. You are likely to encounter issues if
a single row contains multiple kilobytes of data.</p>
</li>
<li>
<p>Secondary indexes are not supported.</p>
</li>
<li>
<p>Multi-row transactions are not supported.</p>
</li>
<li>
<p>Relational features, like foreign keys, are not supported.</p>
</li>
<li>
<p>Identifiers such as column and table names are restricted to be valid UTF-8 strings.
Additionally, a maximum length of 256 characters is enforced.</p>
</li>
<li>
<p>Dropping a column does not immediately reclaim space. Compaction must run first.</p>
</li>
<li>
<p>There is no way to run compaction manually, but dropping the table will reclaim the
space immediately.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_partitioning_limitations"><a class="link" href="#_partitioning_limitations">Partitioning Limitations</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Tables must be manually pre-split into tablets using simple or compound primary
keys. Automatic splitting is not yet possible. Range partitions may be added
or dropped after a table has been created. See
<a href="schema_design.html">Schema Design</a> for more information.</p>
</li>
<li>
<p>Data in existing tables cannot currently be automatically repartitioned. As a workaround,
create a new table with the new partitioning and insert the contents of the old
table.</p>
</li>
<li>
<p>Tablets that lose a majority of replicas (such as 1 left out of 3) require manual
intervention to be repaired.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_cluster_management"><a class="link" href="#_cluster_management">Cluster management</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Rack awareness is not supported.</p>
</li>
<li>
<p>Multi-datacenter is not supported.</p>
</li>
<li>
<p>Rolling restart is not supported.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_server_management"><a class="link" href="#_server_management">Server management</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Production deployments should configure a least 4GB of memory for tablet servers,
and ideally more than 10GB.</p>
</li>
<li>
<p>Write ahead logs (WAL) can only be stored on one disk.</p>
</li>
<li>
<p>Disk failures are not tolerated and tablets servers will crash as soon as one is detected.</p>
</li>
<li>
<p>Failed disks with unrecoverable data require the formatting of all the Kudu data for
that tablet server before it can be started again.</p>
</li>
<li>
<p>Data directories cannot be added/removed; all must be reformatted to change the set
of directories.</p>
</li>
<li>
<p>Tablet servers cannot be gracefully decommissioned.</p>
</li>
<li>
<p>Tablet servers can’t change address/port.</p>
</li>
<li>
<p>Kudu has a hard requirement on having up-to-date NTP. Kudu masters and tablet servers
will crash when out of sync.</p>
</li>
<li>
<p>Kudu releases are only tested with NTP. Other time synchronization providers like Chrony
may or may not work.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_scale"><a class="link" href="#_scale">Scale</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Recommended maximum number of tablet servers is 100.</p>
</li>
<li>
<p>Recommended maximum number of masters is 3.</p>
</li>
<li>
<p>Recommended maximum amount of stored data, post-replication and post-compression,
per tablet server is 4TB.</p>
</li>
<li>
<p>Recommended maximum number of tablets per tablet server is 1000, post-replication.</p>
</li>
<li>
<p>Maximum number of tablets per table for each tablet server is 60, post-replication,
at table-creation time.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_replication_and_backup_limitations"><a class="link" href="#_replication_and_backup_limitations">Replication and Backup Limitations</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Kudu does not currently include any built-in features for backup and restore.
Users are encouraged to use tools such as Spark or Impala to export or import
tables as necessary.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_security_limitations"><a class="link" href="#_security_limitations">Security Limitations</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Authorization is only available at a system-wide, coarse-grained level. Table-level,
column-level, and row-level authorization features are not available.</p>
</li>
<li>
<p>Data encryption at rest is not built in. Kudu has been reported to run correctly
on systems using local block device encryption (e.g. <code>dmcrypt</code>).</p>
</li>
<li>
<p>Kudu server Kerberos principals must follow the pattern <code>kudu/&lt;HOST&gt;@DEFAULT.REALM</code>.
Configuring an alternate Kerberos principal is not supported.</p>
</li>
<li>
<p>Kudu&#8217;s integration with Apache Flume does not support writing to Kudu clusters that
require Kerberos authentication.</p>
</li>
<li>
<p>Kudu client instances retrieve authentication tokens upon first contact with the
cluster. These tokens expire after one week. Use of a single Kudu client instance
for more than one week is only supported by the C++ client but not by the Java client.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_other_known_issues"><a class="link" href="#_other_known_issues">Other Known Issues</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>The following are known bugs and issues with the current release of Kudu. They will
be addressed in later releases. Note that this list is not exhaustive, and is meant
to communicate only the most important known issues.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>If the Kudu master is configured with the <code>-log_force_fsync_all</code> option, tablet servers
and clients will experience frequent timeouts, and the cluster may become unusable.</p>
</li>
<li>
<p>If a tablet server has a very large number of tablets, it may take several minutes
to start up. It is recommended to limit the number of tablets per server to 100 or fewer.
Consider this limitation when pre-splitting your tables. If you notice slow start-up times,
you can monitor the number of tablets per server in the web UI.</p>
</li>
<li>
<p>Kerberos authentication does not function correctly on hosts which contain
capital letters in their hostname.</p>
</li>
<li>
<p>Kerberos authentication does not function correctly if <code>rdns = false</code> is configured
in <code>krb5.conf</code>.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="col-md-3">
<div id="toc" data-spy="affix" data-offset-top="70">
<ul>
<li>
<a href="index.html">Introducing Kudu</a>
</li>
<li>
<a href="release_notes.html">Kudu Release Notes</a>
</li>
<li>
<a href="quickstart.html">Getting Started with Kudu</a>
</li>
<li>
<a href="installation.html">Installation Guide</a>
</li>
<li>
<a href="configuration.html">Configuring Kudu</a>
</li>
<li>
<a href="kudu_impala_integration.html">Using Impala with Kudu</a>
</li>
<li>
<a href="administration.html">Administering Kudu</a>
</li>
<li>
<a href="troubleshooting.html">Troubleshooting Kudu</a>
</li>
<li>
<a href="developing.html">Developing Applications with Kudu</a>
</li>
<li>
<a href="schema_design.html">Kudu Schema Design</a>
</li>
<li>
<a href="security.html">Kudu Security</a>
</li>
<li>
<a href="transaction_semantics.html">Kudu Transaction Semantics</a>
</li>
<li>
<a href="background_tasks.html">Background Maintenance Tasks</a>
</li>
<li>
<a href="configuration_reference.html">Kudu Configuration Reference</a>
</li>
<li>
<a href="command_line_tools_reference.html">Kudu Command Line Tools Reference</a>
</li>
<li>
<span class="active-toc">Known Issues and Limitations</span>
<ul class="sectlevel1">
<li><a href="#_schema">Schema</a>
<ul class="sectlevel2">
<li><a href="#_primary_keys">Primary keys</a></li>
<li><a href="#_columns">Columns</a></li>
<li><a href="#_tables">Tables</a></li>
<li><a href="#_cells_individual_values">Cells (individual values)</a></li>
<li><a href="#_other_usage_limitations">Other usage limitations</a></li>
</ul>
</li>
<li><a href="#_partitioning_limitations">Partitioning Limitations</a></li>
<li><a href="#_cluster_management">Cluster management</a></li>
<li><a href="#_server_management">Server management</a></li>
<li><a href="#_scale">Scale</a></li>
<li><a href="#_replication_and_backup_limitations">Replication and Backup Limitations</a></li>
<li><a href="#_security_limitations">Security Limitations</a></li>
<li><a href="#_other_known_issues">Other Known Issues</a></li>
</ul>
</li>
<li>
<a href="contributing.html">Contributing to Kudu</a>
</li>
<li>
<a href="export_control.html">Export Control Notice</a>
</li>
</ul>
</div>
</div>
</div>
</div>
<footer class="footer">
<div class="row">
<div class="col-md-9">
<p class="small">
Copyright &copy; 2020 The Apache Software Foundation. Last updated 2017-06-08 18:46:50 PDT
</p>
<p class="small">
Apache Kudu, Kudu, Apache, the Apache feather logo, and the Apache Kudu
project logo are either registered trademarks or trademarks of The
Apache Software Foundation in the United States and other countries.
</p>
</div>
<div class="col-md-3">
<a class="pull-right" href="https://www.apache.org/events/current-event.html">
<img src="https://www.apache.org/events/current-event-234x60.png"/>
</a>
</div>
</div>
</footer>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script>
// Try to detect touch-screen devices. Note: Many laptops have touch screens.
$(document).ready(function() {
if ("ontouchstart" in document.documentElement) {
$(document.documentElement).addClass("touch");
} else {
$(document.documentElement).addClass("no-touch");
}
});
</script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"
integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS"
crossorigin="anonymous"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-68448017-1', 'auto');
ga('send', 'pageview');
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/3.1.0/anchor.js"></script>
<script>
anchors.options = {
placement: 'right',
visible: 'touch',
};
anchors.add();
</script>
</body>
</html>