blob: f62bf993ef80281531a6b3b7752fa6f047ac3b63 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<meta name="description" content="A new open source Apache Hadoop ecosystem project, Apache Kudu completes Hadoop's storage layer to enable fast analytics on fast data" />
<meta name="author" content="Cloudera" />
<title>Apache Kudu - Known Issues and Limitations</title>
<!-- Bootstrap core CSS -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css"
integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7"
crossorigin="anonymous">
<!-- Custom styles for this template -->
<link href="/css/kudu.css" rel="stylesheet"/>
<link href="/css/asciidoc.css" rel="stylesheet"/>
<link rel="shortcut icon" href="/img/logo-favicon.ico" />
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.1/css/font-awesome.min.css" />
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>
<div class="kudu-site container-fluid">
<!-- Static navbar -->
<nav class="navbar navbar-default">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="logo" href="/"><img
src="//d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_80px.png"
srcset="//d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_80px.png 1x, //d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_160px.png 2x"
alt="Apache Kudu"/></a>
</div>
<div id="navbar" class="collapse navbar-collapse">
<ul class="nav navbar-nav navbar-right">
<li >
<a href="/">Home</a>
</li>
<li >
<a href="/overview.html">Overview</a>
</li>
<li class="active">
<a href="/docs/">Documentation</a>
</li>
<li >
<a href="/releases/">Releases</a>
</li>
<li >
<a href="/blog/">Blog</a>
</li>
<!-- NOTE: this dropdown menu does not appear on Mobile, so don't add anything here
that doesn't also appear elsewhere on the site. -->
<li class="dropdown">
<a href="/community.html" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
<ul class="dropdown-menu">
<li class="dropdown-header">GET IN TOUCH</li>
<li><a class="icon email" href="/community.html">Mailing Lists</a></li>
<li><a class="icon slack" href="https://getkudu-slack.herokuapp.com/">Slack Channel</a></li>
<li role="separator" class="divider"></li>
<li><a href="/community.html#meetups-user-groups-and-conference-presentations">Events and Meetups</a></li>
<li><a href="/committers.html">Project Committers</a></li>
<li><a href="/ecosystem.html">Ecosystem</a></li>
<!--<li><a href="/roadmap.html">Roadmap</a></li>-->
<li><a href="/community.html#contributions">How to Contribute</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">DEVELOPER RESOURCES</li>
<li><a class="icon github" href="https://github.com/apache/incubator-kudu">GitHub</a></li>
<li><a class="icon gerrit" href="http://gerrit.cloudera.org:8080/#/q/status:open+project:kudu">Gerrit Code Review</a></li>
<li><a class="icon jira" href="https://issues.apache.org/jira/browse/KUDU">JIRA Issue Tracker</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">SOCIAL MEDIA</li>
<li><a class="icon twitter" href="https://twitter.com/ApacheKudu">Twitter</a></li>
<li><a href="https://www.reddit.com/r/kudu/">Reddit</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">APACHE SOFTWARE FOUNDATION</li>
<li><a href="https://www.apache.org/security/" target="_blank">Security</a></li>
<li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
<li><a href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
<li><a href="https://www.apache.org/licenses/" target="_blank">License</a></li>
</ul>
</li>
<li >
<a href="/faq.html">FAQ</a>
</li>
</ul><!-- /.nav -->
</div><!-- /#navbar -->
</div><!-- /.container-fluid -->
</nav>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div class="container">
<div class="row">
<div class="col-md-9">
<h1>Known Issues and Limitations</h1>
<div class="sect1">
<h2 id="_schema"><a class="link" href="#_schema">Schema</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_primary_keys"><a class="link" href="#_primary_keys">Primary keys</a></h3>
<div class="ulist">
<ul>
<li>
<p>The primary key may not be changed after the table is created.
You must drop and recreate a table to select a new primary key.</p>
</li>
<li>
<p>The columns which make up the primary key must be listed first in the schema.</p>
</li>
<li>
<p>The primary key of a row may not be modified using the <code>UPDATE</code> functionality.
To modify a row&#8217;s primary key, the row must be deleted and re-inserted with
the modified key. Such a modification is non-atomic.</p>
</li>
<li>
<p>Columns with <code>DOUBLE</code>, <code>FLOAT</code>, or <code>BOOL</code> types are not allowed as part of a
primary key definition. Additionally, all columns that are part of a primary
key definition must be <code>NOT NULL</code>.</p>
</li>
<li>
<p>Auto-generated primary keys are not supported.</p>
</li>
<li>
<p>Cells making up a composite primary key are limited to a total of 16KB after the internal
composite-key encoding done by Kudu.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_columns"><a class="link" href="#_columns">Columns</a></h3>
<div class="ulist">
<ul>
<li>
<p>CHAR, VARCHAR, DATE, and complex types like ARRAY are not supported.</p>
</li>
<li>
<p>Type and nullability of existing columns cannot be changed by altering the table.</p>
</li>
<li>
<p>The precision and scale of <code>DECIMAL</code> columns cannot be changed by altering the table.</p>
</li>
<li>
<p>Tables can have a maximum of 300 columns.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_tables"><a class="link" href="#_tables">Tables</a></h3>
<div class="ulist">
<ul>
<li>
<p>Tables must have an odd number of replicas, with a maximum of 7.</p>
</li>
<li>
<p>Replication factor (set at table creation time) cannot be changed.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_cells_individual_values"><a class="link" href="#_cells_individual_values">Cells (individual values)</a></h3>
<div class="ulist">
<ul>
<li>
<p>Cells cannot be larger than 64KB before encoding or compression.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_other_usage_limitations"><a class="link" href="#_other_usage_limitations">Other usage limitations</a></h3>
<div class="ulist">
<ul>
<li>
<p>Kudu is primarily designed for analytic use cases. You are likely to encounter issues if
a single row contains multiple kilobytes of data.</p>
</li>
<li>
<p>Secondary indexes are not supported.</p>
</li>
<li>
<p>Multi-row transactions are not supported.</p>
</li>
<li>
<p>Relational features, like foreign keys, are not supported.</p>
</li>
<li>
<p>Identifiers such as column and table names are restricted to be valid UTF-8 strings.
Additionally, a maximum length of 256 characters is enforced.</p>
</li>
<li>
<p>Dropping a column does not immediately reclaim space. Compaction must run first.</p>
</li>
<li>
<p>There is no way to run compaction manually, but dropping the table will reclaim the
space immediately.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_partitioning_limitations"><a class="link" href="#_partitioning_limitations">Partitioning Limitations</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Tables must be manually pre-split into tablets using simple or compound primary
keys. Automatic splitting is not yet possible. Range partitions may be added
or dropped after a table has been created. See
<a href="schema_design.html">Schema Design</a> for more information.</p>
</li>
<li>
<p>Data in existing tables cannot currently be automatically repartitioned. As a workaround,
create a new table with the new partitioning and insert the contents of the old
table.</p>
</li>
<li>
<p>Tablets that lose a majority of replicas (such as 1 left out of 3) require manual
intervention to be repaired.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_cluster_management"><a class="link" href="#_cluster_management">Cluster management</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Rack awareness is not supported.</p>
</li>
<li>
<p>Multi-datacenter is not supported.</p>
</li>
<li>
<p>Rolling restart is not supported.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_server_management"><a class="link" href="#_server_management">Server management</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Production deployments should configure a least 4GB of memory for tablet servers,
and ideally more than 16GB when approaching the data and tablet <a href="#_scale">Scale</a> limits.</p>
</li>
<li>
<p>Write ahead logs (WAL) can only be stored on one disk.</p>
</li>
<li>
<p>Tablet servers cannot be gracefully decommissioned.</p>
</li>
<li>
<p>Tablet servers can’t change address/port.</p>
</li>
<li>
<p>Kudu has a hard requirement on having up-to-date NTP. Kudu masters and tablet servers
will crash when out of sync.</p>
</li>
<li>
<p>Kudu releases are only tested with NTP. Other time synchronization providers like Chrony
may or may not work.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_scale"><a class="link" href="#_scale">Scale</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Recommended maximum number of tablet servers is 100.</p>
</li>
<li>
<p>Recommended maximum number of masters is 3.</p>
</li>
<li>
<p>Recommended maximum amount of stored data, post-replication and post-compression,
per tablet server is 8TB.</p>
</li>
<li>
<p>The maximum number of tablets per tablet server is 2000, post-replication,
but we recommend 1000 tablets or fewer per tablet server.</p>
</li>
<li>
<p>Maximum number of tablets per table for each tablet server is 60, post-replication,
at table-creation time.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_replication_and_backup_limitations"><a class="link" href="#_replication_and_backup_limitations">Replication and Backup Limitations</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Kudu does not currently include any built-in features for backup and restore.
Users are encouraged to use tools such as Spark or Impala to export or import
tables as necessary.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_security_limitations"><a class="link" href="#_security_limitations">Security Limitations</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p>Authorization is only available at a system-wide, coarse-grained level. Table-level,
column-level, and row-level authorization features are not available.</p>
</li>
<li>
<p>Data encryption at rest is not directly built into Kudu. Encryption of
Kudu data at rest can be achieved through the use of local block device
encryption software such as <code>dmcrypt</code>.</p>
</li>
<li>
<p>Kudu server Kerberos principals must follow the pattern <code>kudu/&lt;HOST&gt;@DEFAULT.REALM</code>.
Configuring an alternate Kerberos principal is not supported.</p>
</li>
<li>
<p>Kudu&#8217;s integration with Apache Flume does not support writing to Kudu clusters that
require Kerberos authentication.</p>
</li>
<li>
<p>Server certificates generated by Kudu IPKI are incompatible with
<a href="https://www.bouncycastle.org/">bouncycastle</a> version 1.52 and earlier. See
<a href="https://issues.apache.org/jira/browse/KUDU-2145">KUDU-2145</a> for details.</p>
</li>
</ul>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_other_known_issues"><a class="link" href="#_other_known_issues">Other Known Issues</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>The following are known bugs and issues with the current release of Kudu. They will
be addressed in later releases. Note that this list is not exhaustive, and is meant
to communicate only the most important known issues.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>If the Kudu master is configured with the <code>-log_force_fsync_all</code> option, tablet servers
and clients will experience frequent timeouts, and the cluster may become unusable.</p>
</li>
<li>
<p>If a tablet server has a very large number of tablets, it may take several minutes
to start up. It is recommended to limit the number of tablets per server to 1000
or fewer. The maximum allowed number of tablets per server is 2000.
Consider this limitation when pre-splitting your tables. If you notice slow start-up times,
you can monitor the number of tablets per server in the web UI.</p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="col-md-3">
<div id="toc" data-spy="affix" data-offset-top="70">
<ul>
<li>
<a href="index.html">Introducing Kudu</a>
</li>
<li>
<a href="release_notes.html">Kudu Release Notes</a>
</li>
<li>
<a href="quickstart.html">Getting Started with Kudu</a>
</li>
<li>
<a href="installation.html">Installation Guide</a>
</li>
<li>
<a href="configuration.html">Configuring Kudu</a>
</li>
<li>
<a href="kudu_impala_integration.html">Using Impala with Kudu</a>
</li>
<li>
<a href="administration.html">Administering Kudu</a>
</li>
<li>
<a href="troubleshooting.html">Troubleshooting Kudu</a>
</li>
<li>
<a href="developing.html">Developing Applications with Kudu</a>
</li>
<li>
<a href="schema_design.html">Kudu Schema Design</a>
</li>
<li>
<a href="scaling_guide.html">Kudu Scaling Guide</a>
</li>
<li>
<a href="security.html">Kudu Security</a>
</li>
<li>
<a href="transaction_semantics.html">Kudu Transaction Semantics</a>
</li>
<li>
<a href="background_tasks.html">Background Maintenance Tasks</a>
</li>
<li>
<a href="configuration_reference.html">Kudu Configuration Reference</a>
</li>
<li>
<a href="command_line_tools_reference.html">Kudu Command Line Tools Reference</a>
</li>
<li>
<span class="active-toc">Known Issues and Limitations</span>
<ul class="sectlevel1">
<li><a href="#_schema">Schema</a>
<ul class="sectlevel2">
<li><a href="#_primary_keys">Primary keys</a></li>
<li><a href="#_columns">Columns</a></li>
<li><a href="#_tables">Tables</a></li>
<li><a href="#_cells_individual_values">Cells (individual values)</a></li>
<li><a href="#_other_usage_limitations">Other usage limitations</a></li>
</ul>
</li>
<li><a href="#_partitioning_limitations">Partitioning Limitations</a></li>
<li><a href="#_cluster_management">Cluster management</a></li>
<li><a href="#_server_management">Server management</a></li>
<li><a href="#_scale">Scale</a></li>
<li><a href="#_replication_and_backup_limitations">Replication and Backup Limitations</a></li>
<li><a href="#_security_limitations">Security Limitations</a></li>
<li><a href="#_other_known_issues">Other Known Issues</a></li>
</ul>
</li>
<li>
<a href="contributing.html">Contributing to Kudu</a>
</li>
<li>
<a href="export_control.html">Export Control Notice</a>
</li>
</ul>
</div>
</div>
</div>
</div>
<footer class="footer">
<div class="row">
<div class="col-md-9">
<p class="small">
Copyright &copy; 2020 The Apache Software Foundation. Last updated 2018-10-12 18:37:00 CEST
</p>
<p class="small">
Apache Kudu, Kudu, Apache, the Apache feather logo, and the Apache Kudu
project logo are either registered trademarks or trademarks of The
Apache Software Foundation in the United States and other countries.
</p>
</div>
<div class="col-md-3">
<a class="pull-right" href="https://www.apache.org/events/current-event.html">
<img src="https://www.apache.org/events/current-event-234x60.png"/>
</a>
</div>
</div>
</footer>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script>
// Try to detect touch-screen devices. Note: Many laptops have touch screens.
$(document).ready(function() {
if ("ontouchstart" in document.documentElement) {
$(document.documentElement).addClass("touch");
} else {
$(document.documentElement).addClass("no-touch");
}
});
</script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"
integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS"
crossorigin="anonymous"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-68448017-1', 'auto');
ga('send', 'pageview');
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/3.1.0/anchor.js"></script>
<script>
anchors.options = {
placement: 'right',
visible: 'touch',
};
anchors.add();
</script>
</body>
</html>