blob: dfe9b66970ef24f8ab9c0e27e52f2f02aa7c13a7 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<meta name="description" content="A new open source Apache Hadoop ecosystem project, Apache Kudu completes Hadoop's storage layer to enable fast analytics on fast data" />
<meta name="author" content="Cloudera" />
<title>Apache Kudu - Configuring Apache Kudu</title>
<!-- Bootstrap core CSS -->
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css"
integrity="sha384-1q8mTJOASx8j1Au+a5WDVnPi2lkFfwwEAa8hDDdjZlpLegxhjVME1fgjWPGmkzs7"
crossorigin="anonymous">
<!-- Custom styles for this template -->
<link href="/css/kudu.css" rel="stylesheet"/>
<link href="/css/asciidoc.css" rel="stylesheet"/>
<link rel="shortcut icon" href="/img/logo-favicon.ico" />
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.1/css/font-awesome.min.css" />
<!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries -->
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
<![endif]-->
</head>
<body>
<div class="kudu-site container-fluid">
<!-- Static navbar -->
<nav class="navbar navbar-default">
<div class="container-fluid">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="logo" href="/"><img
src="//d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_80px.png"
srcset="//d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_80px.png 1x, //d3dr9sfxru4sde.cloudfront.net/i/k/apachekudu_logo_0716_160px.png 2x"
alt="Apache Kudu"/></a>
</div>
<div id="navbar" class="collapse navbar-collapse">
<ul class="nav navbar-nav navbar-right">
<li >
<a href="/">Home</a>
</li>
<li >
<a href="/overview.html">Overview</a>
</li>
<li class="active">
<a href="/docs/">Documentation</a>
</li>
<li >
<a href="/releases/">Releases</a>
</li>
<li >
<a href="/blog/">Blog</a>
</li>
<!-- NOTE: this dropdown menu does not appear on Mobile, so don't add anything here
that doesn't also appear elsewhere on the site. -->
<li class="dropdown">
<a href="/community.html" role="button" aria-haspopup="true" aria-expanded="false">Community <span class="caret"></span></a>
<ul class="dropdown-menu">
<li class="dropdown-header">GET IN TOUCH</li>
<li><a class="icon email" href="/community.html">Mailing Lists</a></li>
<li><a class="icon slack" href="https://getkudu-slack.herokuapp.com/">Slack Channel</a></li>
<li role="separator" class="divider"></li>
<li><a href="/community.html#meetups-user-groups-and-conference-presentations">Events and Meetups</a></li>
<li><a href="/committers.html">Project Committers</a></li>
<li><a href="/ecosystem.html">Ecosystem</a></li>
<!--<li><a href="/roadmap.html">Roadmap</a></li>-->
<li><a href="/community.html#contributions">How to Contribute</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">DEVELOPER RESOURCES</li>
<li><a class="icon github" href="https://github.com/apache/incubator-kudu">GitHub</a></li>
<li><a class="icon gerrit" href="http://gerrit.cloudera.org:8080/#/q/status:open+project:kudu">Gerrit Code Review</a></li>
<li><a class="icon jira" href="https://issues.apache.org/jira/browse/KUDU">JIRA Issue Tracker</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">SOCIAL MEDIA</li>
<li><a class="icon twitter" href="https://twitter.com/ApacheKudu">Twitter</a></li>
<li><a href="https://www.reddit.com/r/kudu/">Reddit</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">APACHE SOFTWARE FOUNDATION</li>
<li><a href="https://www.apache.org/security/" target="_blank">Security</a></li>
<li><a href="https://www.apache.org/foundation/sponsorship.html" target="_blank">Sponsorship</a></li>
<li><a href="https://www.apache.org/foundation/thanks.html" target="_blank">Thanks</a></li>
<li><a href="https://www.apache.org/licenses/" target="_blank">License</a></li>
</ul>
</li>
<li >
<a href="/faq.html">FAQ</a>
</li>
</ul><!-- /.nav -->
</div><!-- /#navbar -->
</div><!-- /.container-fluid -->
</nav>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div class="container">
<div class="row">
<div class="col-md-9">
<h1>Configuring Apache Kudu</h1>
<div id="preamble">
<div class="sectionbody">
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
Kudu is easier to configure with <a href="http://www.cloudera.com/content/www/en-us/products/cloudera-manager.html">Cloudera Manager</a>
than in a standalone installation. See Cloudera&#8217;s
<a href="http://www.cloudera.com/documentation/kudu/latest/topics/kudu_installation.html">Kudu documentation</a>
for more details about using Kudu with Cloudera Manager.
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_configure_kudu"><a class="link" href="#_configure_kudu">Configure Kudu</a></h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_configuration_basics"><a class="link" href="#_configuration_basics">Configuration Basics</a></h3>
<div class="paragraph">
<p>To configure the behavior of each Kudu process, you can pass command-line flags when
you start it, or read those options from configuration files by passing them using
one or more <code>--flagfile=&lt;file&gt;</code> options. You can even include the
<code>--flagfile</code> option within your configuration file to include other files. Learn more about gflags
by reading <a href="https://gflags.github.io/gflags/">its documentation</a>.</p>
</div>
<div class="paragraph">
<p>You can place options for masters and tablet servers into the same configuration
file, and each will ignore options that do not apply.</p>
</div>
<div class="paragraph">
<p>Flags can be prefixed with either one or two <code>-</code> characters. This
documentation standardizes on two: <code>--example_flag</code>.</p>
</div>
</div>
<div class="sect2">
<h3 id="_discovering_configuration_options"><a class="link" href="#_discovering_configuration_options">Discovering Configuration Options</a></h3>
<div class="paragraph">
<p>Only the most common configuration options are documented here. For a more exhaustive
list of configuration options, see the <a href="configuration_reference.html">Configuration Reference</a>.</p>
</div>
<div class="paragraph">
<p>To see all configuration flags for a given executable, run it with the <code>--help</code> option.
Take care when configuring undocumented flags, as not every possible
configuration has been tested, and undocumented options are not guaranteed to be
maintained in future releases.</p>
</div>
</div>
<div class="sect2">
<h3 id="_configuring_the_kudu_master"><a class="link" href="#_configuring_the_kudu_master">Configuring the Kudu Master</a></h3>
<div class="paragraph">
<p>To see all available configuration options for the <code>kudu-master</code> executable, run it
with the <code>--help</code> option:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>$ kudu-master --help</pre>
</div>
</div>
<table class="tableblock frame-all grid-all spread">
<caption class="title">Table 1. Supported Configuration Flags for Kudu Masters</caption>
<colgroup>
<col style="width: 25%;">
<col style="width: 25%;">
<col style="width: 25%;">
<col style="width: 25%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Flag</th>
<th class="tableblock halign-left valign-top">Valid Options</th>
<th class="tableblock halign-left valign-top">Default</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>--master_addresses</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>localhost</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma-separated list of all the RPC
addresses for Master consensus-configuration. If not specified, assumes a standalone Master.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>--fs_data_dirs</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma-separated list of
directories where the Master will place its data blocks.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>--fs_wal_dir</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directory where the Master will
place its write-ahead logs. May be the same as <em>one of</em> the directories listed in
<code>--fs_data_dirs</code>, but not a sub-directory of a data directory.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>--log_dir</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>/tmp</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directory to store Master log files.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>For the full list of flags for masters, see the
<a href="configuration_reference.html#master_configuration_reference">Kudu Master Configuration Reference</a>.</p>
</div>
</div>
<div class="sect2">
<h3 id="_configuring_tablet_servers"><a class="link" href="#_configuring_tablet_servers">Configuring Tablet Servers</a></h3>
<div class="paragraph">
<p>To see all available configuration options for the <code>kudu-tserver</code> executable,
run it with the <code>--help</code> option:</p>
</div>
<div class="listingblock">
<div class="content">
<pre>$ kudu-tserver --help</pre>
</div>
</div>
<table class="tableblock frame-all grid-all spread">
<caption class="title">Table 2. Supported Configuration Flags for Kudu Tablet Servers</caption>
<colgroup>
<col style="width: 25%;">
<col style="width: 25%;">
<col style="width: 25%;">
<col style="width: 25%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Flag</th>
<th class="tableblock halign-left valign-top">Valid Options</th>
<th class="tableblock halign-left valign-top">Default</th>
<th class="tableblock halign-left valign-top">Description</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">--fs_data_dirs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma-separated list
of directories where the Tablet Server will place its data blocks.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">--fs_wal_dir</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directory where the Tablet Server will
place its write-ahead logs. May be the same as <em>one of</em> the directories listed in
<code>--fs_data_dirs</code>, but not a sub-directory of a data directory.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">--log_dir</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">/tmp</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directory to store Tablet Server log files</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">--tserver_master_addrs</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">string</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>127.0.0.1:7051</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma separated
addresses of the masters which the tablet server should connect to. The masters
do not read this flag.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">--block_cache_capacity_mb</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">integer</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">512</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Maximum amount of memory allocated to the Kudu Tablet Server&#8217;s block cache.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">--memory_limit_hard_bytes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">integer</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">4294967296</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Maximum amount of memory a Tablet Server can consume before it starts rejecting all incoming writes.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>For the full list of flags for tablet servers, see the
<a href="configuration_reference.html#tablet_server_configuration_reference">Kudu Tablet Server Configuration Reference</a>.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_next_steps"><a class="link" href="#_next_steps">Next Steps</a></h2>
<div class="sectionbody">
<div class="ulist">
<ul>
<li>
<p><a href="quickstart.html">Get Started With Kudu</a></p>
</li>
<li>
<p><a href="developing.html">Developing Applications With Kudu</a></p>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="col-md-3">
<div id="toc" data-spy="affix" data-offset-top="70">
<ul>
<li>
<a href="index.html">Introducing Kudu</a>
</li>
<li>
<a href="release_notes.html">Kudu Release Notes</a>
</li>
<li>
<a href="quickstart.html">Getting Started with Kudu</a>
</li>
<li>
<a href="installation.html">Installation Guide</a>
</li>
<li>
<span class="active-toc">Configuring Kudu</span>
<ul class="sectlevel1">
<li><a href="#_configure_kudu">Configure Kudu</a>
<ul class="sectlevel2">
<li><a href="#_configuration_basics">Configuration Basics</a></li>
<li><a href="#_discovering_configuration_options">Discovering Configuration Options</a></li>
<li><a href="#_configuring_the_kudu_master">Configuring the Kudu Master</a></li>
<li><a href="#_configuring_tablet_servers">Configuring Tablet Servers</a></li>
</ul>
</li>
<li><a href="#_next_steps">Next Steps</a></li>
</ul>
</li>
<li>
<a href="kudu_impala_integration.html">Using Impala with Kudu</a>
</li>
<li>
<a href="administration.html">Administering Kudu</a>
</li>
<li>
<a href="troubleshooting.html">Troubleshooting Kudu</a>
</li>
<li>
<a href="developing.html">Developing Applications with Kudu</a>
</li>
<li>
<a href="schema_design.html">Kudu Schema Design</a>
</li>
<li>
<a href="security.html">Kudu Security</a>
</li>
<li>
<a href="transaction_semantics.html">Kudu Transaction Semantics</a>
</li>
<li>
<a href="background_tasks.html">Background Maintenance Tasks</a>
</li>
<li>
<a href="configuration_reference.html">Kudu Configuration Reference</a>
</li>
<li>
<a href="command_line_tools_reference.html">Kudu Command Line Tools Reference</a>
</li>
<li>
<a href="known_issues.html">Known Issues and Limitations</a>
</li>
<li>
<a href="contributing.html">Contributing to Kudu</a>
</li>
<li>
<a href="export_control.html">Export Control Notice</a>
</li>
</ul>
</div>
</div>
</div>
</div>
<footer class="footer">
<div class="row">
<div class="col-md-9">
<p class="small">
Copyright &copy; 2020 The Apache Software Foundation. Last updated 2017-05-31 14:14:57 PDT
</p>
<p class="small">
Apache Kudu, Kudu, Apache, the Apache feather logo, and the Apache Kudu
project logo are either registered trademarks or trademarks of The
Apache Software Foundation in the United States and other countries.
</p>
</div>
<div class="col-md-3">
<a class="pull-right" href="https://www.apache.org/events/current-event.html">
<img src="https://www.apache.org/events/current-event-234x60.png"/>
</a>
</div>
</div>
</footer>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script>
// Try to detect touch-screen devices. Note: Many laptops have touch screens.
$(document).ready(function() {
if ("ontouchstart" in document.documentElement) {
$(document.documentElement).addClass("touch");
} else {
$(document.documentElement).addClass("no-touch");
}
});
</script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/js/bootstrap.min.js"
integrity="sha384-0mSbJDEHialfmuBBQP6A4Qrprq5OVfW37PRR3j5ELqxss1yVqOtnepnHVP9aJ7xS"
crossorigin="anonymous"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-68448017-1', 'auto');
ga('send', 'pageview');
</script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/anchor-js/3.1.0/anchor.js"></script>
<script>
anchors.options = {
placement: 'right',
visible: 'touch',
};
anchors.add();
</script>
</body>
</html>