| // Licensed to the Apache Software Foundation (ASF) under one |
| // or more contributor license agreements. See the NOTICE file |
| // distributed with this work for additional information |
| // regarding copyright ownership. The ASF licenses this file |
| // to you under the Apache License, Version 2.0 (the |
| // "License"); you may not use this file except in compliance |
| // with the License. You may obtain a copy of the License at |
| // |
| // http://www.apache.org/licenses/LICENSE-2.0 |
| // |
| // Unless required by applicable law or agreed to in writing, |
| // software distributed under the License is distributed on an |
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| // KIND, either express or implied. See the License for the |
| // specific language governing permissions and limitations |
| // under the License. |
| |
| [[configuration]] |
| = Configuring Apache Kudu |
| |
| :author: Kudu Team |
| :imagesdir: ./images |
| :icons: font |
| :toc: left |
| :toclevels: 3 |
| :doctype: book |
| :backend: html5 |
| :sectlinks: |
| :experimental: |
| |
| include::top.adoc[tags=version] |
| |
| == Configure Kudu |
| |
| === Configuration Basics |
| To configure the behavior of each Kudu process, you can pass command-line flags when |
| you start it, or read those options from configuration files by passing them using |
| one or more `--flagfile=<file>` options. You can even include the |
| `--flagfile` option within your configuration file to include other files. Learn more about gflags |
| by reading link:https://gflags.github.io/gflags/[its documentation]. |
| |
| You can place options for masters and tablet servers into the same configuration |
| file, and each will ignore options that do not apply. |
| |
| Flags can be prefixed with either one or two `-` characters. This |
| documentation standardizes on two: `--example_flag`. |
| |
| === Discovering Configuration Options |
| Only the most common configuration options are documented here. For a more exhaustive |
| list of configuration options, see the link:configuration_reference.html[Configuration Reference]. |
| |
| To see all configuration flags for a given executable, run it with the `--help` option. |
| Take care when configuring undocumented flags, as not every possible |
| configuration has been tested, and undocumented options are not guaranteed to be |
| maintained in future releases. |
| |
| [[directory_configuration]] |
| === Directory Configurations |
| Every Kudu node requires the specification of directory flags. The |
| `--fs_wal_dir` configuration indicates where Kudu will place its write-ahead |
| logs. The `--fs_metadata_dir` configuration indicates where Kudu will place |
| metadata for each tablet. It is recommended, although not necessary, that these |
| directories be placed on a high-performance drives with high bandwidth and low |
| latency, e.g. solid-state drives. If `--fs_metadata_dir` is not specified, |
| metadata will be placed in the directory specified by `--fs_wal_dir`. Since |
| a Kudu node cannot tolerate the loss of its WAL or metadata directories, it |
| may be wise to mirror the drives containing these directories in order to |
| make recovering from a drive failure easier; however, mirroring may increase |
| the latency of Kudu writes. |
| |
| The `--fs_data_dirs` configuration indicates where Kudu will write its data |
| blocks. This is a comma-separated list of directories; if multiple values are |
| specified, data will be striped across the directories. If not specified, data |
| blocks will be placed in the directory specified by `--fs_wal_dir`. Note that |
| while a single data directory backed by a RAID-0 array will outperform a single |
| data directory backed by a single storage device, it is better to let Kudu |
| manage its own striping over multiple devices rather than delegating the |
| striping to a RAID-0 array. |
| |
| Additionally, `--fs_wal_dir` and `--fs_metadata_dir` may be the same as _one |
| of_ the directories listed in `--fs_data_dirs`, but must not be sub-directories |
| of any of them. |
| |
| WARNING: Each directory specified by a configuration flag on a given machine |
| should be used by at most one Kudu process. If multiple Kudu processes on the |
| same machine are configured to use the same directory, Kudu may refuse to start |
| up. |
| |
| WARNING: Once `--fs_data_dirs` is set, extra tooling is required to change it. |
| For more details, see the link:administration.html#change_dir_config[Kudu |
| Administration docs]. |
| |
| NOTE: The `--fs_wal_dir` and `--fs_metadata_dir` configurations can be changed, |
| provided the contents of the directories are also moved to match the flags. |
| |
| === Configuring the Kudu Master |
| To see all available configuration options for the `kudu-master` executable, run it |
| with the `--help` option: |
| ---- |
| $ kudu-master --help |
| ---- |
| |
| [cols="m,d,m,d"] |
| .Supported Configuration Flags for Kudu Masters |
| |=== |
| | Flag | Valid Options | Default | Description |
| |
| |--master_addresses | string | localhost | Comma-separated list of all the RPC |
| addresses for Master consensus-configuration. If not specified, assumes a standalone Master. |
| |--fs_data_dirs | string | | List of directories where the Master will place its data blocks. |
| |--fs_metadata_dir | string | | The directory where the Master will place its tablet metadata. |
| |--fs_wal_dir | string | | The directory where the Master will place its write-ahead logs. |
| |--log_dir | string | /tmp | The directory to store Master log files. |
| |=== |
| |
| For the full list of flags for masters, see the |
| link:configuration_reference.html#kudu-master_supported[Kudu Master Configuration Reference]. |
| |
| === Configuring Tablet Servers |
| To see all available configuration options for the `kudu-tserver` executable, |
| run it with the `--help` option: |
| ---- |
| $ kudu-tserver --help |
| ---- |
| |
| .Supported Configuration Flags for Kudu Tablet Servers |
| |=== |
| | Flag | Valid Options | Default | Description |
| |
| |--fs_data_dirs | string | | List of directories where the Tablet Server will place its data blocks. |
| |--fs_metadata_dir | string | | The directory where the Tablet Server will place its tablet metadata. |
| |--fs_wal_dir | string | | The directory where the Tablet Server will place its write-ahead logs. |
| |--log_dir | string | /tmp | The directory to store Tablet Server log files |
| |--tserver_master_addrs | string | `127.0.0.1:7051` | Comma separated |
| addresses of the masters which the tablet server should connect to. The masters |
| do not read this flag. |
| |--block_cache_capacity_mb | integer | 512 | Maximum amount of memory allocated to the Kudu Tablet Server's block cache. |
| |--memory_limit_hard_bytes | integer | 4294967296 | Maximum amount of memory a Tablet Server can consume before it starts rejecting all incoming writes. |
| |=== |
| |
| For the full list of flags for tablet servers, see the |
| link:configuration_reference.html#kudu-tserver_supported[Kudu Tablet Server Configuration Reference]. |
| |
| == Configure Kudu Tables |
| Kudu allows certain configurations to be set per table. To configure the behavior of a Kudu table, |
| you can set these configurations at table creation, or alter them via the Kudu API or Kudu command |
| line tool. |
| |
| .Supported Configurable Properties for Kudu Tables |
| |=== |
| | Configuration | Valid Options | Default | Description |
| |
| | kudu.table.history_max_age_sec | integer | | Number of seconds to retain history for tablets in this table. |
| | kudu.table.maintenance_priority | integer | 0 | Priority level of a table for maintenance. |
| |=== |
| |
| == Next Steps |
| - link:quickstart.html[Get Started With Kudu] |
| - link:developing.html[Developing Applications With Kudu] |