blob: d646db379a51033b99e3a6614dc030714e3a9890 [file] [log] [blame]
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge"/><title>Human-readable Byte Configuration Reference · Apache Druid</title><meta name="viewport" content="width=device-width"/><link rel="canonical" href="https://druid.apache.org/docs/0.23.0/configuration/human-readable-byte.html"/><meta name="generator" content="Docusaurus"/><meta name="description" content="&lt;!--"/><meta name="docsearch:language" content="en"/><meta name="docsearch:version" content="0.23.0" /><meta property="og:title" content="Human-readable Byte Configuration Reference · Apache Druid"/><meta property="og:type" content="website"/><meta property="og:url" content="https://druid.apache.org/index.html"/><meta property="og:description" content="&lt;!--"/><meta property="og:image" content="https://druid.apache.org/img/druid_nav.png"/><meta name="twitter:card" content="summary"/><meta name="twitter:image" content="https://druid.apache.org/img/druid_nav.png"/><link rel="shortcut icon" href="/img/favicon.png"/><link rel="stylesheet" href="https://cdn.jsdelivr.net/docsearch.js/1/docsearch.min.css"/><link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.12.0/styles/default.min.css"/><script async="" src="https://www.googletagmanager.com/gtag/js?id=UA-131010415-1"></script><script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments); }
gtag('js', new Date());
gtag('config', 'UA-131010415-1');
</script><link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.7.2/css/all.css"/><link rel="stylesheet" href="/css/code-block-buttons.css"/><script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/2.0.4/clipboard.min.js"></script><script type="text/javascript" src="/js/code-block-buttons.js"></script><script src="/js/scrollSpy.js"></script><link rel="stylesheet" href="/css/main.css"/><script src="/js/codetabs.js"></script></head><body class="sideNavVisible separateOnPageNav"><div class="fixedHeaderContainer"><div class="headerWrapper wrapper"><header><a href="/"><img class="logo" src="/img/druid_nav.png" alt="Apache Druid"/></a><div class="navigationWrapper navigationSlider"><nav class="slidingNav"><ul class="nav-site nav-site-internal"><li class=""><a href="/technology" target="_self">Technology</a></li><li class=""><a href="/use-cases" target="_self">Use Cases</a></li><li class=""><a href="/druid-powered" target="_self">Powered By</a></li><li class=""><a href="/docs/0.23.0/design/index.html" target="_self">Docs</a></li><li class=""><a href="/community/" target="_self">Community</a></li><li class=""><a href="https://www.apache.org" target="_self">Apache</a></li><li class=""><a href="/downloads.html" target="_self">Download</a></li><li class="navSearchWrapper reactNavSearchWrapper"><input type="text" id="search_input_react" placeholder="Search" title="Search"/></li></ul></nav></div></header></div></div><div class="navPusher"><div class="docMainWrapper wrapper"><div class="container mainContainer docsContainer"><div class="wrapper"><div class="post"><header class="postHeader"><a class="edit-page-link button" href="https://github.com/apache/druid/edit/master/docs/configuration/human-readable-byte.md" target="_blank" rel="noreferrer noopener">Edit</a><h1 id="__docusaurus" class="postHeaderTitle">Human-readable Byte Configuration Reference</h1></header><article><div><span><!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
<p>This page documents configuration properties related to bytes.</p>
<p>These properties can be configured through 2 ways:</p>
<ol>
<li>a simple number in bytes</li>
<li>a number with a unit suffix</li>
</ol>
<h2><a class="anchor" aria-hidden="true" id="a-number-in-bytes"></a><a href="#a-number-in-bytes" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>A number in bytes</h2>
<p>Given that cache size is 3G, there's a configuration as below</p>
<pre><code class="hljs css language-properties"><span class="hljs-comment"># 3G bytes = 3_000_000_000 bytes</span>
<span class="hljs-meta">druid.cache.sizeInBytes</span>=<span class="hljs-string">3000000000 </span>
</code></pre>
<h2><a class="anchor" aria-hidden="true" id="a-number-with-a-unit-suffix"></a><a href="#a-number-with-a-unit-suffix" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>A number with a unit suffix</h2>
<p>When you have to put a large number for some configuration as above, it is easy to make a mistake such as extra or missing 0s. Druid supports a better way, a number with a unit suffix.</p>
<p>Given a disk of 1T, the configuration can be</p>
<pre><code class="hljs css language-properties"><span class="hljs-meta">druid.segmentCache.locations</span>=<span class="hljs-string">[{"path":"/segment-cache-00","maxSize":"1t"},{"path":"/segment-cache-01","maxSize":"1200g"}]</span>
</code></pre>
<p>Note: in above example, both <code>1t</code> and <code>1T</code> are acceptable since it's case-insensitive.
Also, only integers are valid as the number part. For example, you can't replace <code>1200g</code> with <code>1.2t</code>.</p>
<h3><a class="anchor" aria-hidden="true" id="supported-units"></a><a href="#supported-units" aria-hidden="true" class="hash-link"><svg class="hash-link-icon" aria-hidden="true" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Supported Units</h3>
<p>In the world of computer, a unit like <code>K</code> is ambiguous. It means 1000 or 1024 in different contexts, for more information please see <a href="https://en.wikipedia.org/wiki/Binary_prefix">Here</a>.</p>
<p>To make it clear, the base of units are defined in Druid as below</p>
<table>
<thead>
<tr><th>Unit</th><th>Description</th><th>Base</th></tr>
</thead>
<tbody>
<tr><td>K</td><td>Kilo Decimal Byte</td><td>1_000</td></tr>
<tr><td>M</td><td>Mega Decimal Byte</td><td>1_000_000</td></tr>
<tr><td>G</td><td>Giga Decimal Byte</td><td>1_000_000_000</td></tr>
<tr><td>T</td><td>Tera Decimal Byte</td><td>1_000_000_000_000</td></tr>
<tr><td>P</td><td>Peta Decimal Byte</td><td>1_000_000_000_000_000</td></tr>
<tr><td>Ki</td><td>Kilo Binary Byte</td><td>1024</td></tr>
<tr><td>Mi</td><td>Mega Binary Byte</td><td>1024 * 1024</td></tr>
<tr><td>Gi</td><td>Giga Binary Byte</td><td>1024 * 1024 * 1024</td></tr>
<tr><td>Ti</td><td>Tera Binary Byte</td><td>1024 * 1024 * 1024 * 1024</td></tr>
<tr><td>Pi</td><td>Peta Binary Byte</td><td>1024 * 1024 * 1024 * 1024 * 1024</td></tr>
<tr><td>KiB</td><td>Kilo Binary Byte</td><td>1024</td></tr>
<tr><td>MiB</td><td>Mega Binary Byte</td><td>1024 * 1024</td></tr>
<tr><td>GiB</td><td>Giga Binary Byte</td><td>1024 * 1024 * 1024</td></tr>
<tr><td>TiB</td><td>Tera Binary Byte</td><td>1024 * 1024 * 1024 * 1024</td></tr>
<tr><td>PiB</td><td>Peta Binary Byte</td><td>1024 * 1024 * 1024 * 1024 * 1024</td></tr>
</tbody>
</table>
<p>Unit is case-insensitive. <code>k</code>, <code>kib</code>, <code>ki</code>, <code>KiB</code>, <code>Ki</code>, <code>kiB</code> are all acceptable.</p>
<p>Here are some examples</p>
<pre><code class="hljs css language-properties"><span class="hljs-comment"># 1G bytes = 1_000_000_000 bytes</span>
<span class="hljs-meta">druid.cache.sizeInBytes</span>=<span class="hljs-string">1g </span>
</code></pre>
<pre><code class="hljs css language-properties"><span class="hljs-comment"># 256MiB bytes = 256 * 1024 * 1024 bytes</span>
<span class="hljs-meta">druid.cache.sizeInBytes</span>=<span class="hljs-string">256MiB </span>
</code></pre>
<pre><code class="hljs css language-properties"><span class="hljs-comment"># 256Mi = 256MiB = 256 * 1024 * 1024 bytes</span>
<span class="hljs-meta">druid.cache.sizeInBytes</span>=<span class="hljs-string">256Mi</span>
</code></pre>
</span></div></article></div><div class="docs-prevnext"></div></div></div><nav class="onPageNav"><ul class="toc-headings"><li><a href="#a-number-in-bytes">A number in bytes</a></li><li><a href="#a-number-with-a-unit-suffix">A number with a unit suffix</a><ul class="toc-headings"><li><a href="#supported-units">Supported Units</a></li></ul></li></ul></nav></div><footer class="nav-footer druid-footer" id="footer"><div class="container"><div class="text-center"><p><a href="/technology">Technology</a> · <a href="/use-cases">Use Cases</a> · <a href="/druid-powered">Powered by Druid</a> · <a href="/docs/0.23.0/latest">Docs</a> · <a href="/community/">Community</a> · <a href="/downloads.html">Download</a> · <a href="/faq">FAQ</a></p></div><div class="text-center"><a title="Join the user group" href="https://groups.google.com/forum/#!forum/druid-user" target="_blank"><span class="fa fa-comments"></span></a> · <a title="Follow Druid" href="https://twitter.com/druidio" target="_blank"><span class="fab fa-twitter"></span></a> · <a title="Download via Apache" href="https://www.apache.org/dyn/closer.cgi?path=/incubator/druid/{{ site.druid_versions[0].versions[0].version }}/apache-druid-{{ site.druid_versions[0].versions[0].version }}-bin.tar.gz" target="_blank"><span class="fas fa-feather"></span></a> · <a title="GitHub" href="https://github.com/apache/druid" target="_blank"><span class="fab fa-github"></span></a></div><div class="text-center license">Copyright © 2019 <a href="https://www.apache.org/" target="_blank">Apache Software Foundation</a>.<br/>Except where otherwise noted, licensed under <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA 4.0</a>.<br/>Apache Druid, Druid, and the Druid logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</div></div></footer></div><script type="text/javascript" src="https://cdn.jsdelivr.net/docsearch.js/1/docsearch.min.js"></script><script>
document.addEventListener('keyup', function(e) {
if (e.target !== document.body) {
return;
}
// keyCode for '/' (slash)
if (e.keyCode === 191) {
const search = document.getElementById('search_input_react');
search && search.focus();
}
});
</script><script>
var search = docsearch({
apiKey: '2de99082a9f38e49dfaa059bbe4c901d',
indexName: 'apache_druid',
inputSelector: '#search_input_react',
algoliaOptions: {"facetFilters":["language:en","version:0.23.0"]}
});
</script></body></html>