| --- |
| layout: docpage |
| |
| title: "Documentation" |
| |
| is_homepage: false |
| is_sphinx_doc: true |
| |
| doc-title: "Leveled Compaction Strategy" |
| doc-header-links: ' |
| <link rel="top" title="Apache Cassandra Documentation v4.0-rc1" href="../../index.html"/> |
| ' |
| doc-search-path: "../../search.html" |
| |
| extra-footer: ' |
| <script type="text/javascript"> |
| var DOCUMENTATION_OPTIONS = { |
| URL_ROOT: "", |
| VERSION: "", |
| COLLAPSE_INDEX: false, |
| FILE_SUFFIX: ".html", |
| HAS_SOURCE: false, |
| SOURCELINK_SUFFIX: ".txt" |
| }; |
| </script> |
| ' |
| |
| --- |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <div class="doc-navigation"> |
| <div class="doc-menu" role="navigation"> |
| <div class="navbar-header"> |
| <button type="button" class="pull-left navbar-toggle" data-toggle="collapse" data-target=".sidebar-navbar-collapse"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| </div> |
| <div class="navbar-collapse collapse sidebar-navbar-collapse"> |
| <form id="doc-search-form" class="navbar-form" action="../../search.html" method="get" role="search"> |
| <div class="form-group"> |
| <input type="text" size="30" class="form-control input-sm" name="q" placeholder="Search docs"> |
| <input type="hidden" name="check_keywords" value="yes" /> |
| <input type="hidden" name="area" value="default" /> |
| </div> |
| </form> |
| |
| |
| |
| <ul> |
| <li class="toctree-l1"><a class="reference internal" href="../../getting_started/index.html">Getting Started</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../new/index.html">New Features in Apache Cassandra 4.0</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../architecture/index.html">Architecture</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../cql/index.html">The Cassandra Query Language (CQL)</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../data_modeling/index.html">Data Modeling</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../configuration/index.html">Configuring Cassandra</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../index.html">Operating Cassandra</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../tools/index.html">Cassandra Tools</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../troubleshooting/index.html">Troubleshooting</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../development/index.html">Contributing to Cassandra</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../faq/index.html">Frequently Asked Questions</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../plugins/index.html">Third-Party Plugins</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../bugs.html">Reporting Bugs</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../../contactus.html">Contact us</a></li> |
| </ul> |
| |
| |
| |
| </div><!--/.nav-collapse --> |
| </div> |
| </div> |
| </div> |
| <div class="col-md-8"> |
| <div class="content doc-content"> |
| <div class="content-container"> |
| |
| <div class="section" id="leveled-compaction-strategy"> |
| <span id="lcs"></span><h1>Leveled Compaction Strategy<a class="headerlink" href="#leveled-compaction-strategy" title="Permalink to this headline">¶</a></h1> |
| <p>The idea of <code class="docutils literal notranslate"><span class="pre">LeveledCompactionStrategy</span></code> (LCS) is that all sstables are put into different levels where we guarantee |
| that no overlapping sstables are in the same level. By overlapping we mean that the first/last token of a single sstable |
| are never overlapping with other sstables. This means that for a SELECT we will only have to look for the partition key |
| in a single sstable per level. Each level is 10x the size of the previous one and each sstable is 160MB by default. L0 |
| is where sstables are streamed/flushed - no overlap guarantees are given here.</p> |
| <p>When picking compaction candidates we have to make sure that the compaction does not create overlap in the target level. |
| This is done by always including all overlapping sstables in the next level. For example if we select an sstable in L3, |
| we need to guarantee that we pick all overlapping sstables in L4 and make sure that no currently ongoing compactions |
| will create overlap if we start that compaction. We can start many parallel compactions in a level if we guarantee that |
| we wont create overlap. For L0 -> L1 compactions we almost always need to include all L1 sstables since most L0 sstables |
| cover the full range. We also can’t compact all L0 sstables with all L1 sstables in a single compaction since that can |
| use too much memory.</p> |
| <p>When deciding which level to compact LCS checks the higher levels first (with LCS, a “higher” level is one with a higher |
| number, L0 being the lowest one) and if the level is behind a compaction will be started in that level.</p> |
| <div class="section" id="major-compaction"> |
| <h2>Major compaction<a class="headerlink" href="#major-compaction" title="Permalink to this headline">¶</a></h2> |
| <p>It is possible to do a major compaction with LCS - it will currently start by filling out L1 and then once L1 is full, |
| it continues with L2 etc. This is sub optimal and will change to create all the sstables in a high level instead, |
| CASSANDRA-11817.</p> |
| </div> |
| <div class="section" id="bootstrapping"> |
| <h2>Bootstrapping<a class="headerlink" href="#bootstrapping" title="Permalink to this headline">¶</a></h2> |
| <p>During bootstrap sstables are streamed from other nodes. The level of the remote sstable is kept to avoid many |
| compactions after the bootstrap is done. During bootstrap the new node also takes writes while it is streaming the data |
| from a remote node - these writes are flushed to L0 like all other writes and to avoid those sstables blocking the |
| remote sstables from going to the correct level, we only do STCS in L0 until the bootstrap is done.</p> |
| </div> |
| <div class="section" id="stcs-in-l0"> |
| <h2>STCS in L0<a class="headerlink" href="#stcs-in-l0" title="Permalink to this headline">¶</a></h2> |
| <p>If LCS gets very many L0 sstables reads are going to hit all (or most) of the L0 sstables since they are likely to be |
| overlapping. To more quickly remedy this LCS does STCS compactions in L0 if there are more than 32 sstables there. This |
| should improve read performance more quickly compared to letting LCS do its L0 -> L1 compactions. If you keep getting |
| too many sstables in L0 it is likely that LCS is not the best fit for your workload and STCS could work out better.</p> |
| </div> |
| <div class="section" id="starved-sstables"> |
| <h2>Starved sstables<a class="headerlink" href="#starved-sstables" title="Permalink to this headline">¶</a></h2> |
| <p>If a node ends up with a leveling where there are a few very high level sstables that are not getting compacted they |
| might make it impossible for lower levels to drop tombstones etc. For example, if there are sstables in L6 but there is |
| only enough data to actually get a L4 on the node the left over sstables in L6 will get starved and not compacted. This |
| can happen if a user changes sstable_size_in_mb from 5MB to 160MB for example. To avoid this LCS tries to include |
| those starved high level sstables in other compactions if there has been 25 compaction rounds where the highest level |
| has not been involved.</p> |
| </div> |
| <div class="section" id="lcs-options"> |
| <span id="id1"></span><h2>LCS options<a class="headerlink" href="#lcs-options" title="Permalink to this headline">¶</a></h2> |
| <dl class="docutils"> |
| <dt><code class="docutils literal notranslate"><span class="pre">sstable_size_in_mb</span></code> (default: 160MB)</dt> |
| <dd>The target compressed (if using compression) sstable size - the sstables can end up being larger if there are very |
| large partitions on the node.</dd> |
| <dt><code class="docutils literal notranslate"><span class="pre">fanout_size</span></code> (default: 10)</dt> |
| <dd>The target size of levels increases by this fanout_size multiplier. You can reduce the space amplification by tuning |
| this option.</dd> |
| </dl> |
| <p>LCS also support the <code class="docutils literal notranslate"><span class="pre">cassandra.disable_stcs_in_l0</span></code> startup option (<code class="docutils literal notranslate"><span class="pre">-Dcassandra.disable_stcs_in_l0=true</span></code>) to avoid |
| doing STCS in L0.</p> |
| </div> |
| </div> |
| |
| |
| |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |