| --- |
| layout: docpage |
| |
| title: "Documentation" |
| |
| is_homepage: false |
| is_sphinx_doc: true |
| |
| doc-parent: "Getting Started" |
| |
| doc-title: "Production Recommendations" |
| doc-header-links: ' |
| <link rel="top" title="Apache Cassandra Documentation v4.0-beta4" href="../index.html"/> |
| <link rel="up" title="Getting Started" href="index.html"/> |
| <link rel="next" title="New Features in Apache Cassandra 4.0" href="../new/index.html"/> |
| <link rel="prev" title="Client drivers" href="drivers.html"/> |
| ' |
| doc-search-path: "../search.html" |
| |
| extra-footer: ' |
| <script type="text/javascript"> |
| var DOCUMENTATION_OPTIONS = { |
| URL_ROOT: "", |
| VERSION: "", |
| COLLAPSE_INDEX: false, |
| FILE_SUFFIX: ".html", |
| HAS_SOURCE: false, |
| SOURCELINK_SUFFIX: ".txt" |
| }; |
| </script> |
| ' |
| |
| --- |
| <div class="container-fluid"> |
| <div class="row"> |
| <div class="col-md-3"> |
| <div class="doc-navigation"> |
| <div class="doc-menu" role="navigation"> |
| <div class="navbar-header"> |
| <button type="button" class="pull-left navbar-toggle" data-toggle="collapse" data-target=".sidebar-navbar-collapse"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| </div> |
| <div class="navbar-collapse collapse sidebar-navbar-collapse"> |
| <form id="doc-search-form" class="navbar-form" action="../search.html" method="get" role="search"> |
| <div class="form-group"> |
| <input type="text" size="30" class="form-control input-sm" name="q" placeholder="Search docs"> |
| <input type="hidden" name="check_keywords" value="yes" /> |
| <input type="hidden" name="area" value="default" /> |
| </div> |
| </form> |
| |
| |
| |
| <ul class="current"> |
| <li class="toctree-l1 current"><a class="reference internal" href="index.html">Getting Started</a><ul class="current"> |
| <li class="toctree-l2"><a class="reference internal" href="installing.html">Installing Cassandra</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="configuring.html">Configuring Cassandra</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="querying.html">Inserting and querying</a></li> |
| <li class="toctree-l2"><a class="reference internal" href="drivers.html">Client drivers</a></li> |
| <li class="toctree-l2 current"><a class="current reference internal" href="#">Production Recommendations</a><ul> |
| <li class="toctree-l3"><a class="reference internal" href="#tokens">Tokens</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#read-ahead">Read Ahead</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#compression">Compression</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#compaction">Compaction</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#encryption">Encryption</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#ensure-keyspaces-are-created-with-networktopologystrategy">Ensure Keyspaces are Created with NetworkTopologyStrategy</a></li> |
| <li class="toctree-l3"><a class="reference internal" href="#configure-racks-and-snitch">Configure Racks and Snitch</a></li> |
| </ul> |
| </li> |
| </ul> |
| </li> |
| <li class="toctree-l1"><a class="reference internal" href="../new/index.html">New Features in Apache Cassandra 4.0</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../architecture/index.html">Architecture</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../cql/index.html">The Cassandra Query Language (CQL)</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../data_modeling/index.html">Data Modeling</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../configuration/index.html">Configuring Cassandra</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../operating/index.html">Operating Cassandra</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../tools/index.html">Cassandra Tools</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../troubleshooting/index.html">Troubleshooting</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../development/index.html">Contributing to Cassandra</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../faq/index.html">Frequently Asked Questions</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../plugins/index.html">Third-Party Plugins</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../bugs.html">Reporting Bugs</a></li> |
| <li class="toctree-l1"><a class="reference internal" href="../contactus.html">Contact us</a></li> |
| </ul> |
| |
| |
| |
| </div><!--/.nav-collapse --> |
| </div> |
| </div> |
| </div> |
| <div class="col-md-8"> |
| <div class="content doc-content"> |
| <div class="content-container"> |
| |
| <div class="section" id="production-recommendations"> |
| <h1>Production Recommendations<a class="headerlink" href="#production-recommendations" title="Permalink to this headline">¶</a></h1> |
| <p>The <code class="docutils literal notranslate"><span class="pre">cassandra.yaml</span></code> and <code class="docutils literal notranslate"><span class="pre">jvm.options</span></code> files have a number of notes and recommendations for production usage. This page |
| expands on some of the notes in these files with additional information.</p> |
| <div class="section" id="tokens"> |
| <h2>Tokens<a class="headerlink" href="#tokens" title="Permalink to this headline">¶</a></h2> |
| <p>Using more than 1 token (referred to as vnodes) allows for more flexible expansion and more streaming peers when |
| bootstrapping new nodes into the cluster. This can limit the negative impact of streaming (I/O and CPU overhead) |
| as well as allow for incremental cluster expansion.</p> |
| <p>As a tradeoff, more tokens will lead to sharing data with more peers, which can result in decreased availability. To learn more about this we |
| recommend reading <a class="reference external" href="https://github.com/jolynch/python_performance_toolkit/raw/master/notebooks/cassandra_availability/whitepaper/cassandra-availability-virtual.pdf">this paper</a>.</p> |
| <p>The number of tokens can be changed using the following setting:</p> |
| <p><code class="docutils literal notranslate"><span class="pre">num_tokens:</span> <span class="pre">16</span></code></p> |
| <p>Here are the most common token counts with a brief explanation of when and why you would use each one.</p> |
| <table border="1" class="docutils"> |
| <colgroup> |
| <col width="12%" /> |
| <col width="88%" /> |
| </colgroup> |
| <thead valign="bottom"> |
| <tr class="row-odd"><th class="head">Token Count</th> |
| <th class="head">Description</th> |
| </tr> |
| </thead> |
| <tbody valign="top"> |
| <tr class="row-even"><td>1</td> |
| <td>Maximum availablility, maximum cluster size, fewest peers, |
| but inflexible expansion. Must always |
| double size of cluster to expand and remain balanced.</td> |
| </tr> |
| <tr class="row-odd"><td>4</td> |
| <td>A healthy mix of elasticity and availability. Recommended for clusters which will eventually |
| reach over 30 nodes. Requires adding approximately 20% more nodes to remain balanced. |
| Shrinking a cluster may result in cluster imbalance.</td> |
| </tr> |
| <tr class="row-even"><td>16</td> |
| <td>Best for heavily elastic clusters which expand and shrink regularly, but may have issues |
| availability with larger clusters. Not recommended for clusters over 50 nodes.</td> |
| </tr> |
| </tbody> |
| </table> |
| <p>In addition to setting the token count, it’s extremely important that <code class="docutils literal notranslate"><span class="pre">allocate_tokens_for_local_replication_factor</span></code> be |
| set as well, to ensure even token allocation.</p> |
| </div> |
| <div class="section" id="read-ahead"> |
| <span id="id1"></span><h2>Read Ahead<a class="headerlink" href="#read-ahead" title="Permalink to this headline">¶</a></h2> |
| <p>Read ahead is an operating system feature that attempts to keep as much data loaded in the page cache as possible. The |
| goal is to decrease latency by using additional throughput on reads where the latency penalty is high due to seek times |
| on spinning disks. By leveraging read ahead, the OS can pull additional data into memory without the cost of additional |
| seeks. This works well when available RAM is greater than the size of the hot dataset, but can be problematic when the |
| hot dataset is much larger than available RAM. The benefit of read ahead decreases as the size of your hot dataset gets |
| bigger in proportion to available memory.</p> |
| <p>With small partitions (usually tables with no partition key, but not limited to this case) and solid state drives, read |
| ahead can increase disk usage without any of the latency benefits, and in some cases can result in up to |
| a 5x latency and throughput performance penalty. Read heavy, key/value tables with small (under 1KB) rows are especially |
| prone to this problem.</p> |
| <p>We recommend the following read ahead settings:</p> |
| <table border="1" class="docutils"> |
| <colgroup> |
| <col width="39%" /> |
| <col width="61%" /> |
| </colgroup> |
| <thead valign="bottom"> |
| <tr class="row-odd"><th class="head">Hardware</th> |
| <th class="head">Initial Recommendation</th> |
| </tr> |
| </thead> |
| <tbody valign="top"> |
| <tr class="row-even"><td>Spinning Disks</td> |
| <td>64KB</td> |
| </tr> |
| <tr class="row-odd"><td>SSD</td> |
| <td>4KB</td> |
| </tr> |
| </tbody> |
| </table> |
| <p>Read ahead can be adjusted on Linux systems by using the <cite>blockdev</cite> tool.</p> |
| <p>For example, we can set read ahead of <a href="#id2"><span class="problematic" id="id3">``</span></a>/dev/sda1` to 4KB by doing the following:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">blockdev</span> <span class="o">--</span><span class="n">setra</span> <span class="mi">8</span> <span class="o">/</span><span class="n">dev</span><span class="o">/</span><span class="n">sda1</span> |
| </pre></div> |
| </div> |
| <p><strong>Note</strong>: blockdev accepts the number of 512 byte sectors to read ahead. The argument of 8 above is equivilent to 4KB.</p> |
| <p>Since each system is different, use the above recommendations as a starting point and tuning based on your SLA and |
| throughput requirements. To understand how read ahead impacts disk resource usage we recommend carefully reading through the |
| <a class="reference internal" href="../troubleshooting/use_tools.html#use-os-tools"><span class="std std-ref">troubleshooting</span></a> portion of the documentation.</p> |
| </div> |
| <div class="section" id="compression"> |
| <h2>Compression<a class="headerlink" href="#compression" title="Permalink to this headline">¶</a></h2> |
| <p>Compressed data is stored by compressing fixed size byte buffers and writing the data to disk. The buffer size is |
| determined by the <code class="docutils literal notranslate"><span class="pre">chunk_length_in_kb</span></code> element in the compression map of the schema settings.</p> |
| <p>The default setting is 16KB starting with Cassandra 4.0.</p> |
| <p>Since the entire compressed buffer must be read off disk, using too high of a compression chunk length can lead to |
| significant overhead when reading small records. Combined with the default read ahead setting this can result in massive |
| read amplification for certain workloads.</p> |
| <p>LZ4Compressor is the default and recommended compression algorithm.</p> |
| <p>There is additional information on this topic on <a class="reference external" href="https://thelastpickle.com/blog/2018/08/08/compression_performance.html">The Last Pickle Blog</a>.</p> |
| </div> |
| <div class="section" id="compaction"> |
| <h2>Compaction<a class="headerlink" href="#compaction" title="Permalink to this headline">¶</a></h2> |
| <p>There are different <a class="reference internal" href="../operating/compaction/index.html#compaction"><span class="std std-ref">compaction</span></a> strategies available for different workloads. |
| We recommend reading up on the different strategies to understand which is the best for your environment. Different tables |
| may (and frequently do) use different compaction strategies on the same cluster.</p> |
| </div> |
| <div class="section" id="encryption"> |
| <h2>Encryption<a class="headerlink" href="#encryption" title="Permalink to this headline">¶</a></h2> |
| <p>It is significantly easier to set up peer to peer encryption and client server encryption when setting up your production |
| cluster as opposed to setting it up once the cluster is already serving production traffic. If you are planning on using network encryption |
| eventually (in any form), we recommend setting it up now. Changing these configurations down the line is not impossible, |
| but mistakes can result in downtime or data loss.</p> |
| </div> |
| <div class="section" id="ensure-keyspaces-are-created-with-networktopologystrategy"> |
| <h2>Ensure Keyspaces are Created with NetworkTopologyStrategy<a class="headerlink" href="#ensure-keyspaces-are-created-with-networktopologystrategy" title="Permalink to this headline">¶</a></h2> |
| <p>Production clusters should never use SimpleStrategy. Production keyspaces should use the NetworkTopologyStrategy (NTS).</p> |
| <p>For example:</p> |
| <div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">create</span> <span class="n">KEYSPACE</span> <span class="n">mykeyspace</span> <span class="n">WITH</span> <span class="n">replication</span> <span class="o">=</span> |
| <span class="p">{</span><span class="s1">'class'</span><span class="p">:</span> <span class="s1">'NetworkTopologyStrategy'</span><span class="p">,</span> <span class="s1">'datacenter1'</span><span class="p">:</span> <span class="mi">3</span><span class="p">};</span> |
| </pre></div> |
| </div> |
| <p>NetworkTopologyStrategy allows Cassandra to take advantage of multiple racks and data centers.</p> |
| </div> |
| <div class="section" id="configure-racks-and-snitch"> |
| <h2>Configure Racks and Snitch<a class="headerlink" href="#configure-racks-and-snitch" title="Permalink to this headline">¶</a></h2> |
| <p><strong>Correctly configuring or changing racks after a cluster has been provisioned is an unsupported process</strong>. Migrating from |
| a single rack to multiple racks is also unsupported and can result in data loss.</p> |
| <p>Using <code class="docutils literal notranslate"><span class="pre">GossipingPropertyFileSnitch</span></code> is the most flexible solution for on premise or mixed cloud environments. <code class="docutils literal notranslate"><span class="pre">Ec2Snitch</span></code> |
| is reliable for AWS EC2 only environments.</p> |
| </div> |
| </div> |
| |
| |
| |
| |
| <div class="doc-prev-next-links" role="navigation" aria-label="footer navigation"> |
| |
| <a href="../new/index.html" class="btn btn-default pull-right " role="button" title="New Features in Apache Cassandra 4.0" accesskey="n">Next <span class="glyphicon glyphicon-circle-arrow-right" aria-hidden="true"></span></a> |
| |
| |
| <a href="drivers.html" class="btn btn-default" role="button" title="Client drivers" accesskey="p"><span class="glyphicon glyphicon-circle-arrow-left" aria-hidden="true"></span> Previous</a> |
| |
| </div> |
| |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
