blob: 932dc2ff57880abaa11db6f05db2cd2424051e58 [file] [log] [blame]
<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Cassandra Stress &mdash; Apache Cassandra Documentation v4.1</title>
<script type="text/javascript" src="../_static/js/modernizr.min.js"></script>
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<script type="text/javascript" src="../_static/language_data.js"></script>
<script async="async" type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/javascript" src="../_static/js/theme.js"></script>
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/extra.css" type="text/css" />
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Troubleshooting" href="../troubleshooting/index.html" />
<link rel="prev" title="sstableverify" href="sstable/sstableverify.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home"> Apache Cassandra
</a>
<div class="version">
4.1
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../getting_started/index.html">Getting Started</a></li>
<li class="toctree-l1"><a class="reference internal" href="../new/index.html">New Features in Apache Cassandra 4.0</a></li>
<li class="toctree-l1"><a class="reference internal" href="../architecture/index.html">Architecture</a></li>
<li class="toctree-l1"><a class="reference internal" href="../cql/index.html">The Cassandra Query Language (CQL)</a></li>
<li class="toctree-l1"><a class="reference internal" href="../data_modeling/index.html">Data Modeling</a></li>
<li class="toctree-l1"><a class="reference internal" href="../configuration/index.html">Configuring Cassandra</a></li>
<li class="toctree-l1"><a class="reference internal" href="../operating/index.html">Operating Cassandra</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="index.html">Cassandra Tools</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="cqlsh.html">cqlsh: the CQL shell</a></li>
<li class="toctree-l2"><a class="reference internal" href="generatetokens.html">generatetokens</a></li>
<li class="toctree-l2"><a class="reference internal" href="nodetool/nodetool.html">Nodetool</a></li>
<li class="toctree-l2"><a class="reference internal" href="nodetool/nodetool.html#usage">Usage</a></li>
<li class="toctree-l2"><a class="reference internal" href="sstable/index.html">SSTable Tools</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Cassandra Stress</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#usage">Usage</a></li>
<li class="toctree-l3"><a class="reference internal" href="#user-mode">User mode</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#profile">Profile</a></li>
<li class="toctree-l4"><a class="reference internal" href="#lightweight-transaction-support">Lightweight transaction support</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#graphing">Graphing</a></li>
<li class="toctree-l3"><a class="reference internal" href="#faq">FAQ</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../troubleshooting/index.html">Troubleshooting</a></li>
<li class="toctree-l1"><a class="reference internal" href="../development/index.html">Contributing to Cassandra</a></li>
<li class="toctree-l1"><a class="reference internal" href="../faq/index.html">Frequently Asked Questions</a></li>
<li class="toctree-l1"><a class="reference internal" href="../plugins/index.html">Third-Party Plugins</a></li>
<li class="toctree-l1"><a class="reference internal" href="../bugs.html">Reporting Bugs</a></li>
<li class="toctree-l1"><a class="reference internal" href="../contactus.html">Contact us</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="top navigation">
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">Apache Cassandra</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="breadcrumbs navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html">Docs</a> &raquo;</li>
<li><a href="index.html">Cassandra Tools</a> &raquo;</li>
<li>Cassandra Stress</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/tools/cassandra_stress.rst.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<div class="section" id="cassandra-stress">
<span id="id1"></span><h1>Cassandra Stress<a class="headerlink" href="#cassandra-stress" title="Permalink to this headline"></a></h1>
<p>cassandra-stress is a tool for benchmarking and load testing a Cassandra
cluster. cassandra-stress supports testing arbitrary CQL tables and queries
to allow users to benchmark their data model.</p>
<p>This documentation focuses on user mode as this allows the testing of your
actual schema.</p>
<div class="section" id="usage">
<h2>Usage<a class="headerlink" href="#usage" title="Permalink to this headline"></a></h2>
<p>There are several operation types:</p>
<blockquote>
<div><ul class="simple">
<li><p>write-only, read-only, and mixed workloads of standard data</p></li>
<li><p>write-only and read-only workloads for counter columns</p></li>
<li><p>user configured workloads, running custom queries on custom schemas</p></li>
</ul>
</div></blockquote>
<p>The syntax is <cite>cassandra-stress &lt;command&gt; [options]</cite>. If you want more information on a given command
or options, just run <cite>cassandra-stress help &lt;command|option&gt;</cite>.</p>
<dl class="simple">
<dt>Commands:</dt><dd><dl class="simple">
<dt>read:</dt><dd><p>Multiple concurrent reads - the cluster must first be populated by a write test</p>
</dd>
<dt>write:</dt><dd><p>Multiple concurrent writes against the cluster</p>
</dd>
<dt>mixed:</dt><dd><p>Interleaving of any basic commands, with configurable ratio and distribution - the cluster must first be populated by a write test</p>
</dd>
<dt>counter_write:</dt><dd><p>Multiple concurrent updates of counters.</p>
</dd>
<dt>counter_read:</dt><dd><p>Multiple concurrent reads of counters. The cluster must first be populated by a counterwrite test.</p>
</dd>
<dt>user:</dt><dd><p>Interleaving of user provided queries, with configurable ratio and distribution.</p>
</dd>
<dt>help:</dt><dd><p>Print help for a command or option</p>
</dd>
<dt>print:</dt><dd><p>Inspect the output of a distribution definition</p>
</dd>
<dt>legacy:</dt><dd><p>Legacy support mode</p>
</dd>
</dl>
</dd>
<dt>Primary Options:</dt><dd><dl class="simple">
<dt>-pop:</dt><dd><p>Population distribution and intra-partition visit order</p>
</dd>
<dt>-insert:</dt><dd><p>Insert specific options relating to various methods for batching and splitting partition updates</p>
</dd>
<dt>-col:</dt><dd><p>Column details such as size and count distribution, data generator, names, comparator and if super columns should be used</p>
</dd>
<dt>-rate:</dt><dd><p>Thread count, rate limit or automatic mode (default is auto)</p>
</dd>
<dt>-mode:</dt><dd><p>Thrift or CQL with options</p>
</dd>
<dt>-errors:</dt><dd><p>How to handle errors when encountered during stress</p>
</dd>
<dt>-sample:</dt><dd><p>Specify the number of samples to collect for measuring latency</p>
</dd>
<dt>-schema:</dt><dd><p>Replication settings, compression, compaction, etc.</p>
</dd>
<dt>-node:</dt><dd><p>Nodes to connect to</p>
</dd>
<dt>-log:</dt><dd><p>Where to log progress to, and the interval at which to do it</p>
</dd>
<dt>-transport:</dt><dd><p>Custom transport factories</p>
</dd>
<dt>-port:</dt><dd><p>The port to connect to cassandra nodes on</p>
</dd>
<dt>-sendto:</dt><dd><p>Specify a stress server to send this command to</p>
</dd>
<dt>-graph:</dt><dd><p>Graph recorded metrics</p>
</dd>
<dt>-tokenrange:</dt><dd><p>Token range settings</p>
</dd>
</dl>
</dd>
<dt>Suboptions:</dt><dd><p>Every command and primary option has its own collection of suboptions. These are too numerous to list here.
For information on the suboptions for each command or option, please use the help command,
<cite>cassandra-stress help &lt;command|option&gt;</cite>.</p>
</dd>
</dl>
</div>
<div class="section" id="user-mode">
<h2>User mode<a class="headerlink" href="#user-mode" title="Permalink to this headline"></a></h2>
<p>User mode allows you to use your stress your own schemas. This can save time in
the long run rather than building an application and then realising your schema
doesn’t scale.</p>
<div class="section" id="profile">
<h3>Profile<a class="headerlink" href="#profile" title="Permalink to this headline"></a></h3>
<p>User mode requires a profile defined in YAML.
Multiple YAML files may be specified in which case operations in the ops argument are referenced as specname.opname.</p>
<p>An identifier for the profile:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">specname</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">staff_activities</span>
</pre></div>
</div>
<p>The keyspace for the test:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">keyspace</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">staff</span>
</pre></div>
</div>
<p>CQL for the keyspace. Optional if the keyspace already exists:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">keyspace_definition</span><span class="p">:</span> <span class="p p-Indicator">|</span>
<span class="no">CREATE KEYSPACE stresscql WITH replication = {&#39;class&#39;: &#39;SimpleStrategy&#39;, &#39;replication_factor&#39;: 3};</span>
</pre></div>
</div>
<p>The table to be stressed:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">table</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">staff_activities</span>
</pre></div>
</div>
<p>CQL for the table. Optional if the table already exists:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">table_definition</span><span class="p">:</span> <span class="p p-Indicator">|</span>
<span class="no">CREATE TABLE staff_activities (</span>
<span class="no">name text,</span>
<span class="no">when timeuuid,</span>
<span class="no">what text,</span>
<span class="no">PRIMARY KEY(name, when, what)</span>
<span class="no">)</span>
</pre></div>
</div>
<p>Optional meta information on the generated columns in the above table.
The min and max only apply to text and blob types.
The distribution field represents the total unique population
distribution of that column across rows:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">columnspec</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="nt">name</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">name</span>
<span class="nt">size</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">uniform(5..10)</span> <span class="c1"># The names of the staff members are between 5-10 characters</span>
<span class="nt">population</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">uniform(1..10)</span> <span class="c1"># 10 possible staff members to pick from</span>
<span class="p p-Indicator">-</span> <span class="nt">name</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">when</span>
<span class="nt">cluster</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">uniform(20..500)</span> <span class="c1"># Staff members do between 20 and 500 events</span>
<span class="p p-Indicator">-</span> <span class="nt">name</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">what</span>
<span class="nt">size</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">normal(10..100,50)</span>
</pre></div>
</div>
<p>Supported types are:</p>
<p>An exponential distribution over the range [min..max]:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">EXP(min..max)</span>
</pre></div>
</div>
<p>An extreme value (Weibull) distribution over the range [min..max]:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">EXTREME(min..max,shape)</span>
</pre></div>
</div>
<p>A gaussian/normal distribution, where mean=(min+max)/2, and stdev is (mean-min)/stdvrng:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">GAUSSIAN(min..max,stdvrng)</span>
</pre></div>
</div>
<p>A gaussian/normal distribution, with explicitly defined mean and stdev:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">GAUSSIAN(min..max,mean,stdev)</span>
</pre></div>
</div>
<p>A uniform distribution over the range [min, max]:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">UNIFORM(min..max)</span>
</pre></div>
</div>
<p>A fixed distribution, always returning the same value:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">FIXED(val)</span>
</pre></div>
</div>
<p>If preceded by ~, the distribution is inverted</p>
<p>Defaults for all columns are size: uniform(4..8), population: uniform(1..100B), cluster: fixed(1)</p>
<p>Insert distributions:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">insert</span><span class="p">:</span>
<span class="c1"># How many partition to insert per batch</span>
<span class="nt">partitions</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">fixed(1)</span>
<span class="c1"># How many rows to update per partition</span>
<span class="nt">select</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">fixed(1)/500</span>
<span class="c1"># UNLOGGED or LOGGED batch for insert</span>
<span class="nt">batchtype</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">UNLOGGED</span>
</pre></div>
</div>
<p>Currently all inserts are done inside batches.</p>
<p>Read statements to use during the test:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">queries</span><span class="p">:</span>
<span class="nt">events</span><span class="p">:</span>
<span class="nt">cql</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">select * from staff_activities where name = ?</span>
<span class="nt">fields</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">samerow</span>
<span class="nt">latest_event</span><span class="p">:</span>
<span class="nt">cql</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">select * from staff_activities where name = ? LIMIT 1</span>
<span class="nt">fields</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">samerow</span>
</pre></div>
</div>
<p>Running a user mode test:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">cassandra-stress user profile=./example.yaml duration=1m &quot;ops(insert=1,latest_event=1,events=1)&quot; truncate=once</span>
</pre></div>
</div>
<p>This will create the schema then run tests for 1 minute with an equal number of inserts, latest_event queries and events
queries. Additionally the table will be truncated once before the test.</p>
<p>The full example can be found here <a class="reference download internal" download="" href="../_downloads/5f51523351321709939f167577a9b33a/stress-example.yaml"><code class="xref download docutils literal notranslate"><span class="pre">yaml</span></code></a></p>
<dl class="simple">
<dt>Running a user mode test with multiple yaml files::</dt><dd><p>cassandra-stress user profile=./example.yaml,./example2.yaml duration=1m “ops(ex1.insert=1,ex1.latest_event=1,ex2.insert=2)” truncate=once</p>
</dd>
<dt>This will run operations as specified in both the example.yaml and example2.yaml files. example.yaml and example2.yaml can reference the same table</dt><dd><p>although care must be taken that the table definition is identical (data generation specs can be different).</p>
</dd>
</dl>
</div>
<div class="section" id="lightweight-transaction-support">
<h3>Lightweight transaction support<a class="headerlink" href="#lightweight-transaction-support" title="Permalink to this headline"></a></h3>
<p>cassandra-stress supports lightweight transactions. In this it will first read current data from Cassandra and then uses read value(s)
to fulfill lightweight transaction condition(s).</p>
<p>Lightweight transaction update query:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">queries</span><span class="p">:</span>
<span class="nt">regularupdate</span><span class="p">:</span>
<span class="nt">cql</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">update blogposts set author = ? where domain = ? and published_date = ?</span>
<span class="nt">fields</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">samerow</span>
<span class="nt">updatewithlwt</span><span class="p">:</span>
<span class="nt">cql</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">update blogposts set author = ? where domain = ? and published_date = ? IF body = ? AND url = ?</span>
<span class="nt">fields</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">samerow</span>
</pre></div>
</div>
<p>The full example can be found here <a class="reference download internal" download="" href="../_downloads/41488e7fbf1958f77971843a5a2fdece/stress-lwt-example.yaml"><code class="xref download docutils literal notranslate"><span class="pre">yaml</span></code></a></p>
</div>
</div>
<div class="section" id="graphing">
<h2>Graphing<a class="headerlink" href="#graphing" title="Permalink to this headline"></a></h2>
<p>Graphs can be generated for each run of stress.</p>
<img alt="../_images/example-stress-graph.png" src="../_images/example-stress-graph.png" />
<p>To create a new graph:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">cassandra-stress user profile=./stress-example.yaml &quot;ops(insert=1,latest_event=1,events=1)&quot; -graph file=graph.html title=&quot;Awesome graph&quot;</span>
</pre></div>
</div>
<p>To add a new run to an existing graph point to an existing file and add a revision name:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">cassandra-stress user profile=./stress-example.yaml duration=1m &quot;ops(insert=1,latest_event=1,events=1)&quot; -graph file=graph.html title=&quot;Awesome graph&quot; revision=&quot;Second run&quot;</span>
</pre></div>
</div>
</div>
<div class="section" id="faq">
<h2>FAQ<a class="headerlink" href="#faq" title="Permalink to this headline"></a></h2>
<p><strong>How do you use NetworkTopologyStrategy for the keyspace?</strong></p>
<p>Use the schema option making sure to either escape the parenthesis or enclose in quotes:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">cassandra-stress write -schema &quot;replication(strategy=NetworkTopologyStrategy,datacenter1=3)&quot;</span>
</pre></div>
</div>
<p><strong>How do you use SSL?</strong></p>
<p>Use the transport option:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="l l-Scalar l-Scalar-Plain">cassandra-stress &quot;write n=100k cl=ONE no-warmup&quot; -transport &quot;truststore=$HOME/jks/truststore.jks truststore-password=cassandra&quot;</span>
</pre></div>
</div>
<p><strong>Is Cassandra Stress a secured tool?</strong></p>
<p>Cassandra stress is not a secured tool. Serialization and other aspects of the tool offer no security guarantees.</p>
</div>
</div>
</div>
</div>
<footer>
<div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
<a href="../troubleshooting/index.html" class="btn btn-neutral float-right" title="Troubleshooting" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right"></span></a>
<a href="sstable/sstableverify.html" class="btn btn-neutral float-left" title="sstableverify" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
</div>
<hr/>
<div role="contentinfo">
<p>
&copy; Copyright 2020, The Apache Cassandra team
</p>
</div>
Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script type="text/javascript">
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>