| <?xml version="1.0" encoding="UTF-8"?> |
| <!DOCTYPE html |
| PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> |
| <head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
| |
| <meta name="copyright" content="(C) Copyright 2023" /> |
| <meta name="DC.rights.owner" content="(C) Copyright 2023" /> |
| <meta name="DC.Type" content="concept" /> |
| <meta name="DC.Title" content="MIN_SPILLABLE_BUFFER_SIZE Query Option" /> |
| <meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" /> |
| <meta name="prodname" content="Impala" /> |
| <meta name="prodname" content="Impala" /> |
| <meta name="version" content="Impala 3.4.x" /> |
| <meta name="version" content="Impala 3.4.x" /> |
| <meta name="DC.Format" content="XHTML" /> |
| <meta name="DC.Identifier" content="min_spillable_buffer_size" /> |
| <link rel="stylesheet" type="text/css" href="../commonltr.css" /> |
| <title>MIN_SPILLABLE_BUFFER_SIZE Query Option</title> |
| </head> |
| <body id="min_spillable_buffer_size"> |
| |
| |
| <h1 class="title topictitle1" id="ariaid-title1">MIN_SPILLABLE_BUFFER_SIZE Query Option</h1> |
| |
| |
| |
| |
| <div class="body conbody"> |
| |
| <p class="p"> |
| |
| Specifies the minimum size for a memory buffer used when the |
| spill-to-disk mechanism is activated, for example for queries against |
| a large table with no statistics, or large join operations. |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Type:</strong> integer |
| </p> |
| |
| |
| |
| |
| <p class="p"> |
| <strong class="ph b">Default:</strong> |
| </p> |
| |
| <p class="p"> |
| <code class="ph codeph">65536</code> (64 KB) |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Units:</strong> A numeric argument represents a size in bytes; you can also use a suffix |
| of <code class="ph codeph">m</code> or <code class="ph codeph">mb</code> for megabytes, or <code class="ph codeph">g</code> or |
| <code class="ph codeph">gb</code> for gigabytes. If you specify a value with unrecognized formats, |
| subsequent queries fail with an error. |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.10.0</span> |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Usage notes:</strong> |
| </p> |
| |
| <p class="p"> |
| This query option sets a lower bound on the size of the internal |
| buffer size that can be used during spill-to-disk operations. The |
| actual size of the buffer is chosen by the query planner. |
| </p> |
| |
| <p class="p"> |
| If overall query performance is limited by the time needed for spilling, |
| consider increasing the <code class="ph codeph">MIN_SPILLABLE_BUFFER_SIZE</code> setting. |
| Larger buffer sizes result in Impala issuing larger I/O requests to storage |
| devices, which might result in higher throughput, particularly on rotational |
| disks. |
| </p> |
| |
| <p class="p"> |
| The tradeoff with a large value for this setting is increased memory usage during |
| spill-to-disk operations. Reducing this value may reduce memory consumption. |
| </p> |
| |
| <p class="p"> |
| To determine if the value for this setting is having an effect by capping the |
| spillable buffer size, you can see the buffer size chosen by the query planner for |
| a particular query. <code class="ph codeph">EXPLAIN</code> the query while the setting |
| <code class="ph codeph">EXPLAIN_LEVEL=2</code> is in effect. |
| </p> |
| |
| |
| <p class="p"> |
| <strong class="ph b">Examples:</strong> |
| </p> |
| |
| |
| <pre class="pre codeblock"><code> |
| set min_spillable_buffer_size=128KB; |
| |
| </code></pre> |
| |
| <p class="p"> |
| <strong class="ph b">Related information:</strong> |
| </p> |
| |
| <p class="p"> |
| <a class="xref" href="impala_buffer_pool_limit.html">BUFFER_POOL_LIMIT Query Option</a>, |
| <a class="xref" href="impala_default_spillable_buffer_size.html">DEFAULT_SPILLABLE_BUFFER_SIZE Query Option</a>, |
| <a class="xref" href="impala_max_row_size.html">MAX_ROW_SIZE Query Option</a>, |
| <a class="xref" href="impala_scalability.html">Scalability Considerations for Impala</a> |
| </p> |
| |
| |
| </div> |
| |
| <div class="related-links"> |
| <div class="familylinks"> |
| <div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div> |
| </div> |
| </div></body> |
| </html> |