docs/topics/impala_disable_unsafe_spills.xml

<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->
<!DOCTYPE concept PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
<concept rev="2.0.0" id="disable_unsafe_spills">

  <title>DISABLE_UNSAFE_SPILLS Query Option (<keyword keyref="impala20"/> or higher only)</title>
  <titlealts audience="PDF"><navtitle>DISABLE_UNSAFE_SPILLS</navtitle></titlealts>
  <prolog>
    <metadata>
      <data name="Category" value="Impala"/>
      <data name="Category" value="Impala Query Options"/>
      <data name="Category" value="Performance"/>
      <data name="Category" value="Scalability"/>
      <data name="Category" value="Memory"/>
      <data name="Category" value="Developers"/>
      <data name="Category" value="Data Analysts"/>
    </metadata>
  </prolog>

  <conbody>

    <p rev="2.0.0">
      <indexterm audience="hidden">DISABLE_UNSAFE_SPILLS query option</indexterm>
      Enable this option if you prefer to have queries fail when they exceed the Impala memory limit, rather than
      write temporary data to disk.
    </p>

    <p>
      Queries that <q>spill</q> to disk typically complete successfully, when in earlier Impala releases they would have failed.
      However, queries with exorbitant memory requirements due to missing statistics or inefficient join clauses could
      become so slow as a result that you would rather have them cancelled automatically and reduce the memory
      usage through standard Impala tuning techniques.
    </p>

    <p>
      This option prevents only <q>unsafe</q> spill operations, meaning that one or more tables are missing
      statistics or the query does not include a hint to set the most efficient mechanism for a join or
      <codeph>INSERT ... SELECT</codeph> into a partitioned table. These are the tables most likely to result in
      suboptimal execution plans that could cause unnecessary spilling. Therefore, leaving this option enabled is a
      good way to find tables on which to run the <codeph>COMPUTE STATS</codeph> statement.
    </p>

    <p>
      See <xref href="impala_scalability.xml#spill_to_disk"/> for information about the <q>spill to disk</q>
      feature for queries processing large result sets with joins, <codeph>ORDER BY</codeph>, <codeph>GROUP
      BY</codeph>, <codeph>DISTINCT</codeph>, aggregation functions, or analytic functions.
    </p>

    <p conref="../shared/impala_common.xml#common/type_boolean"/>
    <p conref="../shared/impala_common.xml#common/default_false_0"/>

    <p conref="../shared/impala_common.xml#common/added_in_20"/>
  </conbody>
</concept>