docs/topics/impala_perf_resources.xml

<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->
<!DOCTYPE concept PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
<concept id="mem_limits">

  <title>Controlling Impala Resource Usage</title>
  <titlealts audience="PDF"><navtitle>Controlling Resource Usage</navtitle></titlealts>
  <prolog>
    <metadata>
      <data name="Category" value="Impala"/>
      <data name="Category" value="Performance"/>
      <data name="Category" value="Memory"/>
      <data name="Category" value="Scalability"/>
      <data name="Category" value="Resource Management"/>
      <data name="Category" value="Administrators"/>
      <data name="Category" value="Developers"/>
      <data name="Category" value="Data Analysts"/>
    </metadata>
  </prolog>

  <conbody>

    <p>
      Sometimes, balancing raw query performance against scalability requires limiting the amount of resources,
      such as memory or CPU, used by a single query or group of queries. Impala can use several mechanisms that
      help to smooth out the load during heavy concurrent usage, resulting in faster overall query times and
      sharing of resources across Impala queries, MapReduce jobs, and other kinds of workloads across a <keyword keyref="distro"/>
      cluster:
    </p>

    <ul>
      <li>
        The Impala admission control feature uses a fast, distributed mechanism to hold back queries that exceed
        limits on the number of concurrent queries or the amount of memory used. The queries are queued, and
        executed as other queries finish and resources become available. You can control the concurrency limits,
        and specify different limits for different groups of users to divide cluster resources according to the
        priorities of different classes of users. This feature is new in Impala 1.3.
        See <xref href="impala_admission.xml#admission_control"/> for details.
      </li>

      <li>
        <p>
          You can restrict the amount of memory Impala reserves during query execution by specifying the
          <codeph>-mem_limit</codeph> option for the <codeph>impalad</codeph> daemon. See
          <xref href="impala_config_options.xml#config_options"/> for details. This limit applies only to the
          memory that is directly consumed by queries; Impala reserves additional memory at startup, for example to
          hold cached metadata.
        </p>
      </li>

      <li>
        <p>
          For production deployments, implement resource isolation using your cluster management
          tool.
        </p>
      </li>
    </ul>
  </conbody>
</concept>