docs/build/plain-html/topics/impala_live_summary.html - impala - Git at Google

 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html
   PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

 <meta name="copyright" content="(C) Copyright 2023" />
 <meta name="DC.rights.owner" content="(C) Copyright 2023" />
 <meta name="DC.Type" content="concept" />
 <meta name="DC.Title" content="LIVE_SUMMARY Query Option (Impala 2.3 or higher only)" />
 <meta name="DC.Relation" scheme="URI" content="../topics/impala_set.html" />
 <meta name="prodname" content="Impala" />
 <meta name="prodname" content="Impala" />
 <meta name="version" content="Impala 3.4.x" />
 <meta name="version" content="Impala 3.4.x" />
 <meta name="DC.Format" content="XHTML" />
 <meta name="DC.Identifier" content="live_summary" />
 <link rel="stylesheet" type="text/css" href="../commonltr.css" />
 <title>LIVE_SUMMARY Query Option (Impala 2.3 or higher only)</title>
 </head>
 <body id="live_summary">


   <h1 class="title topictitle1" id="ariaid-title1">LIVE_SUMMARY Query Option (<span class="keyword">Impala 2.3</span> or higher only)</h1>


   <div class="body conbody">
     <p class="p"> When the <code class="ph codeph">LIVE_SUMMARY</code> query option is set
       to <code class="ph codeph">TRUE</code>, Impala displays the same output as the
         <code class="ph codeph">SUMMARY</code> command for queries submitted through the
         <span class="keyword cmdname">impala-shell</span> command, with the measurements updated
       in real time as the query progresses. When the query finishes, the final
         <code class="ph codeph">SUMMARY</code> output remains visible in the
         <span class="keyword cmdname">impala-shell</span> console output. </p>

     <p class="p">Starting in <span class="keyword">Impala 3.1</span>, the summary output also
       includes the queuing status consisting of whether the query was queued and
       what was the latest queuing reason.</p>

     <p class="p"><strong class="ph b">Type:</strong>
       <code class="ph codeph">Boolean</code></p>

     <p class="p"><strong class="ph b">Default:</strong>
       <code class="ph codeph">FALSE (0)</code></p>

     <p class="p">
         <strong class="ph b">Command-line equivalent:</strong>
       </p>

     <p class="p"> You can enable this query option within <span class="keyword cmdname">impala-shell</span>
       by starting the shell with the <code class="ph codeph">--live_summary</code>
       command-line option. You can still turn this setting off and on again
       within the shell through the <code class="ph codeph">SET</code> command. </p>

     <p class="p">
         <strong class="ph b">Usage notes:</strong>
       </p>

     <p class="p"> The live summary output can be useful for evaluating long-running
       queries, to evaluate which phase of execution takes up the most time, or
       if some hosts take much longer than others for certain operations,
       dragging overall performance down. By making the information available in
       real time, this feature lets you decide what action to take even before
       you cancel a query that is taking much longer than normal. </p>

     <p class="p"> For example, you might see the HDFS scan phase taking a long time, and
       therefore revisit performance-related aspects of your schema design such
       as constructing a partitioned table, switching to the Parquet file format,
       running the <code class="ph codeph">COMPUTE STATS</code> statement for the table, and so
       on. Or you might see a wide variation between the average and maximum
       times for all hosts to perform some phase of the query, and therefore
       investigate if one particular host needed more memory or was experiencing
       a network problem. </p>

     <p class="p">
         The output from this query option is printed to standard
         error. The output is  displayed only in interactive mode, not when the <code class="ph codeph">-q</code>
         or <code class="ph codeph">-f</code> options are used.
       </p>

     <p class="p"> For a simple and concise way of tracking the progress of an interactive
       query, see <a class="xref" href="impala_live_progress.html#live_progress">LIVE_PROGRESS Query Option (Impala 2.3 or higher only)</a>. </p>

     <p class="p">
         <strong class="ph b">Restrictions:</strong>
       </p>

     <p class="p">
         The <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query options
         currently do not produce any output during <code class="ph codeph">COMPUTE STATS</code> operations.
       </p>

     <div class="p">
         Because the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
         options are available only within the <span class="keyword cmdname">impala-shell</span> interpreter:
         <ul class="ul">
           <li class="li">
             <p class="p">
               You cannot change these query options through the SQL <code class="ph codeph">SET</code>
               statement using the JDBC or ODBC interfaces. The <code class="ph codeph">SET</code> command in
               <span class="keyword cmdname">impala-shell</span> recognizes these names as shell-only options.
             </p>

           </li>


           <li class="li">
             <p class="p">
               Be careful when using <span class="keyword cmdname">impala-shell</span> on a
               pre-<span class="keyword">Impala 2.3</span> system to connect to a system running
               <span class="keyword">Impala 2.3</span> or higher. The older <span class="keyword cmdname">impala-shell</span>
               does not recognize these query option names. Upgrade
               <span class="keyword cmdname">impala-shell</span> on the systems where you intend to use these query
               options.
             </p>

           </li>


           <li class="li">
             <p class="p">
               Likewise, the <span class="keyword cmdname">impala-shell</span> command relies on some information
               only available in <span class="keyword">Impala 2.3</span> and higher to prepare live
               progress reports and query summaries. The <code class="ph codeph">LIVE_PROGRESS</code> and
               <code class="ph codeph">LIVE_SUMMARY</code> query options have no effect when
               <span class="keyword cmdname">impala-shell</span> connects to a cluster running an older version of
               Impala.
             </p>

           </li>

         </ul>

       </div>

     <p class="p">
         <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.3.0</span>
       </p>

     <p class="p">To see how the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
         options work in real time, see
         <a class="xref" href="https://asciinema.org/a/1rv7qippo0fe7h5k1b6k4nexk" target="_blank">this
           animated demo</a>.
       </p>

   </div>

 <div class="related-links">
 <div class="familylinks">
 <div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_set.html">SET Statement</a></div>
 </div>
 </div></body>
 </html>
	<?xml version="1.0" encoding="UTF-8"?>
	<!DOCTYPE html
	PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

	<meta name="copyright" content="(C) Copyright 2023" />
	<meta name="DC.rights.owner" content="(C) Copyright 2023" />
	<meta name="DC.Type" content="concept" />
	<meta name="DC.Title" content="LIVE_SUMMARY Query Option (Impala 2.3 or higher only)" />
	<meta name="DC.Relation" scheme="URI" content="../topics/impala_set.html" />
	<meta name="prodname" content="Impala" />
	<meta name="prodname" content="Impala" />
	<meta name="version" content="Impala 3.4.x" />
	<meta name="version" content="Impala 3.4.x" />
	<meta name="DC.Format" content="XHTML" />
	<meta name="DC.Identifier" content="live_summary" />
	<link rel="stylesheet" type="text/css" href="../commonltr.css" />
	<title>LIVE_SUMMARY Query Option (Impala 2.3 or higher only)</title>
	</head>
	<body id="live_summary">


	<h1 class="title topictitle1" id="ariaid-title1">LIVE_SUMMARY Query Option (<span class="keyword">Impala 2.3</span> or higher only)</h1>




	<div class="body conbody">
	<p class="p"> When the <code class="ph codeph">LIVE_SUMMARY</code> query option is set
	to <code class="ph codeph">TRUE</code>, Impala displays the same output as the
	<code class="ph codeph">SUMMARY</code> command for queries submitted through the
	<span class="keyword cmdname">impala-shell</span> command, with the measurements updated
	in real time as the query progresses. When the query finishes, the final
	<code class="ph codeph">SUMMARY</code> output remains visible in the
	<span class="keyword cmdname">impala-shell</span> console output. </p>

	<p class="p">Starting in <span class="keyword">Impala 3.1</span>, the summary output also
	includes the queuing status consisting of whether the query was queued and
	what was the latest queuing reason.</p>

	<p class="p"><strong class="ph b">Type:</strong>
	<code class="ph codeph">Boolean</code></p>

	<p class="p"><strong class="ph b">Default:</strong>
	<code class="ph codeph">FALSE (0)</code></p>

	<p class="p">
	<strong class="ph b">Command-line equivalent:</strong>
	</p>

	<p class="p"> You can enable this query option within <span class="keyword cmdname">impala-shell</span>
	by starting the shell with the <code class="ph codeph">--live_summary</code>
	command-line option. You can still turn this setting off and on again
	within the shell through the <code class="ph codeph">SET</code> command. </p>

	<p class="p">
	<strong class="ph b">Usage notes:</strong>
	</p>

	<p class="p"> The live summary output can be useful for evaluating long-running
	queries, to evaluate which phase of execution takes up the most time, or
	if some hosts take much longer than others for certain operations,
	dragging overall performance down. By making the information available in
	real time, this feature lets you decide what action to take even before
	you cancel a query that is taking much longer than normal. </p>

	<p class="p"> For example, you might see the HDFS scan phase taking a long time, and
	therefore revisit performance-related aspects of your schema design such
	as constructing a partitioned table, switching to the Parquet file format,
	running the <code class="ph codeph">COMPUTE STATS</code> statement for the table, and so
	on. Or you might see a wide variation between the average and maximum
	times for all hosts to perform some phase of the query, and therefore
	investigate if one particular host needed more memory or was experiencing
	a network problem. </p>

	<p class="p">
	The output from this query option is printed to standard
	error. The output is displayed only in interactive mode, not when the <code class="ph codeph">-q</code>
	or <code class="ph codeph">-f</code> options are used.
	</p>

	<p class="p"> For a simple and concise way of tracking the progress of an interactive
	query, see <a class="xref" href="impala_live_progress.html#live_progress">LIVE_PROGRESS Query Option (Impala 2.3 or higher only)</a>. </p>

	<p class="p">
	<strong class="ph b">Restrictions:</strong>
	</p>

	<p class="p">
	The <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query options
	currently do not produce any output during <code class="ph codeph">COMPUTE STATS</code> operations.
	</p>

	<div class="p">
	Because the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
	options are available only within the <span class="keyword cmdname">impala-shell</span> interpreter:
	<ul class="ul">
	<li class="li">
	<p class="p">
	You cannot change these query options through the SQL <code class="ph codeph">SET</code>
	statement using the JDBC or ODBC interfaces. The <code class="ph codeph">SET</code> command in
	<span class="keyword cmdname">impala-shell</span> recognizes these names as shell-only options.
	</p>

	</li>


	<li class="li">
	<p class="p">
	Be careful when using <span class="keyword cmdname">impala-shell</span> on a
	pre-<span class="keyword">Impala 2.3</span> system to connect to a system running
	<span class="keyword">Impala 2.3</span> or higher. The older <span class="keyword cmdname">impala-shell</span>
	does not recognize these query option names. Upgrade
	<span class="keyword cmdname">impala-shell</span> on the systems where you intend to use these query
	options.
	</p>

	</li>


	<li class="li">
	<p class="p">
	Likewise, the <span class="keyword cmdname">impala-shell</span> command relies on some information
	only available in <span class="keyword">Impala 2.3</span> and higher to prepare live
	progress reports and query summaries. The <code class="ph codeph">LIVE_PROGRESS</code> and
	<code class="ph codeph">LIVE_SUMMARY</code> query options have no effect when
	<span class="keyword cmdname">impala-shell</span> connects to a cluster running an older version of
	Impala.
	</p>

	</li>

	</ul>

	</div>

	<p class="p">
	<strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.3.0</span>
	</p>

	<p class="p">To see how the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
	options work in real time, see
	<a class="xref" href="https://asciinema.org/a/1rv7qippo0fe7h5k1b6k4nexk" target="_blank">this
	animated demo</a>.
	</p>

	</div>

	<div class="related-links">
	<div class="familylinks">
	<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_set.html">SET Statement</a></div>
	</div>
	</div></body>
	</html>