docs/build/plain-html/topics/impala_live_progress.html - impala - Git at Google

 <?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE html
   PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

 <meta name="copyright" content="(C) Copyright 2023" />
 <meta name="DC.rights.owner" content="(C) Copyright 2023" />
 <meta name="DC.Type" content="concept" />
 <meta name="DC.Title" content="LIVE_PROGRESS Query Option (Impala 2.3 or higher only)" />
 <meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
 <meta name="prodname" content="Impala" />
 <meta name="prodname" content="Impala" />
 <meta name="version" content="Impala 3.4.x" />
 <meta name="version" content="Impala 3.4.x" />
 <meta name="DC.Format" content="XHTML" />
 <meta name="DC.Identifier" content="live_progress" />
 <link rel="stylesheet" type="text/css" href="../commonltr.css" />
 <title>LIVE_PROGRESS Query Option (Impala 2.3 or higher only)</title>
 </head>
 <body id="live_progress">


   <h1 class="title topictitle1" id="ariaid-title1">LIVE_PROGRESS Query Option (<span class="keyword">Impala 2.3</span> or higher only)</h1>


   <div class="body conbody">

     <p class="p"> In Impala 3.4.0 and higher when you start the Impala shell in interactive mode, the
         <code class="ph codeph">LIVE_PROGRESS</code> query option is set to <code class="ph codeph">TRUE</code> by default. When
       you run a query, the live progress bar appears in the output of a query. The bar shows roughly
       the percentage of completed processing. When the query finishes, the live progress bar
       disappears from the console output. </p>

     <p class="p">Starting in <span class="keyword">Impala 3.1</span>, the summary output also
       includes the queuing status consisting of whether the query was queued and
       what was the latest queuing reason.</p>

     <p class="p"><strong class="ph b">Type:</strong>
       <code class="ph codeph">Boolean</code></p>

     <p class="p"><strong class="ph b">Default:</strong>
       <code class="ph codeph">TRUE (0)</code></p>


     <p class="p">
         <strong class="ph b">Command-line equivalent:</strong>
       </p>

     <p class="p"> You can turn off <code class="ph codeph">LIVE_PROGRESS</code> within the shell
       using the <code class="ph codeph">SET</code> command. You can also turn off
       LIVE_PROGRESS by starting the shell with the <code class="ph codeph">--disable_live_progress</code>
       command-line option.</p>


     <p class="p">
         <strong class="ph b">Usage notes:</strong>
       </p>

     <p class="p">
         The output from this query option is printed to standard
         error. The output is  displayed only in interactive mode, not when the <code class="ph codeph">-q</code>
         or <code class="ph codeph">-f</code> options are used.
       </p>

     <p class="p">
       For tracking the progress of an interactive query through
       all phases of processing, see <a class="xref" href="impala_live_summary.html#live_summary">LIVE_SUMMARY Query Option (Impala 2.3 or higher only)</a>.
     </p>


     <p class="p">
         <strong class="ph b">Restrictions:</strong>
       </p>

     <p class="p">
       Because the percentage complete figure is calculated using the number of
       issued and completed <span class="q">"scan ranges"</span>, which occur while reading the table
       data, the progress bar might reach 100% before the query is entirely finished.
       For example, the query might do work to perform aggregations after all the
       table data has been read. If many of your queries fall into this category,
       consider using the <code class="ph codeph">LIVE_SUMMARY</code> option for more granular progress
       reporting.
     </p>

     <p class="p">
         The <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query options
         currently do not produce any output during <code class="ph codeph">COMPUTE STATS</code> operations.
       </p>

     <div class="p">
         Because the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
         options are available only within the <span class="keyword cmdname">impala-shell</span> interpreter:
         <ul class="ul">
           <li class="li">
             <p class="p">
               You cannot change these query options through the SQL <code class="ph codeph">SET</code>
               statement using the JDBC or ODBC interfaces. The <code class="ph codeph">SET</code> command in
               <span class="keyword cmdname">impala-shell</span> recognizes these names as shell-only options.
             </p>

           </li>


           <li class="li">
             <p class="p">
               Be careful when using <span class="keyword cmdname">impala-shell</span> on a
               pre-<span class="keyword">Impala 2.3</span> system to connect to a system running
               <span class="keyword">Impala 2.3</span> or higher. The older <span class="keyword cmdname">impala-shell</span>
               does not recognize these query option names. Upgrade
               <span class="keyword cmdname">impala-shell</span> on the systems where you intend to use these query
               options.
             </p>

           </li>


           <li class="li">
             <p class="p">
               Likewise, the <span class="keyword cmdname">impala-shell</span> command relies on some information
               only available in <span class="keyword">Impala 2.3</span> and higher to prepare live
               progress reports and query summaries. The <code class="ph codeph">LIVE_PROGRESS</code> and
               <code class="ph codeph">LIVE_SUMMARY</code> query options have no effect when
               <span class="keyword cmdname">impala-shell</span> connects to a cluster running an older version of
               Impala.
             </p>

           </li>

         </ul>

       </div>


     <p class="p">
         <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.3.0</span>
       </p>


     <p class="p">
         <strong class="ph b">Examples:</strong>
       </p>

 <pre class="pre codeblock"><code>[localhost:21000] &gt; select count(*) from customer;
 +----------+
 | count(*) |
 +----------+
 | 150000   |
 +----------+
 [localhost:21000] &gt; select count(*) from customer t1 cross join customer t2;
 [###################################                                   ] 50%
 [######################################################################] 100%


 </code></pre>

     <p class="p">To see how the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
         options work in real time, see
         <a class="xref" href="https://asciinema.org/a/1rv7qippo0fe7h5k1b6k4nexk" target="_blank">this
           animated demo</a>.
       </p>


   </div>

 <div class="related-links">
 <div class="familylinks">
 <div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
 </div>
 </div></body>
 </html>
	<?xml version="1.0" encoding="UTF-8"?>
	<!DOCTYPE html
	PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

	<meta name="copyright" content="(C) Copyright 2023" />
	<meta name="DC.rights.owner" content="(C) Copyright 2023" />
	<meta name="DC.Type" content="concept" />
	<meta name="DC.Title" content="LIVE_PROGRESS Query Option (Impala 2.3 or higher only)" />
	<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
	<meta name="prodname" content="Impala" />
	<meta name="prodname" content="Impala" />
	<meta name="version" content="Impala 3.4.x" />
	<meta name="version" content="Impala 3.4.x" />
	<meta name="DC.Format" content="XHTML" />
	<meta name="DC.Identifier" content="live_progress" />
	<link rel="stylesheet" type="text/css" href="../commonltr.css" />
	<title>LIVE_PROGRESS Query Option (Impala 2.3 or higher only)</title>
	</head>
	<body id="live_progress">


	<h1 class="title topictitle1" id="ariaid-title1">LIVE_PROGRESS Query Option (<span class="keyword">Impala 2.3</span> or higher only)</h1>




	<div class="body conbody">

	<p class="p"> In Impala 3.4.0 and higher when you start the Impala shell in interactive mode, the
	<code class="ph codeph">LIVE_PROGRESS</code> query option is set to <code class="ph codeph">TRUE</code> by default. When
	you run a query, the live progress bar appears in the output of a query. The bar shows roughly
	the percentage of completed processing. When the query finishes, the live progress bar
	disappears from the console output. </p>

	<p class="p">Starting in <span class="keyword">Impala 3.1</span>, the summary output also
	includes the queuing status consisting of whether the query was queued and
	what was the latest queuing reason.</p>

	<p class="p"><strong class="ph b">Type:</strong>
	<code class="ph codeph">Boolean</code></p>

	<p class="p"><strong class="ph b">Default:</strong>
	<code class="ph codeph">TRUE (0)</code></p>


	<p class="p">
	<strong class="ph b">Command-line equivalent:</strong>
	</p>

	<p class="p"> You can turn off <code class="ph codeph">LIVE_PROGRESS</code> within the shell
	using the <code class="ph codeph">SET</code> command. You can also turn off
	LIVE_PROGRESS by starting the shell with the <code class="ph codeph">--disable_live_progress</code>
	command-line option.</p>


	<p class="p">
	<strong class="ph b">Usage notes:</strong>
	</p>

	<p class="p">
	The output from this query option is printed to standard
	error. The output is displayed only in interactive mode, not when the <code class="ph codeph">-q</code>
	or <code class="ph codeph">-f</code> options are used.
	</p>

	<p class="p">
	For tracking the progress of an interactive query through
	all phases of processing, see <a class="xref" href="impala_live_summary.html#live_summary">LIVE_SUMMARY Query Option (Impala 2.3 or higher only)</a>.
	</p>


	<p class="p">
	<strong class="ph b">Restrictions:</strong>
	</p>

	<p class="p">
	Because the percentage complete figure is calculated using the number of
	issued and completed <span class="q">"scan ranges"</span>, which occur while reading the table
	data, the progress bar might reach 100% before the query is entirely finished.
	For example, the query might do work to perform aggregations after all the
	table data has been read. If many of your queries fall into this category,
	consider using the <code class="ph codeph">LIVE_SUMMARY</code> option for more granular progress
	reporting.
	</p>

	<p class="p">
	The <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query options
	currently do not produce any output during <code class="ph codeph">COMPUTE STATS</code> operations.
	</p>

	<div class="p">
	Because the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
	options are available only within the <span class="keyword cmdname">impala-shell</span> interpreter:
	<ul class="ul">
	<li class="li">
	<p class="p">
	You cannot change these query options through the SQL <code class="ph codeph">SET</code>
	statement using the JDBC or ODBC interfaces. The <code class="ph codeph">SET</code> command in
	<span class="keyword cmdname">impala-shell</span> recognizes these names as shell-only options.
	</p>

	</li>


	<li class="li">
	<p class="p">
	Be careful when using <span class="keyword cmdname">impala-shell</span> on a
	pre-<span class="keyword">Impala 2.3</span> system to connect to a system running
	<span class="keyword">Impala 2.3</span> or higher. The older <span class="keyword cmdname">impala-shell</span>
	does not recognize these query option names. Upgrade
	<span class="keyword cmdname">impala-shell</span> on the systems where you intend to use these query
	options.
	</p>

	</li>


	<li class="li">
	<p class="p">
	Likewise, the <span class="keyword cmdname">impala-shell</span> command relies on some information
	only available in <span class="keyword">Impala 2.3</span> and higher to prepare live
	progress reports and query summaries. The <code class="ph codeph">LIVE_PROGRESS</code> and
	<code class="ph codeph">LIVE_SUMMARY</code> query options have no effect when
	<span class="keyword cmdname">impala-shell</span> connects to a cluster running an older version of
	Impala.
	</p>

	</li>

	</ul>

	</div>


	<p class="p">
	<strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.3.0</span>
	</p>


	<p class="p">
	<strong class="ph b">Examples:</strong>
	</p>

	<pre class="pre codeblock"><code>[localhost:21000] > select count(*) from customer;
	+----------+
	\| count(*) \|
	+----------+
	\| 150000 \|
	+----------+
	[localhost:21000] > select count(*) from customer t1 cross join customer t2;
	[################################### ] 50%
	[######################################################################] 100%


	</code></pre>

	<p class="p">To see how the <code class="ph codeph">LIVE_PROGRESS</code> and <code class="ph codeph">LIVE_SUMMARY</code> query
	options work in real time, see
	<a class="xref" href="https://asciinema.org/a/1rv7qippo0fe7h5k1b6k4nexk" target="_blank">this
	animated demo</a>.
	</p>


	</div>

	<div class="related-links">
	<div class="familylinks">
	<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
	</div>
	</div></body>
	</html>