commit | 61dcc805e536af0f160225cc928aa188aa861225 | [log] [tgz] |
---|---|---|
author | stiga-huang <huangquanlong@gmail.com> | Tue Aug 11 16:35:15 2020 +0800 |
committer | Impala Public Jenkins <impala-public-jenkins@cloudera.com> | Thu Aug 27 04:09:58 2020 +0000 |
tree | df9e3fe11df578ba90d0ec85c636e4dc8587115c | |
parent | 0fcf846592072e35a1d693ee432b8a0216e229ee [diff] |
IMPALA-9225: Query option for retryable queries to spool all results before returning any to the client If we have returned any results to the client in the original query, query retry will be skipped to avoid incorrect results. This patch adds a query option, spool_all_results_for_retries, for retryable queries to spool all results before returning any to the client. It defaults to true. If all query results cannot be contained in the allocated result spooling space, we'll return results and thus disabled query retry on the query. Setting spool_all_results_for_retries to false will fallback to the original behavior - client can fetch results when any of them are ready. So we explicitly set it to false in the retried query since it won't be retried. For non retryable queries or queries that don't enable results spooling, the spool_all_results_for_retries option takes no effect. To implement this, this patch defers the time when results are ready to be fetched. By default, the “rows available” event happens when any results are ready. For a retryable query, when spool_query_results and spool_all_results_for_retries are both true, the “rows available” event happens after all results are spooled or any errors stopping us to do so, e.g. batch queue is full, cancellation or failures. After waiting for the root fragment instance’s Open() finishes, the coordinator will wait until results of BufferedPlanRootSink are ready. BufferedPlanRootSink sets the results ready signal in its Send(), Close(), Cancel(), FlushFinal() methods. Tests: - Add a test to verify that a retryable query will spool all its results when results spooling and spool_all_results_for_retries are enabled. - Add a test to verify that query retry succeeds when a retryable query is still spooling its results (spool_all_results_for_retries=true). - Add a test to verify that the retried query won't spool all results even when results spooling and spool_all_results_for_retries are enabled in the original query. - Add a test to verify that the original query can be canceled correctly. We need this because the added logics for spool_all_results_for_retries are related to the cancellation code path. - Add a test to verify results will be returned when all of them can't fit into the result spooling space, and query retry will be skipped. Change-Id: I462dbfef9ddab9060b30a6937fca9122484a24a5 Reviewed-on: http://gerrit.cloudera.org:8080/16323 Reviewed-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Lightning-fast, distributed SQL queries for petabytes of data stored in Apache Hadoop clusters.
Impala is a modern, massively-distributed, massively-parallel, C++ query engine that lets you analyze, transform and combine data from a variety of data sources:
To learn more about Impala as a business user, or to try Impala live or in a VM, please visit the Impala homepage. Detailed documentation for administrators and users is available at Apache Impala documentation.
If you are interested in contributing to Impala as a developer, or learning more about Impala's internals and architecture, visit the Impala wiki.
Impala only supports Linux at the moment.
This distribution uses cryptographic software and may be subject to export controls. Please refer to EXPORT_CONTROL.md for more information.
See Impala's developer documentation to get started.
Detailed build notes has some detailed information on the project layout and build.