| commit | b9c2e00a6ba1eaf1ed3f63013be60408409152b6 | [log] [tgz] |
|---|---|---|
| author | stiga-huang <huangquanlong@gmail.com> | Fri Mar 01 13:08:31 2024 +0800 |
| committer | Impala Public Jenkins <impala-public-jenkins@cloudera.com> | Wed Mar 06 16:41:58 2024 +0000 |
| tree | b8eaf0ad3aaf8e8eab5bcbd3a1cec1745450b332 | |
| parent | e666e07110216509cbc2d1d874ba4bf9ec32147e [diff] |
IMPALA-12855: Fix NPE in firing RELOAD events when the partition doesn't exist When --enable_reload_events is set to true, catalogd will fire RELOAD events for INVALIDATE/REFRESH statements. When the RELOAD event is fired successfully for a REFRESH statement, we also update lastRefreshEventId of the table/partition. This part could hit NullPointerException when the partition is dropped by concurrent DDLs. This patch ignores updating lastRefreshEventId if the partition doesn't exists. Note that ideally we should hold the table lock of REFRESH until finish firing the RELOAD events and updating lastRefreshEventId. So no concurrent operations can drop the partition. However, when the table is loaded from scratch, we don't actually hold the table write lock. We just load the table and take a read lock to get the thrift object. The partition could still be dropped concurrently after the load and before taking the read lock. So ignoring missing partitions is a simpler solution. Refactors some codes of fireReloadEventAndUpdateRefreshEventId to save some indention and avoid acquiring table lock if no events are fired. Adds error messages in some Precondition checks in methods used by this feature. Also refactors Table.getFullName() to not always constructing the result. Improves logs of not reloading a partition for an event. Tests: - Add e2e test Change-Id: I01af3624bf7cf5cd69935cffa28d54f6a6807504 Reviewed-on: http://gerrit.cloudera.org:8080/21096 Reviewed-by: Csaba Ringhofer <csringhofer@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>
Lightning-fast, distributed SQL queries for petabytes of data stored in open data and table formats.
Impala is a modern, massively-distributed, massively-parallel, C++ query engine that lets you analyze, transform and combine data from a variety of data sources:
The fastest way to try out Impala is a quickstart Docker container. You can try out running queries and processing data sets in Impala on a single machine without installing dependencies. It can automatically load test data sets into Apache Kudu and Apache Parquet formats and you can start playing around with Apache Impala SQL within minutes.
To learn more about Impala as a user or administrator, or to try Impala, please visit the Impala homepage. Detailed documentation for administrators and users is available at Apache Impala documentation.
If you are interested in contributing to Impala as a developer, or learning more about Impala's internals and architecture, visit the Impala wiki.
Impala only supports Linux at the moment. Impala supports x86_64 and has experimental support for arm64 (as of Impala 4.0). Impala Requirements contains more detailed information on the minimum CPU requirements.
Impala runs on Linux systems only. The supported distros are
Other systems, e.g. SLES12, may also be supported but are not tested by the community.
This distribution uses cryptographic software and may be subject to export controls. Please refer to EXPORT_CONTROL.md for more information.
See Impala's developer documentation to get started.
Detailed build notes has some detailed information on the project layout and build.