commit 73171cb7164573349bd53a996a51bb7058b778e0
author Gabor Kaszab <gaborkaszab@cloudera.com> Wed Feb 07 12:33:04 2024 +0100
committer Impala Public Jenkins <impala-public-jenkins@cloudera.com> Thu Mar 28 13:57:07 2024 +0000
tree 2cd2d5ae3b94a1a0e18df0851ff83748603d4f89
parent 3e4fdeece1735de85c17155dda626e8f28af0092

IMPALA-12729: Allow creating primary keys for Iceberg tables

There are writer engines that use Iceberg's identifier-field-ids from
the Iceberg schema to identify the columns to be written into the
equality delete files (Flink, NiFi). So far Impala wasn't able to
populate this identifier-field-ids. This patch introduces the support
for not enforced primary keys for Iceberg tables, where the primary key
is going to be used for setting identifier-field-ids during Iceberg
schema creation.

Example syntax:
CREATE TABLE ice_tbl (
  i int NOT NULL,
  j int,
  s string NOT NULL
  primary key(i, s) not enforced)
PARTITIONED BY SPEC (truncate(10, s))
STORED AS ICEBERG;

There are some constraints with primary keys (PK) following the
behavior of Flink:
 - Only NOT NULL columns can be in the PK.
 - PK is not allowed in the column definition level like
   'i int NOT NULL PRIMARY KEY'.
 - If the table is partitioned then the partition columns have to be
   part of the PK.
 - Float and double columns are not allowed for the PK.
 - Not allowed to drop a column that is used as a PK.

Testing:
 - New E2E tests added for different table creation scenarios.
 - Manual test to use Nifi for writing into a table with PK.

Change-Id: I7bea787acdabd8cb04661f4ddb5c3309af0364a6
Reviewed-on: http://gerrit.cloudera.org:8080/21149
Reviewed-by: Daniel Becker <daniel.becker@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenkins@cloudera.com>