PHOENIX 4.16.0 Release Notes

These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.

  • PHOENIX-6273 | Major | Add support to handle MR Snapshot restore externally

Adds mapreduce configuration param “phoenix.mapreduce.external.snapshot.restore” which when set to true indicates that snapshot-based MapReduce jobs shouldn't try to restore the snapshot themselves, but assume an external application has already done so.

  • PHOENIX-6307 | Major | Build and release official binary distributions with each HBase profile

The phoenix-client and phoenix-server JARs have been renamed to include the supported HBase version.

Instead of phoenix-client-<phoenix.version>.jar, the client is now phoenix-client-hbase-<hbase-major.minor>-<phoenix.version>.jar I.e. The Phoenix 4.16 client for Hbase 1.5 is now called phoenix-client-hbase-1.5-5.1.0.jar

The maven coordinates also have also changed to “org.apache.phoenix:phoenix-client-hbase-<hbase-major.minor>:<phoenix.version>”. I.e the Phoenix 4.15.0 client for Hbase 1.5 is “org.apache.phoenix:phoenix-client:4.15.0-HBase-1.5”, but the phoenix client for Phoenix 4.16 is “org.apache.phoenix:phoenix-client-hbase-1.5:4.16.0”

  • PHOENIX-5265 | Major | [UMBRELLA] Phoenix Test should use object based Plan for result comparison instead of using hard-corded comparison

New API for Explain plan queries that can be used for comparison of individual plan attributes.

  • PHOENIX-6282 | Major | Generate PB files inline with build and remove checked in files

We no longer have generated protobuf Java files available in source code. These files are expected to be generated inline with mvn build. We have also used an optimization with the plugin to ensure protoc is not invoked with mvn build if no .proto file is updated between two consecutive builds.

  • PHOENIX-6086 | Critical | Take a snapshot of all SYSTEM tables before attempting to upgrade them

While upgrading System tables, all system tables where we perform some significant DDL operations, we start taking snapshots of them:

  1. SYSTEM.CATALOG (was already covered before this Jira)
  2. SYSTEM.CHILD_LINK
  3. SYSTEM.SEQUENCE
  4. SYSTEM.STATS
  5. SYSTEM.TASK

If the upgrade doesn't complete successfully, we should get warning log providing all snapshots taken so far, which can be used to restore some snapshots if required.

A sample Warning log:

Failed upgrading System tables. Snapshots for system tables created so far: {SYSTEM:STATS=SNAPSHOT_SYSTEM.STATS_4.15.x_TO_4.16.0_20201202114411, SYSTEM:CATALOG=SNAPSHOT_SYSTEM.CATALOG_4.15.x_TO_4.16.0_20201202114258, SYSTEM:CHILD_LINK=SNAPSHOT_SYSTEM.CHILD_LINK_4.15.x_TO_4.16.0_20201202114405, SYSTEM:SEQUENCE=SNAPSHOT_SYSTEM.SEQUENCE_4.15.x_TO_4.16.0_20201202114407, SYSTEM:TASK=SNAPSHOT_SYSTEM.TASK_4.15.x_TO_4.16.0_20201202114413}

  • PHOENIX-4412 | Critical | Tephra transaction context visibility level returns null instead of SNAPSHOT_ALL

WARNING: No release note provided for this change.

  • PHOENIX-6155 | Major | Prevent doing direct upserts into SYSTEM.TASK from the client

A new coprocessor endpoint to avoid direct upserts into SYSTEM.TASK from the client.

  • PHOENIX-6186 | Major | Store table metadata last modified timestamp in PTable / System.Catalog

Introduces a new field in System.Catalog, LAST_DDL_TIMESTAMP, which is the epoch timestamp at which a table or view is created, or last had a column added or dropped. Child views inherit the max ddl timestamp of their ancestors.

We have new split policy introduced for SYSTEM.TASK which for now is just extending DisabledRegionSplitPolicy. As part of an upgrade to 4.16/5.1, updating split policy will be taken care of unless it was already updated manually.

Hence, before 4.16/5.1 upgrade, if operator has already manually updated split policy of SYSTEM.TASK, an exception will be thrown during upgrade to 4.16/5.1 which would mandate an operator intervention to perform:

  1. Merging SYSTEM.TASK regions into one single region (if multiple regions were already available before 4.16/5.1 upgrade)
  2. Remove split policy of the table manually.
  • PHOENIX-5446 | Major | Support Protobuf shaded clients (thin + thick)

WARNING: No release note provided for this change.

  • PHOENIX-4866 | Blocker | UDFs get error: org.apache.phoenix.schema.FunctionNotFoundException: ERROR 6001 (42F01): Function undefined

WARNING: No release note provided for this change.