tree 35e5bddfd86f1944bd2dfe2d89bd54136bea90e7
parent ec4251dba99fcf3a216c42a23686d74592d75a95
author Enrico Minack <github@enrico.minack.dev> 1709796062 +0100
committer GitHub <noreply@github.com> 1709796062 +0800
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsFcBAABCAAQBQJl6WreCRC1aQ7uu5UhlAAA7PsQACrXUT+GlGGai4Dg1dRARh04
 Wi7JbGC1WxV+jGrxBdymnBHfBvbd+fLsDpng/soj+F3h71RDdMg/iVdIrQpCTR0O
 UZXjMT8wSCoLiIkGJlYVKtMjVdza4OVgJvprO3Jf/SkmH+HppW90Xm1VyvWQsdT2
 fWQ27zJGBbJzeDCRAjuVROu3fOkcfvsOT3dmng76EniIZINToYUNxYsxdGa1LciY
 l25mGr8CY116cHusee3XoVH5WR3SWeGIa8+kMEZ1TA350duJREE3IZgkdjNiLHQF
 yKsEJVRDuHSyQSC2v4qmJ5GBQP0kGwjJhUP8mKJfgkTA0uAeokU8sAVyez9z3rpO
 4U18841lZ/zbq6PRgFcV8knf+8huD6Z0trPtCe2lhim2QVDrrwl2oAoizBZhegYQ
 VdjaxfPBDeARUHCGLK8Ve0j21qaMIE8W8QhMiO7OfWr1uMJ5QcQ98Ibzmf9/3ja/
 AJ4PkPSlWNeYkAX/tc1/ChJvcGTx5qn1hCdtuqpRDf+kDBeTu+3s3/vIkDsE7eYc
 /s7i57gEM0tSFTL66wprsz6cjMuUUeqb8rYqMmuYnkvVpx4B7nWSZ1TYqGBBacis
 sDJ0mdCsktSHLKWFKDvL0ARprrxphKmaaWHEcUY3Gkd4D5+vHwCIeN852IAW40ny
 h3Wvq/YgDyk7FySG9SSl
 =e9A/
 -----END PGP SIGNATURE-----
 

[#731] feat(spark): Make blockid layout configurable for Spark clients (#1528)

### What changes were proposed in this pull request?
Make bit-lengths in block id (block id layout) configurable through dynamic client config from coordinator or client config. Block id layout can be created from RssConf, or where that is not available, is being passed around.
 
- Adds new options (defaults are equivalent to current values in `Constants`):
  - rss.client.blockId.sequenceNoBits
  - rss.client.blockId.partitionIdBits
  - rss.client.blockId.taskAttemptIdBits
- Adds block id layout to two requests (default is layout with current values in `Constants`).

Default values have moved from `Constants` into `BlockIdLayout`. The following replacements exist:
- `PARTITION_ID_MAX_LENGTH`: `BlockIdLayout.DEFAULT.partitionIdBits`
- `TASK_ATTEMPT_ID_MAX_LENGTH`: `BlockIdLayout.DEFAULT.taskAttemptIdBits`
- `ATOMIC_INT_MAX_LENGTH`: `BlockIdLayout.DEFAULT.sequenceNoBits`

### Why are the changes needed?
The bit-lengths of sequence number, partition id and task attempt id in block id are defined in `common/src/main/java/org/apache/uniffle/common/util/Constants.java`. Changing these requires recompiling and redeploying the project. Making this configurable in `coordinator.conf`, `server.conf` or client-side would very useful.

Also see #1512, #749.

Fixes #731.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Tests.