)]}' { "log": [ { "commit": "217e66dbdf2e206b843e6e68acd85fb4feeeea7b", "tree": "c0cd935202c8448e7b781a31ec9fa26bbdea735b", "parents": [ "7c39c3748d2351cdb8b2b5432937c96766fc07bf" ], "author": { "name": "Spicy Shark", "email": "321699413+spicysharkco@users.noreply.github.com", "time": "Sat Oct 24 08:52:39 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sat Oct 24 08:52:39 2026 +0000" }, "message": "fix(spec): read float JSON values like iceberg-java (#3476)\n\n* fix(spec): read float JSON values like iceberg-java\n\nThe Float branch of RawLiteral::try_into rejected a double whose\ndistance to the narrowed f32 exceeded the absolute f32::EPSILON, so\nvalid values such as 100.1 and 123.456 failed. iceberg-java narrows\nwith floatValue() and does no precision check; do the same.\n\n* test(spec): pin non-finite and negative zero double to float\n\nCheck NaN, both infinities, and the sign of zero when narrowing Avro doubles." }, { "commit": "7c39c3748d2351cdb8b2b5432937c96766fc07bf", "tree": "a3728d1ab3ec678ac5fc7a6147e1def83b60c089", "parents": [ "e859e0b7c6f9b614084010f9a210485547e3afd5" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Sat Oct 24 04:26:18 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sat Oct 24 04:26:18 2026 +0000" }, "message": "chore(deps): Bump certifi from 2026.4.22 to 2026.10.19 in /bindings/python (#3498)\n\nchore(deps): Bump certifi in /bindings/python\n\nBumps [certifi](https://github.com/certifi/python-certifi) from 2026.4.22 to 2026.10.19.\n- [Commits](https://github.com/certifi/python-certifi/compare/2026.04.22...2026.10.19)\n\n---\nupdated-dependencies:\n- dependency-name: certifi\n dependency-version: 2026.10.19\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "e859e0b7c6f9b614084010f9a210485547e3afd5", "tree": "3b97bb7bea54ca32a98699b115668f3a397fba93", "parents": [ "d2679a029f50a25f00850731650d7fbab1b03ca8" ], "author": { "name": "linfeng", "email": "33561138+lyne7-sc@users.noreply.github.com", "time": "Sat Oct 24 04:19:29 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sat Oct 24 04:19:29 2026 +0000" }, "message": "chore(deps): bump ctutils to 0.4.5 to fix RUSTSEC-2026-0385 (#3519)" }, { "commit": "d2679a029f50a25f00850731650d7fbab1b03ca8", "tree": "a35b10f2e265585ffac2c3e9a1acab9908f42760", "parents": [ "aa9986fe83d4bdb0d883968b9a105c6e0a9ef745" ], "author": { "name": "JeonDaehong", "email": "90895144+JeonDaehong@users.noreply.github.com", "time": "Fri Oct 23 19:47:48 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 23 19:47:48 2026 +0000" }, "message": "perf(arrow): build the delete row selection from deleted ranges (#3454)\n\n* perf(arrow): build the delete row selection from deleted ranges\n\nbuild_deletes_row_selection walked the delete vector one position at a time, so\na run of deleted rows cost one iteration per row. It now reads the deleted\npositions as ranges, which makes a run one step. Short runs are still walked\nwith `next`, because roaring\u0027s `next_range` binary-searches the rest of an array\ncontainer on every call and is slower than `next` for single positions.\n\nRow groups that are not selected are skipped with `skip_to` instead of walking\ntheir deletes. The new range iterator is crate-private; the public\n`DeleteVector::iter` and `DeleteVectorIterator::advance_to` are unchanged.\n\n* Address review: fix advance_to (#3400), randomized reader test\n\n- DeleteVectorIterator::advance_to now handles a target in a missing later\n bitmap, a target below the current bitmap, and a call before the first next\n (the cases in #3400), with the same three-way check as skip_to. Dropped the\n stale comments about roaring-rs#314 and the git dependency.\n- Randomized test for build_deletes_row_selection: random row group layouts\n and selections over mixed_delete_vectors, against a selection built one\n deleted position at a time.\n- full_range no longer overflows at u64::MAX.\n- One Buffered enum instead of pending/pending_range.\n- Model tests also skip backwards, mixed_delete_vectors has gaps between\n bitmaps, a model test for advance_to, the #3400 cases split per test, and\n doc updates for ranges, skip_to and RUN_PROBE_LEN.\n\n* Address review: deletes past the file end, runs across 2^16, ranges() docs\n\n* Use a seeded StdRng in the model tests and cover empty row groups" }, { "commit": "aa9986fe83d4bdb0d883968b9a105c6e0a9ef745", "tree": "ee768dec9484fb408b1cf4698d6aafcbfa2b4a0c", "parents": [ "257edd593f52cdc85711e4bf23b375f0acf6f423" ], "author": { "name": "ump45nose", "email": "52391318+ump45nose@users.noreply.github.com", "time": "Fri Oct 23 17:17:41 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 23 17:17:41 2026 +0000" }, "message": "fix(expr): treat NaN row group bounds as might-match for inequalities (#3502)\n\n* fix(expr): treat NaN row group bounds as might-match for inequalities\n\n* test(expr): cover sign-bit NaN row group bounds\n\n* test(expr): pin NaN min with a finite max for less-than\n\n---------\n\nCo-authored-by: ump45nose \u003cump45nose@users.noreply.github.com\u003e" }, { "commit": "257edd593f52cdc85711e4bf23b375f0acf6f423", "tree": "a7b2eb111b3791729daefcedd13c8a64ba599c0a", "parents": [ "ca21f1587c6caaeb88dd3a9e1cc245747e1aba08" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Fri Oct 23 00:27:52 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 23 00:27:52 2026 +0000" }, "message": "ci: build and merge up to three merge queue entries at once (#3490)\n\nBuilding one entry at a time serializes the queue, so each PR waits for\nevery PR ahead of it to finish CI. Building three in parallel lets\nindependent entries validate concurrently, and ALLGREEN still requires\nevery entry to pass before it merges. Merging up to three passing entries\ntogether also collapses their pushes to main into one CI run.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "ca21f1587c6caaeb88dd3a9e1cc245747e1aba08", "tree": "1bf85054ebec8139294d2ac1c52ba9ffa9cd8e7f", "parents": [ "1c05452b53f65d4d37f6399c550ca0d2413ded0c" ], "author": { "name": "Michael Taranov", "email": "mixermt@gmail.com", "time": "Thu Oct 22 15:07:39 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 22 15:07:39 2026 +0000" }, "message": "feat(storage): Add HDFS support via opendal services-hdfs-native (#3111)\n\n* feat(storage): Add HDFS support via opendal services-hdfs-native\n\nAdds an opendal-hdfs-native cargo feature plus OpenDalStorageFactory::Hdfs\nand OpenDalStorage::Hdfs variants in iceberg-storage-opendal, using\nOpenDAL\u0027s services-hdfs-native (pure-Rust HDFS RPC, no JNI/libhdfs).\n\nThe NameNode for a path resolves as: the hdfs.name-node property when set\n(comma-separated endpoints enable HA failover), otherwise the path\nauthority. hadoop.-prefixed properties are forwarded to the HDFS client\nconfiguration, overriding values loaded from $HADOOP_CONF_DIR. Operators\nare cached per effective NameNode since each holds live RPC connections.\n\nRevives and updates PR #2441 (by @jordepic) against the current storage\nlayer and opendal 0.58, where name_node became mandatory and the comma\nlist is the HA mechanism.\n\nCloses #2440\n\nCo-authored-by: Jordan Epstein \u003cjordepic@users.noreply.github.com\u003e\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n* refactor: address review — Properties derive for HdfsConfig, drop HDFS CI special-casing\n\n- HdfsConfig is now pub(crate) and parsed via #[derive(Properties)]\n (key/prefix attributes) instead of hand-written TryFrom + TypedBuilder.\n- HDFS integration tests are no longer #[ignore]d and the dedicated CI\n step is gone; they run under the default nextest invocation since\n make docker-up already starts the fixture and the Tests job is\n Linux-only.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n* fix: reject non-hierarchical hdfs URLs; key delete_stream batches by effective NameNode\n\n- Url::parse accepts non-hierarchical forms like `hdfs:x`; the byte-7\n slice then panicked. Require the literal `hdfs://` prefix instead.\n- batch_key_for_path grouped by URL host only, so NameNodes differing\n by port shared one deleter; key by the effective NameNode (configured\n hdfs.name-node, else authority incl. port), matching the operator cache.\n\nCo-Authored-By: Claude Opus 5 \u003cnoreply@anthropic.com\u003e\n\n* refactor(storage): rename hdfs module to hdfs_native, drop unused HdfsConfig\n\nPer review: the opendal module, its functions and the storage/factory\nvariants are now hdfs_native-prefixed (leaving room for a libhdfs-backed\nvariant, see #1130), and the never-consumed HdfsConfig struct is removed\nfrom the core crate — config/hdfs.rs keeps only the property constants\nthat iceberg-storage-opendal uses.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* ci: retrigger after setup-python manifest timeout on macos runner\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): address review — cache newtype, build outside lock, single NameNode rule, env-gated HDFS tests\n\n- HdfsNativeOperatorCache newtype keeps the cache representation out of\n the public API.\n- Operators are built outside the cache lock (the build reads Hadoop XML\n synchronously); a racing first caller\u0027s duplicate is dropped unopened.\n- hdfs_native_effective_name_node is the single source of the\n configured-else-authority rule for both create_operator and the\n delete_stream batch key; unresolvable paths key on themselves.\n- hdfs.name-node is trimmed and an empty value is treated as unset, so it\n no longer shadows the path-authority fallback.\n- HDFS integration tests self-skip unless ICEBERG_TEST_HDFS_ENDPOINT is\n set (as the HF tests do) and the compose services sit behind the hdfs\n profile; the Linux CI job opts in via env.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* chore(deps): bump rustls to 0.23.45 (RUSTSEC-2026-0285)\n\nInherited from main (tracked in #3222); same bump as #3165.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): pin IPv6 authority handling in HDFS path parsing\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): resolve the HDFS NameNode in relativize_path too\n\nAn authority-less path now errors in relativize_path exactly as in create_operator unless hdfs.name-node is set, so the two cannot disagree.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): state the cost of the synchronous HDFS operator build\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): describe the HDFS operator cache lifecycle\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): normalize each entry of the hdfs.name-node list\n\nA space after a comma was passed through to opendal unchanged, which silently broke failover to that NameNode.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* feat(storage): accept PyIceberg\u0027s hdfs.host/hdfs.port for authority-less HDFS paths\n\nThey map to Hadoop\u0027s fs.defaultFS and apply only when the path has no authority, as in PyIceberg; hdfs.name-node stays the explicit override.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): require a port in every hdfs.name-node entry\n\nAn entry without a port can only fail at the first I/O as \u0027invalid socket address\u0027; reject it at build time with the property and entry named. Bare host:port stays accepted, as in Hadoop and opendal.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): state the runtime binding of cached HDFS operators\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): cover two-NameNode delete_stream batches and first-insert-wins caching\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* test(utils): name the HDFS port constant like its siblings\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): rebuild cached HDFS operators once their runtime is gone\n\nEach cache entry parks a sentinel task on the runtime that built the operator and keeps a Weak to a token the task owns. When that runtime shuts down the token dies, the entry reads as stale and the operator is rebuilt on the current runtime instead of hdfs-native panicking on the dead one. Sentinels are aborted when entries drop.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): resolve HDFS NameNodes like Hadoop does\n\nAn authority with a port is used as is; hdfs.name-node serves logical nameservice authorities and authority-less paths, which otherwise fall back to fs.defaultFS. Portless strings from any source are logical names that only hdfs.name-node resolves, reported at resolution instead of at the first I/O.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* refactor(storage): keep the HDFS operator cache out of the public API\n\nOpenDalStorage::HdfsNative now wraps one opaque HdfsNativeStorage with crate-private fields, built only by the factories. The serialized form keeps its nesting; the io-timeout propagation test now covers the HDFS factory too.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* perf(storage): build HDFS operators on a blocking thread\n\nThe build reads the Hadoop XML config synchronously; create_operator is now async and offloads it with spawn_blocking, outside the cache lock as before. The builder itself stays sync and runtime-free.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* ci: run the HDFS integration tests in their own job\n\nThe default suite no longer pulls the Hadoop image or waits on the HDFS health checks; a dedicated required job starts only the two HDFS containers and runs the HDFS test binary.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): stop dropping HDFS properties silently\n\nPyIceberg\u0027s hdfs.user and hdfs.kerberos_ticket log a warning instead of vanishing (opendal\u0027s config has no user field), hdfs.port is validated whether or not a host is set and warns when it has no host to apply to, and a bare hadoop. key is rejected.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): derive the second NameNode spelling from the endpoint host\n\nResolving the host to its IPv4 address gives any hostname endpoint a second batch key; an IP-literal endpoint skips the test explicitly instead of passing as a no-op.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* chore: record tracing as a dependency of iceberg-storage-opendal in Cargo.lock\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): report a missing tokio runtime instead of panicking in HDFS create_operator\n\nThe runtime handle is resolved up front and passed into the cache, so the sentinel is no longer optional and the unreachable no-runtime branch is gone.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): normalize every HDFS NameNode spelling to hdfs://host:port\n\nOne parser classifies configured entries, path authorities and fs.defaultFS: other schemes, logical names, port 0, userinfo, paths and unbracketed IPv6 are rejected, and bare and prefixed spellings share one cache key.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* feat(storage): declare HDFS nameservices with hdfs.name-node.\u003cnameservice\u003e\n\nThe key expands to Hadoop\u0027s dfs.ha.namenodes and dfs.namenode.rpc-address keys in the forwarded options and the resolver reads them back, so nameservices declared through hadoop.* work the same way. Plain hdfs.name-node stays the documented single-cluster default; an undeclared name without it is a pointed error.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): require a declaration for every logical HDFS nameservice\n\nPlain hdfs.name-node serves authority-less paths only; an undeclared logical authority is an error naming the key to set instead of a guess that could reach the wrong cluster.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): reject userinfo and port 0 in HDFS path authorities\n\nUserinfo was silently dropped and port 0 was misread as a logical nameservice; both are DataInvalid now, and the userinfo message names the host only so a password never reaches a log.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): validate the NameNode composed from hdfs.host and hdfs.port\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): use an authority-less path in the cross-runtime HDFS test\n\nA logical authority now requires a declaration, so the test must take the plain-key route.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): tighten the HDFS tests after review\n\nDrop three redundant unit tests, assert the warnings and the password redaction directly, make the serde round trip populate the cache first, check which error each negative case produces, cover the plain key and hdfs.host serving authority-less paths against the fixture, and name the declared-nameservice test for what it does.\n\nCo-Authored-By: Claude Fable 5.1 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): mask userinfo in HDFS errors that echo a path or NameNode\n\nA path that failed to parse, or had another scheme, was echoed whole, so\n`hdfs://user:pw@nn:bad/x` put the password in the error. The NameNode\nconfig errors (`hdfs.name-node`, `hdfs.name-node.\u003cnameservice\u003e`,\n`hdfs.host`/`hdfs.port`, `fs.defaultFS`) echoed their values the same way.\n\nEvery `Invalid hdfs path` error now goes through one helper, and the config\nerrors use the same masking: everything between the scheme and the last\n`@` becomes `***`, so a password holding a raw `/`, `?`, `#` or `://` is\ncovered too.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): recover a poisoned HDFS operator cache lock\n\nA panic elsewhere while the cache lock was held turned every later HDFS\ncall into a permanent `Unexpected` error. Writes only ever replace whole\nentries, so the map stays consistent: both lock sites now recover the\nguard with `PoisonError::into_inner`, and `get`/`insert` are infallible.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): flush the fake NameNode before sampling dials\n\nThe runtime test sampled its dial counter before the listener thread had\nnecessarily counted every dial from the first runtime. It now drops that\nruntime and waits for the listener to close a marker connection; accepts\nare served in order, so every earlier dial is counted first. A failed\naccept no longer stops the listener.\n\nThe runtime-shutdown unit test also asserts that the rebuilt operator is\na new instance, not the stale one with a fresh sentinel.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): state where portless HDFS authorities differ from Hadoop\n\nHadoop resolves a portless authority from hdfs-site.xml, else dials port\n8020. Here it must be declared with `hdfs.name-node.\u003cnameservice\u003e`,\nbecause opendal builds the client as `hdfs://nameservice` and never hands\nhdfs-native the authority. The `HDFS_NAME_NODE` and resolver docs no\nlonger claim Hadoop parity, the error says to add a port or declare the\nnameservice, and the README notes the limitation in its own note.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* refactor(storage): rename the HDFS config module to hdfs_native\n\nMatches its siblings, which are named after their storage feature\n(azdls, gcs, hf, oss, s3). The module is private and re-exported, so the\npublic iceberg::io::HDFS_* paths are unchanged.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): move the HDFS tests out of lib.rs into hdfs_native\n\nThe serde round-trip and relativize_path tests now live next to the code\nthey cover, with the module\u0027s test_hdfs_native_ prefix.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* refactor(storage): read HDFS properties through #[derive(Properties)]\n\nhdfs.name-node, the hdfs.name-node.\u003cnameservice\u003e and hadoop. prefixes,\nhdfs.host and hdfs.port are declared in one struct parsed by the\nProperties macro instead of being looked up by hand. Host and port stay\ntrimmed strings so a blank value still counts as unset, as in PyIceberg.\nValidation, warnings and error messages are unchanged.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* ci: run the HDFS tests in the default suite like the other fixtures\n\nThe HDFS fixture joins the shared bridge network with published ports\ninstead of using host networking behind a compose profile, so make\ndocker-up starts it on every platform: the DataNode advertises 127.0.0.1\nand the tests set dfs.client.use.datanode.hostname. The tests no longer\nskip without ICEBERG_TEST_HDFS_ENDPOINT, and the dedicated Tests (hdfs)\njob is gone.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): build HDFS clients in a crate-owned runtime\n\nAn hdfs-native client spawns its NameNode connections on the runtime\ncurrent when it is built and panics once that runtime is dropped. The\ncache used to detect that with a sentinel task per entry, which needed\nHandle::try_current, and the build ran under spawn_blocking, which made\ncreate_operator async for every backend. Clients are now built inside a\ncrate-owned runtime started on first use, so a cached operator works\nfrom any caller\u0027s runtime: the sentinel, Handle::try_current and\nspawn_blocking are gone, and create_operator is sync again as on main.\nThe Hadoop XML read (~0.1 ms) runs inline.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* refactor(storage): type HDFS path authorities and validate declarations at parse\n\nhdfs_native_parse_path returns an HdfsNativeAuthority (NameNode,\nNameservice, Default), so the resolver matches on it instead of\nre-parsing the authority to learn whether it has a port. Every HA\nnameservice declaration, from hdfs.name-node.\u003cnameservice\u003e or passed\nthrough as hadoop.dfs.ha.namenodes.*, and the fs.defaultFS that serves\nauthority-less paths are now validated when the config is parsed, so a\nbroken one fails the first call rather than only the paths that use it.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* refactor(storage): drop index slicing and combinator chains in the HDFS backend\n\nThe nameservice lookup is a for loop with let-else, and the userinfo\nmasking and the relative-path extraction use split_once/rsplit_once\ninstead of slicing at computed indices. A test pins that doubled slashes\nafter the authority are still trimmed.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): attach the hdfs.port parse error as the source\n\nThe parse error was interpolated into the message; it is now attached\nas the error\u0027s source, which iceberg prints after the message.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* fix(storage): share one HDFS client across host name spellings\n\nhost_str keeps the host\u0027s case for hdfs:// URLs, so hdfs://NN1:8020 and\nhdfs://nn1:8020 cached two clients for one NameNode. Endpoint hosts are\nnow lowercased by one helper used by the config normalizer and the path\nparser, as Hadoop\u0027s FileSystem cache key does; nameservice names stay\ncase-sensitive like the Hadoop keys that declare them. The cache\u0027s doc\nstates that it is unbounded by design.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): note the upstream gap behind the HDFS client runtime\n\nopendal\u0027s hdfs-native service does not expose hdfs-native\u0027s\nClientBuilder::with_io_runtime (Kimahriman/hdfs-native#255); a TODO at\nthe crate-owned runtime records that a runtime could be passed in once\nit does.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* test(storage): make the HDFS HA test fail over for real\n\nThe declared-nameservice test listed the single-node fixture twice,\nwhich drove opendal\u0027s list path but never needed a failover. Its first\nNameNode is now a local port nothing listens on, so every call has to\nfail over through opendal\u0027s synthetic nameservice; an opendal change\nthat kept only the first entry of the list would fail it.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* chore: retrigger CI\n\nDocker Hub rate-limited the previous run\u0027s image pulls, so the default test job never started its fixtures.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* chore: retrigger CI\n\nThe previous run hit Docker Hub\u0027s unauthenticated pull rate limit again before any test ran.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): document how to enable and configure the HDFS backend\n\nThe iceberg-storage-opendal README gains an HDFS section: enabling the\nopendal-hdfs-native feature, the factory, the hdfs.* and hadoop.*\nproperties, the Hadoop config, user and Kerberos it reads, and the\nportless-authority limitation. The experimental-backends note now points\nto it.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n* docs(storage): drop the HDFS README section and link follow-up issues\n\nConfiguration docs for every backend are tracked in #3510; the README\nkeeps the one-line note on the portless-authority limitation. The TODO on\nthe crate-owned HDFS runtime now points at #3509.\n\nCo-Authored-By: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Jordan Epstein \u003cjordepic@users.noreply.github.com\u003e\nCo-authored-by: Claude Opus 5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "1c05452b53f65d4d37f6399c550ca0d2413ded0c", "tree": "166a6f96923b86d14859d5033bdf48e36a0666c6", "parents": [ "86e837f90d16074d67218db05ffc2629443dafd5" ], "author": { "name": "Dhruv Arya", "email": "dhruv.arya@databricks.com", "time": "Thu Oct 22 13:49:50 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 22 13:49:50 2026 +0000" }, "message": "fix(spec): accept partition specs produced by spec evolution (#3431)\n\n* fix(spec): accept partition specs produced by spec evolution\n\nPartition spec evolution produces specs that the spec builders rejected:\n\n- A source column can have more than one partition field. Adding hour(ts) to a\n spec that has day(ts) is allowed, and format version 1 tables keep each\n removed field as a `void` placeholder. Binding an unbound spec\n (`UnboundPartitionSpec::bind`, `PartitionSpecBuilder::new_from_unbound`, and\n so `TableMetadataBuilder::add_partition_spec`) and loading the legacy v1\n `partition-spec` field ran the authoring-time redundancy check and rejected\n these specs. Both now skip it, as Java does.\n `PartitionSpecBuilder::add_unbound_field` and\n `UnboundPartitionSpecBuilder::add_partition_field` still reject redundant\n fields.\n- Removing identity(category) from a format version 1 spec leaves a void field\n named `category`. Both the spec builder and the table metadata builder\n allowed only identity fields to share the name of their source column. Void\n fields may now do the same, matching Java\u0027s `alwaysNull`.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* fix(spec): address review feedback on evolved spec binding\n\nShare the identity-or-void name rule through `Transform::may_share_name_with_source`\nand name void in its error. Check names before redundancy in `add_unbound_field`,\ndocument the authoring-vs-bind split, and pin it with tests, including a v2\nday+hour default spec that survives later commits.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n---------\n\nCo-authored-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\nCo-authored-by: Kevin Liu \u003ckevinjqliu@users.noreply.github.com\u003e" }, { "commit": "86e837f90d16074d67218db05ffc2629443dafd5", "tree": "b76522f3203f229765e69bd2b1fa977330acb45d", "parents": [ "5c27157675a7babb611638786195dfe24ad41dab" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Thu Oct 22 09:30:29 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 22 09:30:29 2026 +0000" }, "message": "chore: clean up audit ignores and cite LEGAL-732 in deny.toml (#3486)\n\n* chore: remove stale quick-xml advisory ignores\n\nCargo.lock now resolves quick-xml to 0.41.0, which contains the fix for\nRUSTSEC-2026-0194 and RUSTSEC-2026-0195. `cargo audit` passes without\nthe ignores, so they no longer suppress anything.\n\nThe rsa (RUSTSEC-2023-0071) and paste (RUSTSEC-2024-0436) ignores stay:\nboth crates are still in Cargo.lock and `cargo audit` fails without them.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* chore: cite ASF LEGAL-732 for CDLA-Permissive-2.0 in deny.toml\n\nCDLA-Permissive-2.0 was allowed without a justification. ASF LEGAL-732\ntreats it as Category A, the same reference Apache OpenDAL cites in its\ndeny.toml. Document it so reviewers can see why it is on the allowlist.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "5c27157675a7babb611638786195dfe24ad41dab", "tree": "2190086207253a7e94e71eaec2605e0893f88edc", "parents": [ "000e791d5229d9d136cda2b41fa0d7563fa58e1e" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Thu Oct 22 09:07:11 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 22 09:07:11 2026 +0000" }, "message": "fix(expr): widen promoted partition values and bounds to prevent false pruning (#3483)\n\n* fix(expr): widen promoted partition values before comparison\n\nA partition column promoted int-\u003elong (or float-\u003edouble) keeps its\noriginally-written values in manifests produced before the promotion.\nThe entry partition tuple is decoded under the manifest\u0027s own narrower\ntype, while the scan predicate is bound to the current wider type.\n`Datum::partial_cmp`/`PartialEq` require matching types, so the\n`ExpressionEvaluator` compared e.g. an int partition value against a\nlong predicate literal, got no ordering, and silently pruned the data\nfile, dropping matching rows from filtered scans.\n\nThe manifest-list `ManifestEvaluator` is unaffected: it decodes\n`FieldSummary` bounds with `Datum::try_from_bytes`, which already widens\na 4-byte int bound to long. Mirror that on the entry path by widening\nthe partition value to the reference\u0027s type before comparing. Decimals\nalready compare across precisions, so only int-\u003elong and float-\u003edouble\nneed widening.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* fix(expr): widen promoted column bounds in metrics pruning\n\nCompanion to the partition-tuple widening: InclusiveMetricsEvaluator\ncompares a data file\u0027s column bounds against the scan predicate, but\nthose bounds are decoded under the manifest\u0027s embedded (narrower) schema\nwhile the predicate is bound to the current (wider) type. For an int\ncolumn promoted to long (or float to double), the inequality and IN\nchecks compared mismatched types, got no ordering, and discarded files\nthat could match, dropping rows; the eq check failed open and merely\nskipped valid pruning.\n\nWiden each stored bound to the reference\u0027s type before comparing, via\nthe same `Datum::promote_to` helper, in the inequality, eq, and IN paths.\n\n* fix(scan): widen promoted partition tuples at manifest read\n\nReplaces the comparison-time partition promotion with a reader-side\nprojection, per review.\n\nThe earlier comparison-time promotion only reached the pruning\nevaluators, so a promoted partition column still mismatched in the delete\nindex: a data file\u0027s int partition never matched an eq delete or DV keyed\nby long, dropping or mis-applying deletes. Projecting the partition tuple\nfixes pruning and delete matching in one place.\n\n* fix(scan): tighten promotion gate, borrow unpromoted bounds, guard tuple length\n\nAddresses review feedback:\n- Gate `partition_types_differ` on the type pairs `promoted_partition` actually\n widens (int-\u003elong/unknown, float-\u003edouble/unknown) instead of any primitive\n difference, so a decimal-precision widening or a dropped non-numeric source no\n longer deep-clones every manifest entry on every scan.\n- `InclusiveMetricsEvaluator::promote_bound` returns `Cow` and `Datum::promote_to`\n returns `Option`, so a bound that does not widen (e.g. a string or binary) is\n borrowed instead of cloned per data file during planning.\n- `DataFile::promoted_partition` returns `DataInvalid` when the stored tuple and\n the spec partition type have different lengths, instead of silently dropping\n trailing values (and the partition-scoped deletes they would match).\n- Drop the two delete-index unit tests that only re-exercised `promoted_partition`\n plus `Struct` equality, already covered by the data_file and scan-level tests.\n\n* test(scan): add end-to-end promotion scan regressions for deletes\n\nAdds plan_files scan regressions over a promoted partition column, routing an\nold-schema manifest through the reader-side widening:\n- A v3 deletion vector written after the promotion against the older data file.\n Without the widening the DV partition check errors (long(7) vs int(7)); the\n test guards that the data file\u0027s partition widens so the DV attaches.\n- An equality delete written before the promotion against the older data file,\n so both sides widen to long. Confirms the pre-existing match is preserved.\n\nParameterizes the equality-delete fixture on whether the delete predates the\npromotion, and adds a v3 deletion-vector fixture.\n\n* fix(scan): copy dropped partition sources through unchanged\n\nA dropped partition source resolves to the Unknown primitive type.\nWidening its stored value to the promotion ceiling keyed on the backing\nliteral, so a date (stored as an int) was silently promoted to long.\nThe promotion gate skips a manifest whose only type difference is a\ndropped source, so a data manifest that widened a dropped date diverged\nfrom the delete manifest that left it alone, detaching the delete.\n\nCopy dropped-source values through unchanged and restrict the gate to\ngenuine int to long and float to double promotions on live columns.\n\n* fix(scan): normalize dropped numeric partition sources by written type\n\n* Feedback\n\n---------\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e" }, { "commit": "000e791d5229d9d136cda2b41fa0d7563fa58e1e", "tree": "aff0df2df0b0d3756888729582b3082fd9c9fa3f", "parents": [ "8fbaf47b4a3d3d599b1cb01cb5aa86b403c4b688" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Wed Oct 21 09:45:25 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 21 09:45:25 2026 +0000" }, "message": "build: disable unused zstd legacy feature (#3495)\n\n* build: drop unused brotli and zstd legacy features\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* build: keep parquet default features\n\nBrotli is still a supported Parquet codec for reads and writes.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "8fbaf47b4a3d3d599b1cb01cb5aa86b403c4b688", "tree": "82b26751b7bf2e75cff966fe2e353c119ccd4907", "parents": [ "b9d285567526b384585b1613c76b46ab5ef063dd" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Wed Oct 21 09:39:28 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 21 09:39:28 2026 +0000" }, "message": "chore: deny unused licenses and license exceptions in cargo-deny (#3497)\n\n* Deny unused allowed licenses and license exceptions\n\nCo-authored-by: Daniel Carl Jones \u003cdjonesoa@amazon.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* Format deny.toml with taplo\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Daniel Carl Jones \u003cdjonesoa@amazon.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "b9d285567526b384585b1613c76b46ab5ef063dd", "tree": "38230269e82642373427af6a8a6ce0256e4fee8d", "parents": [ "219571a4185df2bc6e144dbbcb72cf016714b61b" ], "author": { "name": "Dhruv Arya", "email": "dhruv.arya@databricks.com", "time": "Wed Oct 21 07:58:40 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 21 07:58:40 2026 +0000" }, "message": "feat(scan): support timestamp-based time travel (#3260)\n\n* feat(scan): support timestamp-based time travel\n\n* test(scan): verify timestamp scans read historical rows\n\n* docs: shorten timestamp time travel docstrings\n\n* fix: refine timestamp selection API and edge-case coverage\n\n* docs: add timestamp time travel changelog entry\n\n* test: consolidate timestamp time travel coverage\n\n* refactor: use a single snapshot selector with last-call-wins semantics\n\n* docs: clarify timestamp history selection and PyIceberg differences\n\n* test: share the Parquet fixture row count\n\n* docs: note Java timestamp ordering compatibility\n\n* docs: clarify minimum timestamp difference from Java\n\n* docs: describe snapshot selection enum\n\n* test: adapt timestamp time travel tests to relocated scan fixture\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* fix: name the timestamp when an as-of-time snapshot is missing\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* refactor: spell out the as-of-time tie-break and i64::MIN edge\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n---------\n\nCo-authored-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e" }, { "commit": "219571a4185df2bc6e144dbbcb72cf016714b61b", "tree": "30887ffd53bdd93cbf02dcfdfd7c650afb4f0708", "parents": [ "e4b9c3ef20335bbc41b079cfd795aafc5fc9be60" ], "author": { "name": "Dhruv Arya", "email": "dhruv.arya@databricks.com", "time": "Tue Oct 20 21:07:52 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 21:07:52 2026 +0000" }, "message": "fix: update rustls-webpki and webpki-roots for emSign advisories (#3496)\n\n* fix: update rustls-webpki and webpki-roots for emSign advisories\n\nBump rustls-webpki to 0.103.16 and webpki-roots / webpki-root-certs to\n1.0.10 to address RUSTSEC-2026-0382, RUSTSEC-2026-0381 and\nRUSTSEC-2026-0383. Ignore RUSTSEC-2026-0381 for the webpki-roots 0.26.11\nshim pulled in by sqlx-core 0.8.6, which re-exports the fixed 1.0.10.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* fix: drop unused sqlx TLS dev feature instead of ignoring RUSTSEC-2026-0381\n\nThe SQL catalog tests only use SQLite, so the tls-rustls feature on the\nsqlx dev-dependency is unused. Removing it drops webpki-roots from the\nlockfile, so the audit ignore is no longer needed.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n---------\n\nCo-authored-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e" }, { "commit": "e4b9c3ef20335bbc41b079cfd795aafc5fc9be60", "tree": "a569ce20466d94ec18b176124c0d79dfea68ef1f", "parents": [ "3764d5927b7f523f28c40d103e4a4df7fd16581e" ], "author": { "name": "Xander", "email": "zander181@googlemail.com", "time": "Tue Oct 20 21:00:52 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 21:00:52 2026 +0000" }, "message": "fix(spec): reuse the highest matching partition field id, not an arbitrary one (#3413)\n\n* fix(spec): reuse the highest matching partition field id, not an arbitrary one\n\n`reuse_partition_field_ids` collects (source_id, transform) -\u003e field_id\ninto a HashMap with no merge function, so when several specs give one\npair different field ids the last one visited wins. `partition_specs` is\nitself a HashMap, so the winner changes from run to run and the same\ninput metadata yields different output.\n\nKeep the highest id of each pair, as Iceberg Java does:\n\n Collectors.toMap(\n field -\u003e Pair.of(field.sourceId(), field.transform().toString()),\n PartitionField::fieldId,\n Math::max)\n\nDivergent ids for one pair cannot be produced by this crate, since the\nreuse rule keeps them converged, but they can be read from tables\nwritten before the rule existed or by writers that do not apply it.\n\n* comments\n\n* Deterministic test\n\n* clippy" }, { "commit": "3764d5927b7f523f28c40d103e4a4df7fd16581e", "tree": "3d5c0f7a4709f11b62212ed0255b862f23c0c296", "parents": [ "464225dc9d1e644fb62a5564072685f75e6c1824" ], "author": { "name": "Yuang Gao", "email": "91290127+YuangGao@users.noreply.github.com", "time": "Tue Oct 20 19:35:59 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 19:35:59 2026 +0000" }, "message": "feat(rest): support freshness-aware table loading (#3377)\n\n* feat(rest): support freshness-aware table loading\n\n* fix(rest): reject out-of-range table cache expiry\n\n* fix(rest): address review feedback on table cache\n\n* fix(rest): invalidate before sending changes and narrow credential check\n\n* test(rest): pin table cache invalidation order and credential keys" }, { "commit": "464225dc9d1e644fb62a5564072685f75e6c1824", "tree": "a3f7588ffb342f053445525a6a739d5b1a6ba7f5", "parents": [ "a9bd6c6ed9736f73508a2a89b1b42a7976f2bbea" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Tue Oct 20 19:35:10 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 19:35:10 2026 +0000" }, "message": "fix(reader): prune data columns for empty projections (#3473)\n\n* fix(reader): prune data columns for empty projections\n\nAn empty projection (e.g. `COUNT(*)`) was excluded from the metadata-only path, so every column of every data file was read. Treat it as metadata-only so the RowNumber virtual column supplies the row count with zero data columns read.\n\n* test(reader): cover empty projection with deletes and predicates\n\nRename metadata_only_projection to no_data_columns_projected, since it\nnow also covers the empty projection. Assert the output shape in the\nbytes-read test, and add empty-projection row-count tests for a\npositional delete and a predicate on a non-projected column." }, { "commit": "a9bd6c6ed9736f73508a2a89b1b42a7976f2bbea", "tree": "17f27144bb15b6ffd58ec884a5d6f733fd28b0c7", "parents": [ "da5cc19a9f7c9e8ea14dc40d4a82974cd26def42" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Tue Oct 20 09:52:44 2026 -0700" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 09:52:44 2026 -0700" }, "message": "docs: clarify how committers bypass the merge queue (#3489)\n\nRemoving a queued PR is required before a direct merge. Document only the\nCLI, since the web UI shows the bypass checkbox only while a required\ncheck is pending or failing.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "da5cc19a9f7c9e8ea14dc40d4a82974cd26def42", "tree": "a71164b719da747edbbf3fef03923a85356e3b06", "parents": [ "78f1762aca16cc4745bc913d6e782416b5705299" ], "author": { "name": "NoahKusaba", "email": "97474796+NoahKusaba@users.noreply.github.com", "time": "Tue Oct 20 03:46:45 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 03:46:45 2026 +0000" }, "message": "feat(table): add Table::with_runtime that keeps the manifest cache (#3406)\n\n* feat(table): add Table::with_runtime that keeps the manifest cache\n\nA table gets its runtime only when it is built, and rebuilding it through\nTable::builder() starts with an empty ObjectCache. An engine that caches a\nTable and wants its scans to run on the runtime executing the query has no\nway to switch runtimes without losing the cached manifests and manifest\nlists.\n\n* docs(table): tighten Table::with_runtime docs\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Kevin Liu \u003ckevin.jq.liu@gmail.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "78f1762aca16cc4745bc913d6e782416b5705299", "tree": "80b72eb4994d16918c8e6aff1a8058cd9c6fae8b", "parents": [ "0669f080668f31d2a9c602b5b774ec40743b5ed4" ], "author": { "name": "Nikhil Dabas", "email": "ndabas@gmail.com", "time": "Tue Oct 20 03:32:51 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 03:32:51 2026 +0000" }, "message": "ci: add Windows ARM64 build (#3181)\n\nAdd Windows ARM64 build\n\nCo-authored-by: Kevin Liu \u003ckevinjqliu@users.noreply.github.com\u003e" }, { "commit": "0669f080668f31d2a9c602b5b774ec40743b5ed4", "tree": "d9aecc11968ad460574f53f4f7d2ee579be6a11c", "parents": [ "0a345f202d9233481884f0d7a3ae3a4b6181044b" ], "author": { "name": "Russell Spitzer", "email": "russell.spitzer@GMAIL.COM", "time": "Tue Oct 20 03:23:29 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 20 03:23:29 2026 +0000" }, "message": "feat(transaction): record the Iceberg-Rust writer version information in snapshot summaries (#3363)\n\n* feat(transaction): record the Rust writer in snapshot summaries\n\nA commit from iceberg-rust was indistinguishable from any other client that\nomits writer identity. Stamp engine-name, engine-version, and iceberg-version\non every snapshot so mixed-writer tables can attribute the commit.\n\nGenerated-by: Cursor Grok 4.7\n\n* Remove the writer-identity comments.\n\nGenerated-by: Cursor Grok 4.7\n\n* Keep a caller-supplied engine name and version.\n\niceberg-version still identifies this library. engine-name and\nengine-version are filled only when the caller did not set them, so an\nembedding engine can record itself through snapshot properties.\n\nGenerated-by: Cursor Grok 4.7\n\n* Keep the writer-identity tests on snapshot properties.\n\nA data file and an unrelated property do not affect those keys.\nAlso list the new summary constants alphabetically.\n\nGenerated-by: Cursor Grok 4.7\n\n* Write only iceberg-version into snapshot summaries.\n\nengine-name and engine-version name the embedding engine, which this\nlibrary does not know. Callers can still set them. iceberg-version\nremains the library identity, matching Java and Go.\n\nGenerated-by: Cursor Grok 4.7\n\n* Drop the caller engine-identity snapshot test.\n\nThose keys are ordinary snapshot properties now, so preserving a\ncaller value is already covered by the existing property test.\n\nGenerated-by: Cursor Grok 4.7\n\n* Stop asserting that engine identity keys are absent.\n\nThe snapshot summary no longer writes those keys, so the test only\nchecks iceberg-version.\n\nGenerated-by: Cursor Grok 4.7\n\n* Update crates/iceberg/src/transaction/append.rs\n\nCo-authored-by: Matt Butrovich \u003cmbutrovich@users.noreply.github.com\u003e\n\n* Move the iceberg-version summary key next to the other keys.\n\nThe transaction path still writes the value. The property name now\nlives with the rest of the summary keys the library defines.\n\nGenerated-by: Cursor Grok 4.7\n\n* Set iceberg-version from the snapshot summary module.\n\nThe key stays private next to the other summary keys, and the commit\npath calls set_iceberg_version after the totals are computed.\n\nGenerated-by: Cursor Grok 4.7\n\n* Restore the blank line before the SnapshotProduceOperation docs.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Matt Butrovich \u003cmbutrovich@users.noreply.github.com\u003e\nCo-authored-by: Kevin Liu \u003ckevin.jq.liu@gmail.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "0a345f202d9233481884f0d7a3ae3a4b6181044b", "tree": "fee7a8e0f096d0348a766a4943732d49e97535e3", "parents": [ "1be091a2135a5908537a5b45786ae7207a723aab" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Mon Oct 19 23:53:31 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 19 23:53:31 2026 +0000" }, "message": "ci: replace classic branch protection with a ruleset so the bypass works (#3481)\n\n* ci: move branch protection into the Merge Queue ruleset\n\nClassic branch protection can only be bypassed by admins, and while it\nexists GitHub blocks direct merges for the ruleset\u0027s bypass actors even\nwhen every requirement is met. Move the approval, linear-history,\ndeletion and force-push rules into the Merge Queue ruleset and disable\nclassic protection, as apache/seatunnel does, so iceberg-public members\ncan use the breakglass. Requirements are otherwise unchanged.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* ci: keep the approval rule out of the bypassable ruleset\n\nPut deletion, force-push, linear-history and approval rules in a separate\nBranch Protection ruleset that only apache/root can bypass, so the\niceberg-public breakglass skips only the merge queue and required checks.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* ci: use infra\u0027s default branch protection ruleset template\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* ci: keep dismiss_stale_reviews explicit\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "1be091a2135a5908537a5b45786ae7207a723aab", "tree": "5e855c378cdeea7d000d880c1916a5f9611307af", "parents": [ "697abcee75b503fc106f445dae8bcfc401524dcc" ], "author": { "name": "Marco Kroll", "email": "marco.kroll@databricks.com", "time": "Sat Oct 17 20:40:52 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sat Oct 17 20:40:52 2026 +0000" }, "message": "fix(transaction): prevent committing to a read-only table (#3463)\n\n* Fix committing to a read-only table\n\nPrevent committing to a read-only table. This affects for all tables\nobtained via `StaticTable::into_table`.\n\n* Update CHANGELOG.md\n\nCo-authored-by: Andrei Tserakhau \u003ctserakhau@double.cloud\u003e\n\n* Update `commit` doc string\n\nCo-authored-by: Andrei Tserakhau \u003ctserakhau@double.cloud\u003e\n\n* Improve readonly test setup\n\n---------\n\nCo-authored-by: Marco Kroll \u003cmarco.h.kroll+iceberg@gmail.com\u003e\nCo-authored-by: Andrei Tserakhau \u003ctserakhau@double.cloud\u003e" }, { "commit": "697abcee75b503fc106f445dae8bcfc401524dcc", "tree": "c5f981c01479fdca6c51c078e6c0152c7c98fcd2", "parents": [ "8b59642650d1da12270a8a570e0ae7acfa86a9c6" ], "author": { "name": "Xuanyi Li", "email": "shanelxy@outlook.com", "time": "Fri Oct 16 22:12:57 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 16 22:12:57 2026 +0000" }, "message": "feat: recognize geometry and geography types [1/3] (#2933)\n\nfeat: recognize geometry and geography types\n\nCo-authored-by: Danny C Jones \u003c4976211+dannycjones@users.noreply.github.com\u003e\nCo-authored-by: Shawn Chang \u003c42792772+CTTY@users.noreply.github.com\u003e" }, { "commit": "8b59642650d1da12270a8a570e0ae7acfa86a9c6", "tree": "33cc48d7985bd64f10c54244a2dcdbc4a1d443c2", "parents": [ "061925453a91c2e505dece727c8873eb408b8f37" ], "author": { "name": "blackmwk", "email": "257669749+blackmwk@users.noreply.github.com", "time": "Fri Oct 16 17:40:33 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 16 17:40:33 2026 +0000" }, "message": "test(glue): cover KMS property forwarding (#3478)\n\nRecord the full map received by the KMS factory and check initialization error propagation. Label missing and empty warehouse cases and document the crate-private Glue properties.\n\nFollow-up to apache/iceberg-rust#3102.\n\nGenerated-by: Codex" }, { "commit": "061925453a91c2e505dece727c8873eb408b8f37", "tree": "4f1887a44643853881dddf47756e1f3849767693", "parents": [ "f83bec42298a1cf51a42ea2aee26c041a15db5b4" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Fri Oct 16 17:07:12 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 16 17:07:12 2026 +0000" }, "message": "ci: let committers bypass the Merge Queue ruleset for urgent fixes (#3458)\n\nGive apache/iceberg-public a pull_request-only bypass on the Merge Queue\nruleset so an approved urgent fix can skip the queue and required\nchecks. Direct pushes stay blocked, and the approval in classic\nprotection still applies. The team holds committers who opted in via\nBoxer; the regular committers team is secret and cannot be a bypass\nactor. apache/root can always bypass.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "f83bec42298a1cf51a42ea2aee26c041a15db5b4", "tree": "178b14479fc8f4c125ad7880e0726c4e6298cf85", "parents": [ "5dc649bec6aee9cc25f33e9b6bb2462378646e18" ], "author": { "name": "Joseph Lenton", "email": "JosephLenton@users.noreply.github.com", "time": "Thu Oct 15 20:04:13 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 20:04:13 2026 +0000" }, "message": "fix: arrow schema to schema supports uuids (#2930)\n\n* chore: add a test capturing the change\n\n* fix: add uuid support to arrow_schema_to_schema\n\n* chore: add uuid to the large arrow_schema_to_schema_auto_assign_ids test\n\n* chore: update the public api\n\n* feat: add a type check on the uuid conversion\n\n* chore: add a test for schema_to_arrow_schema for uuid\n\n* feat: add UUID support for schema_to_arrow_schema\n\n* docs: add uuid extension to visit_field documentation\n\n* refactor: remove duplicate UUID extension import from test\n\n* chore: add tests for fixed size binary of wrong size\n\n* feat: enable parquet writing uuid with uuid extension\n\n* chore: fmt Cargo.toml" }, { "commit": "5dc649bec6aee9cc25f33e9b6bb2462378646e18", "tree": "d7ee2356aea21e56d61125ee201a1a1b3a3cbb08", "parents": [ "3bb8167a2c83fe11229b1cf67de0e641c49c8ef6" ], "author": { "name": "Matt Butrovich", "email": "mbutrovich@users.noreply.github.com", "time": "Thu Oct 15 19:59:40 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 19:59:40 2026 +0000" }, "message": "ci: Limit open, non-draft PRs to 3 for users without write access (#3465)\n\n* enforce open pull request limit and add to CONTRIBUTING.md\n\n* trim comment length\n\n* Update CONTRIBUTING.md\n\nCo-authored-by: Shawn Chang \u003cyxchang@amazon.com\u003e\n\n---------\n\nCo-authored-by: Shawn Chang \u003cyxchang@amazon.com\u003e" }, { "commit": "3bb8167a2c83fe11229b1cf67de0e641c49c8ef6", "tree": "2a3652c2c330aee096cc5c9039eb4462a112d112", "parents": [ "e4bec84bcca5ba4f3e19583e95b1de7c1fab2559" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Thu Oct 15 18:47:28 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 18:47:28 2026 +0000" }, "message": "ci: move required status checks into the Merge Queue ruleset (#3459)\n\nClassic branch protection can only be bypassed by admins. Defining the\nrequired checks in the Merge Queue ruleset lets that ruleset\u0027s bypass\nactors skip them. The checks, their contexts and the non-strict policy\nare unchanged, and approvals and linear history stay in classic\nprotection.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "e4bec84bcca5ba4f3e19583e95b1de7c1fab2559", "tree": "016151d6bfb5f6cb26a370396f7be532189d1d99", "parents": [ "b3470c8463c281c6e00ea9eee402881bb6c6d1fd" ], "author": { "name": "Jegor van Opdorp", "email": "jegorvanopdorp@gmail.com", "time": "Thu Oct 15 18:28:41 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 18:28:41 2026 +0000" }, "message": "feat(transaction): let a snapshot operation report removed files and rewrite manifests (#3253)\n\n* Let a snapshot operation report the files it removed\n\nThe snapshot summary could only count what the caller asked to remove, so an\noperation that skips a path it never had, or is handed the same path twice,\ninflated deleted-data-files and deleted-records. SnapshotProduceOperation now\nreports the entries it actually marked Deleted, each with the schema and\npartition spec of the manifest that recorded it, and the summary counts those.\n\nTwo changes follow from it. The manifests are produced before the summary,\nsince the counts are only known once the operation has run, which means the\nadded data files are cloned rather than taken. And the full-table truncate that\ndrops the previous totals is gated on a new operation hook instead of on the\noperation being Overwrite: a partial overwrite keeps the totals, and with\ntruncate off update_totals needs a saturating subtraction so removing more than\nthe previous total does not underflow.\n\n* Keep manifest names unique across a commit\n\nThe manifest name is built inline in new_manifest_writer from the metadata\nlocation, the commit uuid and a counter, so anything else that needs to write a\nmanifest for the same commit has to reinvent it, and a name invented elsewhere\nis neither unique within the commit nor findable by orphan cleanup after a\nfailed one. Pull the name out into new_manifest_path, and make the counter an\nAtomicU64 so it can be bumped behind a shared reference.\n\n* Write a manifest under the schema and spec it was written with\n\nnew_manifest_writer hardcoded the table\u0027s current schema and default partition\nspec, so an operation that rewrites an existing manifest could not keep the\nschema and spec that manifest\u0027s entries were recorded under, and building a\nwriter by hand instead means losing the encryption manager the method already\nbranches on: on an encrypted table the manifest would go out in plaintext while\nits key metadata said otherwise. The schema and partition spec become arguments,\nand the added-files path passes the table\u0027s current ones.\n\n* test(transaction): cover partial overwrite and per-file spec in snapshot summary\n\nCo-authored-by: Marco Kroll \u003c216147982+mkroll-db@users.noreply.github.com\u003e\n\n* Drop the full-table truncation from the snapshot summary\n\nJava and PyIceberg derive totals from the files an operation marks deleted, with no truncation path.\n\n* Omit a summary total that would go negative, as Java does\n\n* Name the removed-file fields and take the manifest spec by ref\n\n* Take the added files again instead of cloning them for the summary\n\n* Assert an operation reports exactly the files it marked deleted\n\n* Say in the removed_data_files doc what it must match\n\n* Summarize deleted files from the producer\u0027s list, as pyiceberg does\n\nDrops the removed_data_files hook and the manifest-first reorder, and truncation defaults back to every overwrite.\n\n* Check that deletion summaries match manifest entries\n\n* Format dropped-source integer partition values\n\n---------\n\nCo-authored-by: Marco Kroll \u003c216147982+mkroll-db@users.noreply.github.com\u003e" }, { "commit": "b3470c8463c281c6e00ea9eee402881bb6c6d1fd", "tree": "398da21ae23ed3b178a3f40e2799ddf88e80d6bb", "parents": [ "ba849a6ed3030c237e46be3e1d39e67cebf035a7" ], "author": { "name": "blackmwk", "email": "257669749+blackmwk@users.noreply.github.com", "time": "Thu Oct 15 18:12:58 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 18:12:58 2026 +0000" }, "message": "refactor(glue): derive catalog configuration from properties (#3102)\n\n* refactor(glue): derive catalog configuration from properties\n\nUse the shared property derive for Glue catalog settings while preserving warehouse validation and forwarding AWS and storage properties.\n\nTracks [apache/iceberg-rust#3096](https://github.com/apache/iceberg-rust/issues/3096).\n\nGenerated-by: Codex\n\n* refactor(glue): simplify catalog builder and tests\n\nDerive Default for GlueCatalogBuilder, use the conventional tests module name, and remove the FileIO forwarding test requested in review.\n\nGenerated-by: Codex" }, { "commit": "ba849a6ed3030c237e46be3e1d39e67cebf035a7", "tree": "ddc0dde6bf50a061b1985942323eed05c81f8a77", "parents": [ "5c00d464915449344c774fd910eec31488a01980" ], "author": { "name": "Yuang Gao", "email": "91290127+YuangGao@users.noreply.github.com", "time": "Thu Oct 15 08:08:16 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 08:08:16 2026 +0000" }, "message": "ci: fail on rustdoc warnings (#3389)" }, { "commit": "5c00d464915449344c774fd910eec31488a01980", "tree": "7b781ffeecb7a6e399ac736d2d7bae9af314c882", "parents": [ "e6b2ed4fb9cd51ba772074ff874f0b8cb7b6bef8" ], "author": { "name": "NoahKusaba", "email": "97474796+NoahKusaba@users.noreply.github.com", "time": "Thu Oct 15 08:07:58 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 08:07:58 2026 +0000" }, "message": "feat(catalog-loader): give CatalogLoader the BoxedCatalogBuilder settings (#3408)\n\n#3286 added with_runtime to BoxedCatalogBuilder, but CatalogLoader, the\none-line entry point, still had no way to set a runtime, so a catalog\nloaded through it always fell back to Runtime::current(). It had no way\nto set a storage or KMS client factory either, so it couldn\u0027t load a SQL\ncatalog at all: SqlCatalog requires a storage factory.\n\nAdd with_storage_factory, with_kms_client_factory and with_runtime to\nCatalogLoader, and apply them to the builder in load(). Raised in review\nof #3286." }, { "commit": "e6b2ed4fb9cd51ba772074ff874f0b8cb7b6bef8", "tree": "d2949d74ec4f95943dd8812fd9d72b086c820ca1", "parents": [ "a72aa958fd54728e1dd22f448c1107cf858e9213" ], "author": { "name": "JeonDaehong", "email": "90895144+JeonDaehong@users.noreply.github.com", "time": "Thu Oct 15 08:07:24 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 08:07:24 2026 +0000" }, "message": "fix(spec): format float and double partition values like iceberg-java (#3358)\n\n* fix(spec): format float and double partition values like iceberg-java\n\n* test(spec): move float to_human_string test to datum tests" }, { "commit": "a72aa958fd54728e1dd22f448c1107cf858e9213", "tree": "61ff578d8543bb5855db33464b0ab93ebe0f23f5", "parents": [ "e948f627352b520bc3a37c76502a7464f9312f86" ], "author": { "name": "Shawn Chang", "email": "schang@apache.org", "time": "Thu Oct 15 08:06:43 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 08:06:43 2026 +0000" }, "message": "feat(transaction): Introduce stateful TransactionAction trait (#3265)\n\n* feat(transaction): Introduce stateful TransactionAction trait\n\nRework the TransactionAction trait per the stateful transaction RFC\n(milestone 1): each action declares an associated retry-persistent\nState type created via new_state(), commit() borrows the action\nimmutably and its state mutably per attempt, and a consuming cleanup()\nplus CommitStatus define terminal-cleanup semantics (not yet invoked;\nwired in a later milestone).\n\nTransactions now store one entry per action, pairing immutable intent\nwith exclusively-owned execution state behind an object-safe\nErasedActionEntry adapter. Cloning a transaction creates a new\nexecution of the action plan: actions are duplicated and paired with\nfresh state, so retry state is never shared between executions.\n\nAll existing actions migrate with State \u003d () and no-op cleanup.\n\n* chore: Regenerate public-api baseline for action Clone impls\n\n* better comments\n\n* address comments\n\n* address comments\n\n* Address comments, add stateless tx action for convenience\n\n* clean up\n\n* rebase\n\n* minor\n\n* minor\n\n* fix import\n\n* actually test states reset and carried\n\n* address comments\n\n* update comments\n\n* fix clippy\n\n* testing" }, { "commit": "e948f627352b520bc3a37c76502a7464f9312f86", "tree": "1413c8ad9ed0f08e0fbbb69e252ccb2c9bf29807", "parents": [ "582b9e103513f21094f61cd1f09fdbf39dccf13e" ], "author": { "name": "NoahKusaba", "email": "97474796+NoahKusaba@users.noreply.github.com", "time": "Thu Oct 15 08:06:17 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 08:06:17 2026 +0000" }, "message": "fix(scan): don\u0027t project a stale schema after a schema update with no write (#2904)\n\n* init\n\n* cleanup\n\n* minimal tests\n\n* fix conflicts and cleanup\n\n* test(scan): cover each schema change after the last write\n\nTest the other schema changes that reach the current-schema branch of\nTableScanBuilder::build, each through a real read of the fixture\u0027s data\nfiles and with full assertions on what comes back:\n\n- a column dropped and added back under the same name reads as null,\n where it previously returned the old column\u0027s values\n- a renamed column reads under its new name\n- a dropped column is no longer selectable from the current state, but\n a scan pinned to the snapshot still reads it\n- a filter on an added column binds; is_null matches every row\n\nMerge the two current-state tests for an added column into one that\nreads the column, and assert the full error of the time-travel test.\n\nDocument on TableScanBuilder::snapshot_id that pinning a snapshot also\nselects the schema it was written with, and rename PlanContext\u0027s\nsnapshot_schema to scan_schema, since it now holds the current schema\nfor a scan of the current state.\n\n* refactor(scan): keep PlanContext\u0027s snapshot_schema name, and document what it holds\n\n* ci: retrigger\n\n* test(scan): check snapshot and schema in unit tests, move data checks to integration tests\n\nAdd TableTestFixture::new_with_column_added_after_last_write and slim the\nscan unit tests to assert the scan\u0027s snapshot and projected schema. The data\nchecks for added, dropped, re-added and renamed columns move to\nread_schema_changed_after_last_write.rs, against tables provisioned by Spark.\n\n* test: remove integration tests for scans after a schema change\n\nEnd-to-end checks are tracked in apache/datafusion-iceberg#75 as sqllogictests." }, { "commit": "582b9e103513f21094f61cd1f09fdbf39dccf13e", "tree": "66f2c69c77ea0bc459186f823fd18022eca64b58", "parents": [ "bdee10e629f3d2b68336b2dc9a5a7a6fd2ae7e3b" ], "author": { "name": "Giladi", "email": "dkgs1998@gmail.com", "time": "Thu Oct 15 05:44:23 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 15 05:44:23 2026 +0000" }, "message": "feat(delete-vector): make the delete_vector module public (#3461)\n\nAn engine that uses the crate for metadata but reads Parquet with a reader\nof its own cannot go through ArrowReader, and so far had to reimplement the\ndeletion-vector-v1 decoding to apply deletion vectors. Expose the module,\nthe way puffin already is, with the docs deny(missing_docs) then requires,\nan is_empty next to len, and the public API snapshot regenerated.\n\nCo-authored-by: Claude \u003cnoreply@anthropic.com\u003e" }, { "commit": "bdee10e629f3d2b68336b2dc9a5a7a6fd2ae7e3b", "tree": "b0026e7f17b0f302bc6ccb23c484b0a88d0b8bce", "parents": [ "e23c5dbc33d67c723a28dda58aa19407519d8dcc" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Wed Oct 14 23:18:13 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 23:18:13 2026 +0000" }, "message": "ci: pin ubuntu-latest runners to ubuntu-24.04 (#3456)\n\nubuntu-latest jobs across the apache org are queuing for 50-80 minutes,\nwhile ubuntu-24.04 jobs on the same image start within minutes.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "e23c5dbc33d67c723a28dda58aa19407519d8dcc", "tree": "87e27b2955a82c9db122e810e834fe2c5600a4cd", "parents": [ "f7e036cfff13b0718f11095062e2de88a9fe9a75" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Wed Oct 14 23:15:23 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 23:15:23 2026 +0000" }, "message": "ci: run lightweight jobs on ubuntu-slim (#3457)\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "f7e036cfff13b0718f11095062e2de88a9fe9a75", "tree": "775fce46d7d722a45c419a23a11a73a98ba8e99c", "parents": [ "a269dad56bd90d6b5f8cda026f25274642b78baf" ], "author": { "name": "Shawn Chang", "email": "schang@apache.org", "time": "Wed Oct 14 23:14:11 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 23:14:11 2026 +0000" }, "message": "ci: Cancel superseded workflow runs (#3444)" }, { "commit": "a269dad56bd90d6b5f8cda026f25274642b78baf", "tree": "6d6ae3c9028e739874f61de6a68cc722cbe65115", "parents": [ "a853455db9ed693ceae74f5d2edc1aa346169542" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Wed Oct 14 22:12:51 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 22:12:51 2026 +0000" }, "message": "fix(reader): rewrite NOT out of the read predicate before pruning (#3416)\n\n* fix(reader): rewrite NOT out of the read predicate before pruning\n\nThe reader fans a FileScanTask\u0027s predicate to the row-group metrics\nevaluator and the page-index evaluator, both conservative (inclusive)\nfilters that cannot soundly negate a NOT node. The row-group evaluator\u0027s\nnaive `!inner` flips a MIGHT_MATCH bound to CANT_MATCH. For NOT(x \u003e 5) over\na row group spanning [0, 10] that prunes a group which does contain matching\nrows (silent data loss). The page-index evaluator rejects NOT outright,\naborting the read. A FileScanTask built or deserialized outside TableScan\ncarries an un-normalized predicate, so a raw NOT reaches both.\n\nRewrite NOT out of the predicate once, where it is first bound in the read\npipeline, so every consumer sees a NOT-free predicate.\n\n* PR feedback" }, { "commit": "a853455db9ed693ceae74f5d2edc1aa346169542", "tree": "4a34a0ca9917e72690364902cf95ca5ae00c841b", "parents": [ "8bfe970f0028acc98ade0199ce34300e000fdba6" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Wed Oct 14 21:36:31 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 21:36:31 2026 +0000" }, "message": "chore(deps): Bump rustls to 0.23.46 to fix RUSTSEC advisories (#3455)\n\nFixes RUSTSEC-2026-0346, RUSTSEC-2026-0347, RUSTSEC-2026-0348 and\nRUSTSEC-2026-0349, which fail the security audit check.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "8bfe970f0028acc98ade0199ce34300e000fdba6", "tree": "5064d336d1b4b5ab103b57e65026f674bc4060af", "parents": [ "1270e329b517a88bdb8eb641b4a59f45bdc0e872" ], "author": { "name": "blackmwk", "email": "257669749+blackmwk@users.noreply.github.com", "time": "Wed Oct 14 13:47:17 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 13:47:17 2026 +0000" }, "message": "ci: increase merge queue status check timeout (#3443)\n\n* ci: increase merge queue status check timeout\n\n* ci: set merge queue status check timeout to 360 minutes" }, { "commit": "1270e329b517a88bdb8eb641b4a59f45bdc0e872", "tree": "8119e27e5c8de551642a2e3d76e167968da4c157", "parents": [ "b154603a710d730d2f43f8af4db257cc137237ef" ], "author": { "name": "Shawn Chang", "email": "schang@apache.org", "time": "Wed Oct 14 03:42:36 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Oct 14 03:42:36 2026 +0000" }, "message": "chore(deps): Bump quinn-proto and event-listener to fix RUSTSEC advisories (#3425)\n\nchore(deps): Bump quinn-proto to fix RUSTSEC advisories\n\nUpdate Cargo.lock to pick up quinn-proto 0.11.19, which fixes\nRUSTSEC-2026-0338/0339/0340/0341/0342. Also bump event-listener\n(RUSTSEC-2026-0221, unsound) and the yanked chacha20 and spin\nreleases so cargo audit runs clean." }, { "commit": "b154603a710d730d2f43f8af4db257cc137237ef", "tree": "295942a232cc32fbf55117912011f6e9dbb3fd6c", "parents": [ "b55cda1cb9e503336bdd41e2b2489415216577d7" ], "author": { "name": "advancedxy", "email": "807537+advancedxy@users.noreply.github.com", "time": "Mon Oct 12 19:41:02 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 19:41:02 2026 +0000" }, "message": "fix(transform): correct decimal and binary truncate literals (#3243)\n\n* fix(transform): correct decimal and binary truncate literals\n\nThe Arrow array path already preserves decimal precision and scale and\nsupports binary truncation. The literal path instead reset decimal scale\nto zero and rejected binary values. For example, truncate[50](10.65)\nreturned 1050 instead of 10.50.\n\nAlign literal truncation with the Arrow array path: preserve the source\ndecimal precision and scale, and return the first W bytes of binary\nvalues, or the entire value when shorter. Decimal projection boundary\nadjustments must also preserve precision and scale to keep their values\ncorrect before truncation.\n\n* fix(transform): address truncate literal review feedback" }, { "commit": "b55cda1cb9e503336bdd41e2b2489415216577d7", "tree": "317a2360c5bbbe95318272af3ad4927b2b8a3d69", "parents": [ "693339072b87eebf075479393a8ef55182f27aca" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Mon Oct 12 18:27:09 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 18:27:09 2026 +0000" }, "message": "fix(reader): apply name mapping to nested fields (#3384)\n\n* fix(reader): apply name mapping to nested fields\n\napply_name_mapping_to_arrow_schema assigned field IDs to top-level\ncolumns only. The projection then never selected the leaves of a\nstruct, list, or map in a file without embedded IDs, and those columns\nread as NULL whenever a top-level column was projected too.\n\nApply the mapping at every depth, as Java\u0027s ApplyNameMapping does.\nStruct children are looked up within their parent\u0027s entry, and list\nelements and map keys and values as element, key, and value, whatever\nthe file calls them.\n\nCloses #3374\n\n* Size collections up front, shorten comments, dedupe tests\n\n- Build each rebuilt Fields from Vec::with_capacity instead of a\n fallible collect, and the map entries from a two-element array.\n- Size the maps and index list in\n get_arrow_projection_mask_with_field_ids.\n- Keep one reader test for the reported struct case and cover every\n nested Arrow type in one schema-level test.\n\n* Address review: skip entries without a field ID, pin types and errors\n\n- Treat a mapping entry without a field ID as missing, so its nested\n fields keep no IDs either.\n- Note that the map shape checks are defensive, and report the entry\n count when one fails.\n- Note that the projection treats ListView and LargeListView as leaves.\n- Test unmapped children, an entry without a field ID, the fixed-size\n list size, the map ordering flag, and a predicate on a nested field." }, { "commit": "693339072b87eebf075479393a8ef55182f27aca", "tree": "192e06caeba7b80ff4beb1d3064c3c8aabf715e6", "parents": [ "acd610997de78f986ece9f89c3bf476de5e1f5a5" ], "author": { "name": "DeviousCardi", "email": "115358213+DeviousCardi@users.noreply.github.com", "time": "Mon Oct 12 08:48:39 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Oct 12 08:48:39 2026 +0000" }, "message": "fix(arrow): decode short BYTE_ARRAY decimal row-group stats (#3364)\n\nget_parquet_stat_min_as_datum / get_parquet_stat_max_as_datum decoded\nBYTE_ARRAY decimal bounds with a strict 16-byte conversion. Parquet\nstores these as minimal-length big-endian two\u0027s-complement values, so\nany bound shorter than 16 bytes failed to convert and aborted the\nrow-group read.\n\nUse the variable-length, sign-extending i128_from_be_bytes helper, as\nthe FIXED_LEN_BYTE_ARRAY arm already does. Bounds longer than 16 bytes\nstill return DataInvalid. Empty bounds also return DataInvalid instead\nof decoding as zero, which could otherwise prune matching row groups.\n\nCloses #3280" }, { "commit": "acd610997de78f986ece9f89c3bf476de5e1f5a5", "tree": "3ef8a259311941617351fef163e5561cef04b1d6", "parents": [ "d3e9f456e29b10ba95f11b3a2bc5c9dc546e1d93" ], "author": { "name": "Minh Vu", "email": "vuhoangminh97@gmail.com", "time": "Sun Oct 11 18:29:22 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 18:29:22 2026 +0000" }, "message": "perf(reader): speed up row group filtering and name mapping (#3269)\n\n* perf(reader): optimize row group filter intersection\n\n* perf(reader): add row group intersection benchmark\n\n* style: format iceberg benchmark manifest\n\n* perf(reader): avoid allocations in name mapping lookup\n\n* perf(reader): prune row groups from split candidates\n\n* fix(reader): format tests and preallocate name mapping index\n\n* fix(reader): tighten candidate pruning and name mapping coverage" }, { "commit": "d3e9f456e29b10ba95f11b3a2bc5c9dc546e1d93", "tree": "8f01055754b18791740b8c68b8572efb257fe8d5", "parents": [ "9092073f988b3e6401d4ca00e2b50b7a53434ee1" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Sun Oct 11 18:28:33 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 18:28:33 2026 +0000" }, "message": "perf(reader): read leaf field IDs from metadata in the projection mask (#3396)\n\nget_arrow_projection_mask_with_field_ids carried each leaf\u0027s field ID\nfrom its first pass to its second through a HashMap\u003cFieldRef, i32\u003e.\nHashing an Arrow Field hashes its whole DataType and allocates a sorted\ncopy of its metadata keys, and the map grew from empty. Read the ID\nfrom the leaf\u0027s PARQUET:field_id metadata in both passes instead.\n\nCloses #3394" }, { "commit": "9092073f988b3e6401d4ca00e2b50b7a53434ee1", "tree": "cd7e9e1f2d50cc1cfa41bfa1f0f80e5cb24ab65e", "parents": [ "2fddb1601a9af1c97a559baa9865e1494d93756c" ], "author": { "name": "NoahKusaba", "email": "97474796+NoahKusaba@users.noreply.github.com", "time": "Sun Oct 11 18:20:15 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 18:20:15 2026 +0000" }, "message": "fix(arrow): strip metadata from nested fields and normalize container field names (#3303)\n\n* fix(arrow): strip metadata from list elements and map entries\n\nstrip_metadata_from_schema failed with \"Field stack underflow in list\" on\nany schema with a list or map column. MetadataStripVisitor pushed a field\nonto its stack only in before_field, which the visitor does not call for\nlist elements, map keys or map values, so rebuilding those popped from an\nempty stack. Push in before_list_element, before_map_key and\nbefore_map_value too.\n\nThe stack now holds each field\u0027s name and nullability rather than a\nField with a placeholder type, and one pop_field helper rebuilds fields\nfor struct, list, map and primitive.\n\nDocument the two normalizations the function makes besides removing\nmetadata, which callers comparing stripped schemas rely on: a map\u0027s\nentries field is renamed to DEFAULT_MAP_FIELD_NAME, and a\ndictionary-encoded field becomes its value type.\n\nCloses #3297\n\n* test(arrow): strip metadata from fields with repeated nested names\n\n* refactor(arrow): rename MetadataStripVisitor\u0027s field stack helpers to start_field and finish_field\n\n* docs(arrow): clarify visitor field hooks and strip_metadata_from_schema naming\n\nbefore_field and after_field run for every schema and struct field, not\nfor list elements or map keys and values. strip_metadata_from_schema\nkeeps list element names, and renames only a map\u0027s entries field.\n\n* normalize list and map field names when stripping metadata" }, { "commit": "2fddb1601a9af1c97a559baa9865e1494d93756c", "tree": "5908bd138792f0ee96ceec82184fa37cb1f0c990", "parents": [ "050ddfda869068cf2d19683e45d447314181415b" ], "author": { "name": "Minh Vu", "email": "vuhoangminh97@gmail.com", "time": "Sun Oct 11 18:19:26 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 18:19:26 2026 +0000" }, "message": "fix(spec): reject boolean literals for non-boolean types (#3334)\n\n* fix(spec): reject boolean literals for non-boolean types\n\n* test(spec): polish boolean literal validation coverage" }, { "commit": "050ddfda869068cf2d19683e45d447314181415b", "tree": "21ecd477f62413a3ae7c71d37cb63b4f2b9b3a83", "parents": [ "a76678363835a3035c38b9ab53785e9d9b762d24" ], "author": { "name": "Ethan Woodbury", "email": "woodburyethan60@gmail.com", "time": "Sun Oct 11 18:16:21 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 18:16:21 2026 +0000" }, "message": "fix(arrow): reconcile evolved nested structs by field id, not position (#3255)\n\n* fix(arrow): reconcile evolved nested structs by field id, not position\n\nWhen a struct gains a field (schema evolution) after a data file was written,\nthe file\u0027s struct has fewer children than the table schema. RecordBatchTransformer\npromoted such columns with arrow_cast::cast, which matches struct/list/map children\npositionally -- so every child after the gap shifts and types collide, e.g.\n\u0027Casting from Utf8 to Struct\u0027 when a scalar lands on an added list\u003cstruct\u003e.\n\nReplace the flat cast with a PromotePlan that is resolved once per file when\nthe BatchTransform is built: nested struct children are matched to the target\nschema by PARQUET:field_id, recursing through list/large-list/map, so the\nper-batch work is just index lookups and array assembly. Absent optional\nchildren are filled with typed nulls; absent required children error, as do\nabsent children with an initial-default (not yet wired up for nested fields)\nand source structs whose children carry no field ids, rather than silently\nproducing wrong data. Primitives still use cast for valid Iceberg promotions.\nMirrors iceberg-java\u0027s by-field-id nested readers.\n\nCloses #2617\n\n* test(arrow): cover appended nested struct field on promote\n\nAdd a unit test for struct\u003ca, b\u003e -\u003e struct\u003ca, b, c\u003e, the append case of\nnested schema evolution. The existing middle-insert test already covers\nby-id matching; this pins the \"expected 3 got 2\" failure mode.\n\n* fix(arrow): reconcile nested reorder and rename by field id\n\nAlways build a PromotePlan for a present column and pass through only\nwhen that plan is a no-op. equals_datatype ignored nested names and\nfield ids, so a reorder or same-type rename kept the file layout.\n\nId-less nested children still match by position when their types line\nup, which covers maps. Nested initial-default stays unsupported (#3261).\n\n* fix(arrow): error when only some nested fields have ids\n\nThe positional fallback stays for a fully id-less struct. If some\nchildren have field ids and some do not, null-filling the rest would\ndrop data, so that now errors. A same-type id-less reorder is still\npositional; it cannot be detected without ids.\n\n* fix(arrow): cast id-less nested fields by position\n\nName mapping only stamps top-level ids. When the child counts match,\npair by position and let build check each child, so an int still\npromotes to long. A different count, or ids on only some children,\nstill errors.\n\n* test(arrow): compare remaining promote results as whole arrays\n\nUse invalid_data! for the PromotePlan DataInvalid errors, matching main.\n\n* fix(arrow): accept either list offset width from the file\n\nA Parquet Arrow schema hint can make a list column LargeList while the\ntable schema is List. Main cast that column; PromotePlan rejected it.\n\n* fix(arrow): apply nested identity constants and drop LargeList targets\n\nA missing nested field that is an identity partition source takes that\npartition value before initial-default or null. schema_to_arrow_schema\nalways emits List, so a LargeList target now errors instead of building\na plan; a LargeList file column still promotes to List.\n\nThe remaining promote tests compare whole arrays. Error tests check\nErrorKind, including field ids on only some children.\n\n* Simplify PromotePlan and dedup nested promotion tests\n\n* fix(arrow): address final review on nested field-id promotion\n\n- Add a reader-level test that writes an old nested schema to Parquet and\n reads it back through an evolved one, pinning that parquet-rs attaches\n field ids to struct children, list elements, and map keys and values\n- Reject duplicate source field ids and nested sources read as a\n non-nested type when the plan is built\n- Report non-struct map entries as DataInvalid\n- Pin the id-less same-type reorder fallback with a test until #1845\n- Assert map data types, since map array equality skips field names\n- Point at #3378 where the projection mask skips all-missing structs\n\n* test(arrow): cover sliced list promotion with a null slot\n\n---------\n\nCo-authored-by: Jordan Epstein \u003cjordan.epstein@imc.com\u003e" }, { "commit": "a76678363835a3035c38b9ab53785e9d9b762d24", "tree": "af061bbe1b2c9a60e995efd58d3319f54553d849", "parents": [ "d63e8610f2152158882f8773f3745f3f32752912" ], "author": { "name": "Xander", "email": "zander181@googlemail.com", "time": "Sun Oct 11 14:25:41 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 14:25:41 2026 +0000" }, "message": "feat(RTAS) Assign fresh ids (#3066)\n\n* feat(RTAS) Assign fresh ids\n\n* update tests with more types\n\n* avoid clone\n\n* document assign better\n\n* relax test\n\n* additional test\n\n* comment\n\n* Add debug assert\n\n* lowercase\n\n* doc\n\n* not nested\n\n* test\n\n* test comments\n\n* comments\n\n* starts_from" }, { "commit": "d63e8610f2152158882f8773f3745f3f32752912", "tree": "a5ad75e8b1a7a439af789da2263fd289f8b8e21f", "parents": [ "e42840bc98cbf91a67c014162ed68036a8452d7a" ], "author": { "name": "Manu Zhang", "email": "OwenZhang1990@gmail.com", "time": "Sun Oct 11 13:53:01 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 13:53:01 2026 +0000" }, "message": "fix(scan): reject ambiguous case-insensitive fields (#3070)\n\n* fix(scan): reject ambiguous case-insensitive fields\n\nCo-authored-by: Codex \u003ccodex@openai.com\u003e\n\n* fix(schema): clarify ambiguous case-insensitive lookup\n\nExpose the checked lookup, document collision semantics, and cover the public fallback. Simplify the binding and scan callers.\n\nCo-authored-by: Codex \u003ccodex@openai.com\u003e\n\n* fix(schema): improve case-insensitive collision diagnostics\n\nList sorted colliding field names with invalid_data!, simplify optional lookup, and loosen error assertions while checking the reported colliders.\n\nCo-authored-by: Codex \u003ccodex@openai.com\u003e\n\n---------\n\nCo-authored-by: Codex \u003ccodex@openai.com\u003e" }, { "commit": "e42840bc98cbf91a67c014162ed68036a8452d7a", "tree": "9d12d576599831f5eb8405715ec4a8323d992813", "parents": [ "b2e90754b1f9eaea741d353d51ae644aa9cef479" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Sun Oct 11 10:37:04 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 11 10:37:04 2026 +0000" }, "message": "test(reader): widen FLBA decimal page-index coverage and clarify docs (#3386)\n\ntest(reader): widen FLBA decimal page-index coverage and clarify guard docs\n\nAddress review feedback on #3328:\n- Reframe the width-guard doc and inline comment as defence-in-depth\n against a non-conforming writer\n- Hoist the duplicated min/max decode into a single closure.\n- Add page-index tests" }, { "commit": "b2e90754b1f9eaea741d353d51ae644aa9cef479", "tree": "26ac1a5f80e93dfda3a79cf20bcd93f0e2532150", "parents": [ "b8467510f8da71b8f180affdef7b1a2d3dd4cfa9" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Fri Oct 09 23:19:20 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 23:19:20 2026 +0000" }, "message": "chore(deps): Bump fsspec from 2026.4.0 to 2026.6.0 in /bindings/python (#3366)\n\nBumps [fsspec](https://github.com/fsspec/filesystem_spec) from 2026.4.0 to 2026.6.0.\n- [Commits](https://github.com/fsspec/filesystem_spec/compare/2026.4.0...2026.6.0)\n\n---\nupdated-dependencies:\n- dependency-name: fsspec\n dependency-version: 2026.6.0\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "b8467510f8da71b8f180affdef7b1a2d3dd4cfa9", "tree": "adefaff5e93a6348b00ba950fbc2616514e636c7", "parents": [ "e8ec7f8403554496db976700eb993459e0ab7f9b" ], "author": { "name": "Jiajia Li", "email": "lijiajia@outlook.com", "time": "Fri Oct 09 23:17:10 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 23:17:10 2026 +0000" }, "message": "feat(rest): add a SigV4 request signer (#3082)\n\n* feat(rest): add a SigV4 request signer\n\n* fix(rest): mute aws-sigv4 tracing and tighten payload hashing\n\nThe crate traces the request it signs, and its redaction list covers\n`authorization` but not the `Original-` copy we make, so a delegate\u0027s\nbearer token could reach trace logs. Also reject non-UTF-8 headers\nrather than leave them unsigned, and hash a present-but-empty body as\nJava does.\n\n* refactor(rest): take credentials per call, as Java\u0027s signer does\n\nJava holds one `Aws4Signer` across sessions and resolves credentials from\nan `AwsCredentialsProvider` per request, so the signer itself carries no\ncredential state. Follow that: drop `AwsCredentials`, take the AWS\ncrate\u0027s `Credentials` as a `sign` argument, and leave the provider to the\nauth session.\n\nAlso sign `HttpRequest` rather than the concrete request type, name the\namz headers, split `convert_headers`/`update_request_headers` after their\nJava counterparts, collapse the one-file `sigv4` module, and put the AWS\ndependencies behind a `sigv4` feature.\n\n* docs(rest): trim the SigV4 signer comments\n\n* refactor(rest): break sign_at into named steps\n\n* docs(rest): state the redirect and log-always limits of the signer\n\n* fix(rest): keep relocation headers out of the signature, and the tracing flag untouched\n\n* docs(rest): name the log bridge that stays unmuted\n\n* docs(rest): log-always also bypasses the muted subscriber\n\n* fix(rest): return an error instead of unwrapping the content hash header\n\n* refactor(rest): make PayloadHashMode non_exhaustive and SigV4Signer::new take impl Into\u003cString\u003e\n\n* docs(rest): note the base64 header quirk and credential freshness\n\n* docs(rest): keep the plus note next to the rewrite it explains\n\n* fix(rest): spell out the headers Java leaves unsigned\n\n* test(rest): check signatures against Iceberg Java\n\n* test(rest): pin the signing tests to a fixed time\n\n* test(rest): check standard mode against the AWS test suite\n\n* test(rest): drop the helper-only AWS vector and the hmac dev-dependency\n\n* fix(rest): decide the tracing mute with public tracing API\n\n* docs(rest): tighten the payload hash and credential notes\n\n* test(rest): bound the live-clock stamp by readings around the call\n\n* chore(rest): update the public API for the signer changes\n\n* docs(rest): simplify the SigV4 signer comments\n\n* refactor(rest): build SigV4Signer with a typed builder\n\n* fix(rest): propagate a failed userinfo strip instead of signing the URL as is\n\n* docs(rest): name the flag the tracing mute is gated to avoid\n\n* docs(rest): say that as_bytes merges an absent body with an empty one\n\n* chore(rest): update the public API for the signer builder\n\n* fix(rest): validate requests before signing\n\n* fix(rest): address SigV4 review findings" }, { "commit": "e8ec7f8403554496db976700eb993459e0ab7f9b", "tree": "749fb305f8ebc29d826f17724c30ac5cd267e1cd", "parents": [ "1f3bc34058e3f3e45336bc5428f37b92649a608b" ], "author": { "name": "Danny C Jones", "email": "djonesoa@amazon.com", "time": "Fri Oct 09 22:31:27 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 22:31:27 2026 +0000" }, "message": "chore(release): port the dependency license check from shell to Python (#3379)\n\nThis change migrates from the old shell script for cargo-deny to a new Python script.\nThis new script will contain logic for dealing with dependencies and their licenses. This commit simply translates the existing logic.\n\nIn later changes, this script will be extended with:\n- Clarifications for existing dependencies where licenses are misreported, with additional verifications added in the script itself\n- Logic for generating the attribution documentation for the pyiceberg-core package." }, { "commit": "1f3bc34058e3f3e45336bc5428f37b92649a608b", "tree": "5443baff30916cade486146f5d55052ee663aff5", "parents": [ "3c952e1719a4902334e44b648698999cf3a4214d" ], "author": { "name": "Feng Zhang", "email": "f.feng.zhang@gmail.com", "time": "Fri Oct 09 17:29:50 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 17:29:50 2026 +0000" }, "message": "fix(spec): treat -0.0 and 0.0 as distinct float literals (#3327)\n\n* fix(spec): treat -0.0 and 0.0 as distinct float literals\n\nPrimitiveLiteral equality and ordering now follow Java\u0027s Float.compare and\nDouble.compare: -0.0 is less than 0.0 and all NaNs are equal. This stops the\npartitioned writers, the record batch splitter and the delete file index from\nmerging -0.0 and 0.0 identity partitions.\n\nCloses #3325\n\n* Address review: fail closed in PrimitiveLiteral fallback, test IN\n\n- Handle AboveMax and BelowMin explicitly and return None when the\n fallback arm sees two values of the same variant, with a debug\n assertion, instead of calling them equal.\n- Cite the spec\u0027s partition equality rule and document the\n variant_index ordering invariant.\n- Stop pinning the cross-variant order in tests.\n- Test IN with signed zeros in the expression evaluator." }, { "commit": "3c952e1719a4902334e44b648698999cf3a4214d", "tree": "89a2184e1b61565a270782b8e66ce1178de02952", "parents": [ "5b82ab99aeaf50bf4f5b3b776ac9cd6380a5f3e3" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Fri Oct 09 16:15:35 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 16:15:35 2026 +0000" }, "message": "feat(reader): page-index pruning for FIXED_LEN_BYTE_ARRAY decimals (#3328)\n\nDecimals with precision \u003e 18 are stored as FIXED_LEN_BYTE_ARRAY, which the\npage-index evaluator skipped, so inequality and IN predicates on them got no\npage pruning (over-selection only, never dropped rows). Decode those bounds\nwith i128_from_be_bytes, matching the row-group statistics path.\n\nGuard against truncated ColumnIndex bounds: a bound whose width differs from\nthe column\u0027s declared type_length signals a column_index_truncate_length\ntruncation, and a truncated byte prefix no longer preserves the two\u0027s\ncomplement numeric ordering of decimals. Reject it so the column skips pruning\nrather than decode the wrong number. parquet-rs never truncates decimal column\nindexes, so this only defends against files written by other implementations.\n\nBenchmark\n\nA throwaway benchmark on a 1,000,000-row single-column decimal(30,2) file\n(precision \u003e 18 -\u003e FIXED_LEN_BYTE_ARRAY), 100 pages in one row group, read\nfully in memory so timing is decode-only. A `col \u003e threshold` predicate sweeps\nselectivity. \"read full file\" is today\u0027s behavior (FLBA decimals skip page\npruning); \"read ms\" applies the page-index RowSelection this change produces.\n\n 1,000,000 rows in 100 pages, 1 row group\n read full file : 3.431 ms\n\n predicate (keep) | rows read | eval us | read ms | speedup\n 1% | 10,000 | 17.5 | 0.038 | 90.0x\n 10% | 100,000 | 17.1 | 0.330 | 10.4x\n 50% | 500,000 | 16.6 | 1.629 | 2.1x\n 100% | 1,000,000 | 16.3 | 3.397 | 1.0x\n\nThe win tracks predicate selectivity; eval() adds a constant ~17 us per row\ngroup per scan, negligible against the millisecond read. The benefit only\nmaterializes where page-index row selection runs: MoR tables or\nwith_row_selection_enabled(true). Reproducer in the PR description.\n\nFollow-up to #3314." }, { "commit": "5b82ab99aeaf50bf4f5b3b776ac9cd6380a5f3e3", "tree": "1adf20b377221b9633936c0147201fa82a430733", "parents": [ "b0d8baaf966813f82363279db46d6688bf455d24" ], "author": { "name": "Matt Butrovich", "email": "mbutrovich@users.noreply.github.com", "time": "Fri Oct 09 16:08:16 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 16:08:16 2026 +0000" }, "message": "perf(manifest): upgrade to apache-avro 0.22 and read manifest entries without Value (#3354)\n\n* perf(manifest): upgrade to apache-avro 0.22 and read manifest entries without Value\n\n* add test cases, update docs\n\n* address PR feedback\n\n* address PR feedback.\n\n* address PR feedback" }, { "commit": "b0d8baaf966813f82363279db46d6688bf455d24", "tree": "de4667354e5d352663018e8263756b4059a184a6", "parents": [ "49b91d1c9508758f116bdc6947d2f2cba9f9980b" ], "author": { "name": "hylin", "email": "linhongyu510@gmail.com", "time": "Fri Oct 09 12:36:50 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 12:36:50 2026 +0000" }, "message": "fix(scan): validate deletion vector delete tasks (#3145)\n\n* docs: design delete task validation\n\n* fix(scan): validate delete task construction\n\n* refactor(scan): use validated delete task builder\n\n* refactor(scan): centralize deletion vector validation\n\n* style(scan): format validated delete task changes\n\n* fix(scan): satisfy public API and license checks\n\n* test(scan): adapt merged reader fixtures\n\n* refactor(scan): encapsulate delete file metadata\n\n* fix(scan): propagate invalid delete task errors\n\n* fix(scan): unwrap the delete-file builder in the scan test\n\n`FileScanTaskDeleteFile::builder().build()` returns `Result` since this\nbranch made it validate deletion-vector metadata, and this call site was\nmissed, so `cargo check --all-targets` failed with E0308 once the branch\nwas rebased onto a main that also compiles the lib test target.\n\nThe task built here is a complete equality delete, so validation cannot\nfail; `.unwrap()` matches how the other tests in this file build tasks.\n\n* refactor(scan): make delete-file byte offsets unsigned\n\nAddresses @blackmwk\u0027s review. content_offset and content_size_in_bytes\nare byte quantities, so Option\u003cu64\u003e makes a negative unrepresentable\ninstead of something every layer re-checks.\n\nThat collapses the three findings into one change:\n\n- \u0027Why not change it to u64?\u0027 / \u0027Ditto\u0027: the getters and fields are u64.\n- \u0027This duplicates a lot.\u0027: the duplication existed only to build the\n \u0027deletion vector\u0027 vs \u0027delete file\u0027 prefix for two near-identical\n negative-value errors. Both are gone, and the remaining presence\n checks collapse to one lookup plus one error site.\n- \u0027I think this should also apply to non deletion vector?\u0027: it now does,\n and unconditionally. The i64 -\u003e u64 conversion happens once in\n TryFrom\u003c\u0026DeleteFileContext\u003e, where the manifest value enters, so every\n delete-file kind is covered rather than only Puffin position deletes.\n\nAlso addresses @mbutrovich\u0027s deserialization concern from the same\nreview: FileScanTaskDeleteFile derives plain Deserialize, so a scan plan\nrebuilt on a worker never passes through build(). With u64 fields, serde\nitself rejects a negative offset, so a hand-written or corrupted plan\nfails before any loader code runs -- stronger than the runtime check it\nreplaces.\n\nDownstream, validate_deletion_vector_task loses its two\nu64::try_from conversions; the values are already u64.\n\nTests: the negative cases moved from the builder to the manifest\nboundary (test_delete_context_rejects_negative_*), including the non-DV\ncase, and the two loader tests now assert serde refuses the value.\n\ncargo test -p iceberg --lib: 1683 passed, 0 failed.\ncargo fmt --all -- --check and cargo clippy -p iceberg --all-targets\n-D warnings: clean. public-api.txt updated for the two getters.\n\nSigned-off-by: linhongyu510 \u003clinhongyu510@gmail.com\u003e\n\n* refactor(scan): validate delete files on deserialize and enforce size pairing\n\nAddress review follow-ups on the delete-file validation design:\n\n- Route FileScanTaskDeleteFile deserialization through a FileScanTaskDeleteFileSerde\n mirror that builds via builder().build(), so a deletion vector rebuilt from a\n serialized scan plan (planner-\u003eworker path) passes validate() like a manifest-\n built task. A malformed plan now fails at deserialize time instead of reaching a\n loader expect/ok_or_else.\n- Enforce the spec pairing rule for every delete-file kind: content_size_in_bytes\n is required whenever content_offset is present, not only for deletion vectors.\n- Replace two builder-only duplicate tests with one driving the missing-coordinate\n check through get_deletes_for_data_file, where the manifest entry is converted.\n- Reword PR-history-laden comments to describe the code as it is now.\n\ncargo test -p iceberg --lib: 1683 passed, 0 failed.\n\n* docs(scan): clarify deletion vector loader extraction\n\n---------\n\nSigned-off-by: linhongyu510 \u003clinhongyu510@gmail.com\u003e\nCo-authored-by: linhongyu510 \u003clinhongyu510@users.noreply.github.com\u003e\nCo-authored-by: linhongyu510 \u003cmailto:linhongyu510@gmail.com\u003e" }, { "commit": "49b91d1c9508758f116bdc6947d2f2cba9f9980b", "tree": "def9d79a653cfdfa912c8eb832740f2fa4d8c266", "parents": [ "7f9691fc7ed395ecb062a14ae918cb3707f22d30" ], "author": { "name": "Danny C Jones", "email": "djonesoa@amazon.com", "time": "Fri Oct 09 08:59:03 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 09 08:59:03 2026 +0000" }, "message": "chore: Remove generated DEPENDENCIES.rust.tsv files, and associated scripts and guides (#3338)" }, { "commit": "7f9691fc7ed395ecb062a14ae918cb3707f22d30", "tree": "58857051a01b0679b03f916dd156d151f7a25ede", "parents": [ "776648bee4c52ece8a057cfcf0d7756416299ba8" ], "author": { "name": "Jannik Steinmann", "email": "jannik.steinmann@datadoghq.com", "time": "Thu Oct 08 23:55:44 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 08 23:55:44 2026 +0000" }, "message": "feat(rest_catalog::auth) Contextual Sessions (#3081)\n\n* Add trait method `AuthManager::contextual_session`\n\n* Implement contextual session handling\n\n* Produce an auth_session per operation and re-use it for multiple Http\nrequests\n\n* Rename client to RestCatalogClient\n\n* Mark context in contextual_session as used in API docs\n\n* Drop Java-derived comment on releasing resources\n\n* Revert \"Produce an auth_session per operation and re-use it for multiple Http\"\n\nThis reverts commit 32a9dc5f2f1e35012bdcc48fcd9167af0fcc585b.\n\n* Expand test coverage\n\n* Pass auth session to client on each call\n\nThis prevents cloning the HeaderMap of the client because a new session\nis passed for every HTTP request anyway\n\n* Leave post_form unauthenticated\n\n* Implementor guidance" }, { "commit": "776648bee4c52ece8a057cfcf0d7756416299ba8", "tree": "6bf87904970f068441ee3f0c0063125a044d7652", "parents": [ "ab047b5e2f6e2ba6b999731653a3f7722b84e6f7" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Thu Oct 08 11:09:32 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 08 11:09:32 2026 +0000" }, "message": "docs: add agent guidance to reduce verbosity (#3337)\n\n* docs: add agent guidance on code comments and API surface\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: add agent guidance on PR descriptions\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: separate always-flag review rules\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: soften always-flag wording\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: drop refactor test from comment guidance\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: base pub guidance on intended API\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: clarify visibility guidance\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: add reuse and test duplication guidance\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs: trim reuse guidance\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* Apply batched suggestions from code review\n\nCo-authored-by: Matt Butrovich \u003cmbutrovich@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\nCo-authored-by: blackmwk \u003c257669749+blackmwk@users.noreply.github.com\u003e\nCo-authored-by: Matt Butrovich \u003cmbutrovich@users.noreply.github.com\u003e" }, { "commit": "ab047b5e2f6e2ba6b999731653a3f7722b84e6f7", "tree": "e61b93a4abaf3449a21fb4571a8206fa9f4af34c", "parents": [ "4ee006f50c372d8311e70f6afc67bf2499372b83" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:19:11 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:19:11 2026 +0000" }, "message": "chore(deps): Bump urllib3 from 2.7.0 to 2.8.0 in /bindings/python (#3313)\n\nBumps [urllib3](https://github.com/urllib3/urllib3) from 2.7.0 to 2.8.0.\n- [Release notes](https://github.com/urllib3/urllib3/releases)\n- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)\n- [Commits](https://github.com/urllib3/urllib3/compare/2.7.0...2.8.0)\n\n---\nupdated-dependencies:\n- dependency-name: urllib3\n dependency-version: 2.8.0\n dependency-type: indirect\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "4ee006f50c372d8311e70f6afc67bf2499372b83", "tree": "8cb6cba6e51f8ce70ae1decf5c28f780d6e72304", "parents": [ "19f705726ab483b6285014d5cb878c541de72079" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:18:35 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:18:35 2026 +0000" }, "message": "chore(deps): Bump the codeql-action group with 2 updates (#3316)\n\nBumps the codeql-action group with 2 updates: [github/codeql-action/init](https://github.com/github/codeql-action) and [github/codeql-action/analyze](https://github.com/github/codeql-action).\n\n\nUpdates `github/codeql-action/init` from 4.37.9 to 4.38.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)\n\nUpdates `github/codeql-action/analyze` from 4.37.9 to 4.38.2\n- [Release notes](https://github.com/github/codeql-action/releases)\n- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/github/codeql-action/compare/cdf488f595d80d6e07e03d4674febd5ab45fa938...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)\n\n---\nupdated-dependencies:\n- dependency-name: github/codeql-action/init\n dependency-version: 4.38.2\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: codeql-action\n- dependency-name: github/codeql-action/analyze\n dependency-version: 4.38.2\n dependency-type: direct:production\n update-type: version-update:semver-minor\n dependency-group: codeql-action\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "19f705726ab483b6285014d5cb878c541de72079", "tree": "229c4214d57525c8aefbbb4ad640a8a8a16a698b", "parents": [ "42f8d81f5038d390cb92180e4bea629ea766750d" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:18:09 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:18:09 2026 +0000" }, "message": "chore(deps): Bump taiki-e/install-action from 2.87.8 to 2.87.21 (#3317)\n\nBumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.87.8 to 2.87.21.\n- [Release notes](https://github.com/taiki-e/install-action/releases)\n- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/taiki-e/install-action/compare/d438492cf8a250514fa2d34b30bc3c0dc37c65ff...4cef1412cce204788f482e778a0b9187f9626a29)\n\n---\nupdated-dependencies:\n- dependency-name: taiki-e/install-action\n dependency-version: 2.87.20\n dependency-type: direct:production\n update-type: version-update:semver-patch\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "42f8d81f5038d390cb92180e4bea629ea766750d", "tree": "18070efbe227a3c648fb9ed494e180edd2a857b5", "parents": [ "73cadf8dda9a4b8fa3f2cb941272dbf64206402a" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:17:47 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:17:47 2026 +0000" }, "message": "chore(deps): Bump astral-sh/setup-uv from 10.0.1 to 10.2.0 (#3318)\n\nBumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 10.0.1 to 10.2.0.\n- [Release notes](https://github.com/astral-sh/setup-uv/releases)\n- [Commits](https://github.com/astral-sh/setup-uv/compare/20cfd1bf945f4377ade1205e4dbc17946fc9a30d...c18668ad3cf93ea998bef934396af7bb5c839dc7)\n\n---\nupdated-dependencies:\n- dependency-name: astral-sh/setup-uv\n dependency-version: 10.2.0\n dependency-type: direct:production\n update-type: version-update:semver-minor\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "73cadf8dda9a4b8fa3f2cb941272dbf64206402a", "tree": "2995dfbff14e840e50eaeb4946b3f341075d46bb", "parents": [ "08cc6986bb3e3aef61a24fc38f869d124c1c0c05" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:17:24 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:17:24 2026 +0000" }, "message": "chore(deps): Bump zizmorcore/zizmor-action from 0.6.3 to 0.6.4 (#3319)\n\nBumps [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) from 0.6.3 to 0.6.4.\n- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)\n- [Commits](https://github.com/zizmorcore/zizmor-action/compare/70fb788f84895a7701f5643d103d587e460b5c99...cc914d7f3750a2d13d75c7f184a1060aa0e9d482)\n\n---\nupdated-dependencies:\n- dependency-name: zizmorcore/zizmor-action\n dependency-version: 0.6.4\n dependency-type: direct:production\n update-type: version-update:semver-patch\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "08cc6986bb3e3aef61a24fc38f869d124c1c0c05", "tree": "9f32d6a3c5b0331db875b00a867744d108d0b240", "parents": [ "05e03d179521260a1f4a230cd35542c55b009dc5" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:17:04 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:17:04 2026 +0000" }, "message": "chore(deps): Bump crate-ci/typos from 1.50.1 to 1.50.2 (#3320)\n\nBumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.50.1 to 1.50.2.\n- [Release notes](https://github.com/crate-ci/typos/releases)\n- [Changelog](https://github.com/crate-ci/typos/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/crate-ci/typos/compare/d43b6c087ac471e2ea7b8af622ff15f05c0c365b...512fc24f32f44ab01972217aaaf3dc86ec234d53)\n\n---\nupdated-dependencies:\n- dependency-name: crate-ci/typos\n dependency-version: 1.50.2\n dependency-type: direct:production\n update-type: version-update:semver-patch\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "05e03d179521260a1f4a230cd35542c55b009dc5", "tree": "1cf67574fc677219dd1d33263e2efd5e3ab55bd9", "parents": [ "78fafa99fa77903545dfd3301c8db716f3482b2a" ], "author": { "name": "dependabot[bot]", "email": "49699333+dependabot[bot]@users.noreply.github.com", "time": "Tue Oct 06 23:16:36 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:16:36 2026 +0000" }, "message": "chore(deps): Bump reqwest from 0.12.28 to 0.13.4 (#3321)\n\nBumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.12.28 to 0.13.4.\n- [Release notes](https://github.com/seanmonstar/reqwest/releases)\n- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)\n- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.12.28...v0.13.4)\n\n---\nupdated-dependencies:\n- dependency-name: reqwest\n dependency-version: 0.13.4\n dependency-type: direct:production\n update-type: version-update:semver-minor\n...\n\nSigned-off-by: dependabot[bot] \u003csupport@github.com\u003e\nCo-authored-by: dependabot[bot] \u003c49699333+dependabot[bot]@users.noreply.github.com\u003e" }, { "commit": "78fafa99fa77903545dfd3301c8db716f3482b2a", "tree": "2dd9b628a6d44e4c9ec2985ff9f3d7372eb6c258", "parents": [ "f2409e9f5eaff1c643985013d927e17fdf16e6ec" ], "author": { "name": "Yuhao Su", "email": "31772373+yuhao-su@users.noreply.github.com", "time": "Tue Oct 06 23:11:47 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 23:11:47 2026 +0000" }, "message": "fix(storage-azdls): drop the leading slash from relative paths (#3311)\n\nADLS relative paths kept the leading `/` of the URL path, unlike the other\nbackends. OpenDAL\u0027s Deleter passes paths to the service without normalizing\nthem, so FileIO::delete_stream sent `DELETE \u003cfilesystem\u003e//dir/file`, which\nADLS rejects with 400 InvalidUri.\n\nCo-authored-by: Claude Opus 5.5 \u003cnoreply@anthropic.com\u003e" }, { "commit": "f2409e9f5eaff1c643985013d927e17fdf16e6ec", "tree": "4deef7bbeb4d8bdc3fbed323c67c9c6d5463be62", "parents": [ "ecba0b537a1f78ec8125c1a26f6dfab3f95b509e" ], "author": { "name": "Liang-Chi Hsieh", "email": "viirya@gmail.com", "time": "Tue Oct 06 17:48:48 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Oct 06 17:48:48 2026 +0000" }, "message": "fix: fill schema-added nested columns with typed NULL arrays on read (#2635)\n\n* fix: fill schema-added nested columns with typed NULL arrays on read\n\nWhen a column of a nested type (list, map, or a struct with nested\nchildren) is added to the table schema after data files were written,\nreading those older files failed with \"unexpected target column type\":\nthe helpers that materialize missing columns only handled primitive\ntypes plus structs with primitive-only children, via hand-written\nper-type NULL branches.\n\nBuild the all-NULL column with arrow\u0027s new_null_array instead, which\nsupports every Arrow type (including arbitrarily nested ones), and drop\nthe per-type NULL branches from create_primitive_array_repeated and\ncreate_primitive_array_single_element.\n\nCloses #2618\n\n* test(arrow): restructure #2618 nested-null test; trim comments\n\nAddress review feedback on #2635:\n- Extract the evolved schema into `schema_with_added_nested_columns` and\n reorder the test to read top-to-bottom as a story (old file -\u003e evolved\n schema -\u003e expect typed NULLs), per @kevinjqliu.\n- Assert the all-NULL struct still carries its full nested shape (child\n fields `a` and the nested list `ys`), locking in that the type-preserving\n new_null_array fill keeps nested children.\n- Trim the over-explanatory comments on the new_null_array NULL-fill paths and\n note that the literal `1` is new_null_array\u0027s row count.\n\n* test: cover typed null helpers and clarify struct default support\n\n* test: compare full evolved schema and link default follow-up" }, { "commit": "ecba0b537a1f78ec8125c1a26f6dfab3f95b509e", "tree": "370ed68e0046c6cd61e453959313ac9fdb6120b4", "parents": [ "af1da4c5bc86178c38c1c6db0578bdd9fb7021e3" ], "author": { "name": "Andy Grove", "email": "agrove@apache.org", "time": "Sun Oct 04 21:10:38 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 04 21:10:38 2026 +0000" }, "message": "fix(transform): floor day transform for pre-epoch timestamps (#3323)\n\n* fix(transform): floor day transform for pre-epoch timestamps\n\n`Day::day_timestamp_micro` and `Day::day_timestamp_nano` took the whole\nseconds of a timestamp with `/`, which truncates toward zero, while the\nfraction of the second came from `rem_euclid`, which floors. For a\nnegative timestamp that is not on a whole second the seconds came out\none too high, so a timestamp in the last second of a day before\n1969-12-31 got the next day.\n\nCompute the day with `div_euclid` on the length of a day instead, the\nway `Hour` already does. This gives the same days as Iceberg Java\u0027s\n`DateTimeUtil`, except for a pre-epoch timestamp exactly 999999\nmicroseconds (999999999 nanoseconds) after midnight, which Java puts in\nthe day before. iceberg-rust already returned the calendar day for\nthose, as PyIceberg and the `year`, `month` and `hour` transforms do,\nand following Java there would make the transform non-monotonic.\n\nFiles written before this change can hold those rows one day too high.\nInclusive projection already widens pre-epoch `day` predicates by one\nday to cover such files, as Java does, but only for `timestamp`\ncolumns. Apply it to every timestamp type.\n\n* refactor(transform): list timestamp types for day projection adjustment\n\nMatch how transform.rs lists the timestamp types elsewhere, and keep the\nadjustment opt-in if `day` ever gains another source type. No behavior\nchange: `day` accepts `date` and the four timestamp types.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* chore(transform): trim day transform comments\n\nDrop comments that narrate history or restate the test code. The\nhistory is in the PR description.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Kevin Liu \u003ckevin.jq.liu@gmail.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "af1da4c5bc86178c38c1c6db0578bdd9fb7021e3", "tree": "8770bc9fa92cbc8d43cc5ca2756dedfec0fe5bf1", "parents": [ "8e81f33daa31e495b94ef9e0bc5ff614355f451c" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Sun Oct 04 16:06:52 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 04 16:06:52 2026 +0000" }, "message": "feat(storage-opendal)!: make the per-IO-operation timeout configurable (#3263)\n\n* feat(storage-opendal)!: make the per-IO-operation timeout configurable\n\n`iceberg-storage-opendal` wraps every FileIO operator in `TimeoutLayer::new()`,\nwhose 10s `io_timeout` bounds each `read`/`write` and each method call on a\nreturned reader or writer. Nothing in the property or builder surface can\noverride it, so an operation that legitimately needs longer fails\ndeterministically: the `RetryLayer` above re-sends the same request, which\ncannot fit in the budget either.\n\nAdd `client.io-timeout-ms`, parsed into an `OpenDalClientConfig` built with\n`#[derive(Properties)]` and handed to `TimeoutLayer::with_io_timeout`. The\nconfig\u0027s fields are private, so later client settings such as retry are\nadditive rather than breaking. The default is unchanged.\n\nBREAKING CHANGE: `OpenDalStorage` variants carry a `client` field, and\n`OpenDalStorage::Memory` is now a struct variant.\n\n* refactor(storage-opendal): address review feedback on the IO timeout\n\nQuote the rejected value in the parse error, so an empty input shows as\n`value: \"\"`. `TimeoutLayer` exposes no getters, so a new test compares\nits `Debug` output to pin `DEFAULT_IO_TIMEOUT_MS` to OpenDAL\u0027s default.\nThe parsing test now uses the constant instead of repeating the literal.\n\nAdd a serde round-trip test for the `client` field, including a payload\nthat omits it. Drop the `DEFAULT` associated const, which only fed a\nfallback arm that cannot run. Update the upstream\n`test_writer_close_returns_stored_size` for the struct variant, and reuse\n`StorageConfig::with_prop` and `empty_resolving_storage` in the tests.\n\n* refactor(storage-opendal): rename the IO timeout key to opendal.io-timeout-ms\n\nOnly `iceberg-storage-opendal` honors the key, so namespace it under\n`opendal.` and define it as `OPENDAL_IO_TIMEOUT_MS` next to\n`OpenDalClientConfig`, rather than in `iceberg::io`. This matches how\n`ADLS_SAS_TOKEN` and `GCS_ALLOW_ANONYMOUS` sit next to the configs that\nparse them. With the key moved, the `iceberg` crate needs no change.\n\n* refactor(storage-opendal): rename the client field to client_config\n\n`client` reads like an HTTP client handle. Rename the field on every\n`OpenDalStorage` variant, and the `OpenDalStorage::client()` accessor,\nto `client_config` to match `OpenDalClientConfig`. `opendal_config`\nwould be ambiguous next to the backend `config` field, which holds an\nOpenDAL config too.\n\n`test_default_io_timeout_matches_opendal` compares `Debug` output. Both\nsides go through the same derived impl, so a format change cannot make\nthem differ. A `Debug` that stopped printing `io_timeout` would make the\ncheck pass vacuously, so also assert that a different `io_timeout`\nproduces a different string.\n\n* fix(storage-opendal): reject a zero IO timeout when deserializing\n\n`parse_io_timeout_ms` rejected zero, but the derived `Deserialize` did\nnot go through it. A serialized `OpenDalStorage` carrying\n`\"io_timeout_ms\": 0` was rebuilt without error, and the zero reached\n`TimeoutLayer`.\n\nStore the timeout as a `NonZeroU64`. `str::parse` and serde\u0027s\n`NonZeroU64` impl both reject zero, so both construction paths share\none check. The serialized form is still a plain integer. The generated\ngetter now returns `\u0026NonZeroU64`, because the `Properties` derive only\nreturns by value for the primitive types it recognizes.\n\nThe parsing test also covers the edges of the valid range. `1` and\n`u64::MAX` are accepted, and `u64::MAX + 1` is rejected.\n\n* refactor(storage-opendal): return the IO timeout as a Duration\n\n`OpenDalClientConfig::io_timeout_ms()` returned `\u0026NonZeroU64`, because the\n`Properties` derive only returns primitives by value. Replace the generated\ngetter with `io_timeout()`, which returns a `Duration`, and keep the\nmillisecond field private.\n\nExpose the default as `OPENDAL_IO_TIMEOUT_MS_DEFAULT`, following the\n`*_DEFAULT` consts in `iceberg`, so the key\u0027s doc links to it instead of\nrepeating the number. Keep the `ParseIntError` as the source of a rejected\nvalue, as `parse_pool_property` does, so the message says why it failed.\n\nDocument that the serialized form of `OpenDalStorage` is not stable across\ncrate versions, as `FileIO::serialize_all` already does for its own.\n\nTest that the configured timeout reaches `TimeoutLayer`. A memory operator\nbehind `ConcurrentLimitLayer::new(0)` stalls every IO call, and paused tokio\ntime skips the wait and the retry backoff. Check propagation through\n`OpenDalResolvingStorage` for every enabled scheme, not only S3.\n\n* refactor(storage-opendal): take the client config defaults from the derive\n\nBuild `OpenDalClientConfig::default()` from `from_properties` with no\nproperties, so each default is declared once in `#[property]` and serde\nand `from_properties` cannot disagree as settings are added.\n\nSay in the `parse_io_timeout_ms` doc that it exists to quote the value\nand name the unit, and that `NonZeroU64` is what rejects zero. Fold the\nvalid-value asserts in the parsing test into one loop.\n\n* refactor(storage-opendal): build the client config default directly\n\n`Default` delegated to `from_properties` with no properties and called\n`expect`. A later setting without a `#[property]` default would make it\npanic, and because of the container-level `#[serde(default)]`, serde\ncalls `default()` on every `client_config` it deserializes. Build the\nstruct directly, and keep the two defaults equal with\n`test_default_matches_property_defaults`.\n\nName OpenDAL\u0027s 60-second control-operation budget in the\n`OPENDAL_IO_TIMEOUT_MS` doc, along with the calls it covers here,\n`exists` and `metadata`. `test_default_timeouts_match_opendal` now pins\nboth defaults that the doc states.\n\nPin the serialized-form break with a test that rejects the old\n`\"LocalFs\"`, `\"Memory\"` and `{\"Memory\": null}` forms. Match the stall\ntest\u0027s error on `{ timeout: 45 }`, so a 450-second budget cannot pass.\n\n* refactor(storage-opendal): keep the client config accessors crate-private\n\n`OpenDalStorage::client_config()`, `OpenDalClientConfig::io_timeout()` and\n`OPENDAL_IO_TIMEOUT_MS_DEFAULT` have no callers outside the crate. Keep them\nout of the public API so they can be exposed later without a breaking change.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* refactor(storage-opendal): trim redundant docs, comments and test\n\nFold the `u64` default into the `NonZeroU64` const, drop\n`test_default_matches_property_defaults` (one field, one shared const), shorten\ndocs and comments that restated the code, and keep the original `op` binding in\nthe `Memory` arm of `create_operator`.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* docs(storage-opendal): clarify the IO timeout and serialization docs\n\nName OpenDAL\u0027s `io_timeout` and `timeout` settings so it is clear the property\nbounds each IO call per retry attempt and leaves control operations such as\n`stat` at 60 seconds. Align the `OpenDalStorage` serialization note with the\nwording on `FileIO::serialize_all`.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* test(storage-opendal): check OpenDalStorageFactory propagates the IO timeout\n\nEvery `OpenDalStorageFactory::build` arm could fall back to the default\n`OpenDalClientConfig` without any test failing. Build each enabled backend with\na 45000 ms timeout and read it back from the serialized storage.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: ovoievodin \u003co_voievodin@apple.com\u003e\nCo-authored-by: Kevin Liu \u003ckevin.jq.liu@gmail.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "8e81f33daa31e495b94ef9e0bc5ff614355f451c", "tree": "c7acd9a1b0547dd9b16d3e07c4fc982fb9c663a8", "parents": [ "3c91b3942ab40ec005dfbb9fdbccea848392cccb" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Sun Oct 04 15:36:50 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Oct 04 15:36:50 2026 +0000" }, "message": "ci: check dependency licenses with cargo-deny on every PR (#3293)\n\n* ci: check dependency licenses with cargo-deny on every PR\n\nRun `dev/release/dependencies.sh check` in the `lint` job, so a dependency\nwhose license is not allowed by `deny.toml` fails the pull request instead\nof being found at release time.\n\nSet `[graph] all-features \u003d true` in `deny.toml`. Without it, cargo-deny\nonly follows default features. Since #3143 that graph no longer includes\nthe optional OpenDAL backends, so the check skipped the MPL-2.0 crates that\n`deny.toml` has exceptions for.\n\nCloses #3234.\n\n* Tighten docs/comments, reuse dependencies.sh in create_rc.sh\n\n- Trim comments and docs that over-explain.\n- Condense the CONTRIBUTING.md license guidance.\n- create_rc.sh calls dependencies.sh check so the release uses the\n pinned cargo-deny version, same as CI.\n- Drop unused bzip2-1.0.6 from the deny.toml allow-list.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: Kevin Liu \u003ckevin.jq.liu@gmail.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "3c91b3942ab40ec005dfbb9fdbccea848392cccb", "tree": "dc6a38d43f86a94b0600ed171b67c1ab70cfe509", "parents": [ "ffa864bf2fd3ba27ff221746f993d4bf7810440b" ], "author": { "name": "NoahKusaba", "email": "97474796+NoahKusaba@users.noreply.github.com", "time": "Fri Oct 02 18:56:13 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 18:56:13 2026 +0000" }, "message": "feat(catalog-loader): add with_runtime to BoxedCatalogBuilder (#3286)\n\n* init\n\n* address review: doc sibling methods, tidy test, note break in changelog" }, { "commit": "ffa864bf2fd3ba27ff221746f993d4bf7810440b", "tree": "fb62da868176c03c6d75f08036eaa0d0aa5656d0", "parents": [ "e9c22bf7148a2fadd8dc94879371866dea89c883" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Fri Oct 02 13:57:07 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 13:57:07 2026 +0000" }, "message": "fix(reader): page-index pruning drops rows for INT32/INT64 decimals (#3314)\n\nPage-index bounds for INT32/INT64 columns were built as Int/Long literals\nregardless of the Iceberg field type. For decimal fields, query datums use\nInt128, and Datum::partial_cmp only compares decimals when both sides are\nInt128, so every comparison returned None. This silently pruned all matching\npages for inequality and IN predicates when page-index row selection was\nenabled.\n\nConvert INT32/INT64 page bounds to Int128 for decimal fields, matching the\nrow-group statistics path.\n\nFollow-up: FIXED_LEN_BYTE_ARRAY decimals (precision \u003e 18) still get no\npage-index pruning, even though the row-group stats path decodes them. This\nonly over-selects pages (never drops rows) and is pre-existing. Decoding them\nhere later must guard against truncated ColumnIndex bounds\n(column_index_truncate_length).\n\nCloses #3264\n\nCo-authored-by: Fokko Driesprong \u003cfokko@apache.org\u003e" }, { "commit": "e9c22bf7148a2fadd8dc94879371866dea89c883", "tree": "2bad23854fe855738efb99400ba60ffb32cb7177", "parents": [ "fc682d7a050cdc05e2b90ccf04192034dc8f1a1c" ], "author": { "name": "Minh Vu", "email": "vuhoangminh97@gmail.com", "time": "Fri Oct 02 09:57:45 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 09:57:45 2026 +0000" }, "message": "perf(arrow): skip rebuilding projected arrays without parent nulls (#3282)\n\n* perf(arrow): skip rebuilding top-level projected arrays\n\n* perf(arrow): drop in-crate benchmark harness\n\n* perf(arrow): skip rebuilding nested projected arrays\n\n* perf(arrow): tighten projector zero-copy path\n\n* perf(arrow): polish projector zero-copy path" }, { "commit": "fc682d7a050cdc05e2b90ccf04192034dc8f1a1c", "tree": "e0cf3108922adaa4122f917f6daa88ce8611460c", "parents": [ "19603861d545f8edbc5016e45cb1bacb0579cbf8" ], "author": { "name": "Jeremy Schoemaker", "email": "jeremy@shoemoney.com", "time": "Fri Oct 02 09:57:27 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Oct 02 09:57:27 2026 +0000" }, "message": "fix(spec): Harden decimal and fixed type string parsing (#3256)\n\n* fix(spec): Harden decimal and fixed type string parsing\n\ndeserialize_decimal and deserialize_fixed used trim_start_matches and\ntrim_end_matches, which strip repeated occurrences and never require the\nprefix, so \"decimal(decimal(5, 2)))))\" and \"fixed[16\" both parsed\nsuccessfully. Precision was also unbounded, so \"decimal(50, 2)\" was\naccepted into Schema and TableMetadata even though the crate\u0027s own\nMAX_DECIMAL_PRECISION is 38, and the failure only surfaced much later in\nArrow or Avro conversion.\n\nParse strictly with strip_prefix and strip_suffix, and reject precision\noutside 1..\u003d38 and scale greater than precision.\n\nSigned-off-by: Jeremy Schoemaker \u003cjeremy@shoemoney.com\u003e\n\n* fix(spec): Clarify zero precision errors and verify serde roundtrips\n\n* test(spec): cover decimal and fixed parsing edge cases\n\n* fix(spec): align decimal serde validation with constructor\n\n* fix(spec): preserve type parse errors and canonical serde coverage\n\n---------\n\nSigned-off-by: Jeremy Schoemaker \u003cjeremy@shoemoney.com\u003e" }, { "commit": "19603861d545f8edbc5016e45cb1bacb0579cbf8", "tree": "e82c5ebd05d862a8c8a7d7ed03c1b102f148dfb1", "parents": [ "de19a3986fbdf399f09f71cd8c08e2573630c80f" ], "author": { "name": "kid", "email": "lucian1412@outlook.com", "time": "Thu Oct 01 14:54:21 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 01 14:54:21 2026 +0000" }, "message": "feat: add core COW rewrite primitive (#2752)\n\n* feat: add core COW rewrite primitive\n\nAdd a core copy-on-write rewrite primitive that plans candidate data\nfiles, applies a caller-provided RecordBatch rewriter, writes\nreplacement data files, and returns removed/added/unchanged file lists\nfor future overwrite-style commit actions.\n\n- Plan candidates through the existing scan path so manifest pruning\n and delete-file application stay consistent with normal reads; the\n row predicate is cleared only for the full-file rewrite input.\n- Stream rewritten batches to the replacement writer: buffer only the\n prefix before the first changed batch, then open the writer lazily;\n unchanged files open no writer, fully-deleted files produce no\n replacement.\n- Bind the replacement writer and partition key to the planned\n snapshot\u0027s schema so rewrites survive schema evolution.\n- Build the replacement parquet writer via\n ParquetWriterBuilder::from_table_properties so replacement files\n honor the table\u0027s write.parquet.* properties, carry the table\u0027s\n encryption manager so encrypted tables are not downgraded to\n plaintext, and refuse to run when write.format.default is not\n parquet.\n- Keep the planner internal; CowRewriteBuilder is the only public\n entry and CowRewriteFile fields sit behind accessors.\n\nCover delete-style, update-style, no-op, full-file delete, no-match\ndelete, delete-file planning, writer edge cases, schema evolution,\nmulti-source replacement path uniqueness, and partition-value\npreservation (including the null partition) end-to-end.\n\nPart of #2269.\n\nCo-Authored-By: Claude \u003cnoreply@anthropic.com\u003e\n\n* fix: address COW rewrite review comments\n\n- derive the effective change flag as `changed || output.is_none()` so a\n rewriter returning {changed: false, output: None} cannot silently drop\n rows while leaving the file marked unchanged; add regression tests\n- treat a candidate whose visible rows are all removed by delete files as\n removed with no replacement, so it can be compacted away with its\n delete files instead of landing in unchanged_data_files forever\n- honor write.object-storage.enabled when building replacement writers\n (add the missing table property) so object-storage-layout tables keep\n hash-entropy paths\n- count output_rows only for rows actually written to replacement files\n- derive Debug for CowBatchRewrite\n- document the sync rewriter contract, the read-only CowRewriteFile\n surface, and the prefix buffer\u0027s whole-file worst case\n- drop avoidable FileScanTask/DataFile clones and replace the writer\n expect() with a let-else invariant\n\n* fix(cow_rewrite): flush kept prefix after a dropped batch\n\nA dropped batch could mark a file changed while skipping the Some(output) branch that initialized the writer and flushed kept rows. Open the writer whenever a changed file has buffered or current output, then flush the prefix independently of the current batch output. Cover the keep-first/drop-second data-loss case and full coverage by a real position-delete file; clarify delete-file and schema contracts.\n\n* fix(cow_rewrite): guard partition preservation and document rewrite() trade-offs\n\nAddress the three non-blocking follow-ups from the approval review of\napache/iceberg-rust#2752:\n\n- Surface the prefix-buffer memory ceiling in the public rewrite()\n rustdoc: a file that never changes (or whose first change is at EOF)\n buffers its entire decoded contents, one file at a time, which can be\n several GB for a compaction-sized file.\n- Add a debug-only partition guard: replacement files record the source\n partition verbatim, so a rewriter that mutates a partition-source\n column would silently write rows under a partition they do not belong\n to. In debug builds every emitted batch is checked against the source\n file\u0027s partition; release builds are unchanged.\n- Document in the module docs that keeping the source partition spec and\n staying unsorted is, like the snapshot-schema choice, a deliberate\n divergence from Java\u0027s RewriteDataFiles / Spark DML.\n\n* fix(cow_rewrite): use relocated scan test fixture\n\n---------\n\nCo-authored-by: Claude \u003cnoreply@anthropic.com\u003e" }, { "commit": "de19a3986fbdf399f09f71cd8c08e2573630c80f", "tree": "44f0858f300e6eaad153daa2501d92f31aca6e6e", "parents": [ "36eae2969b0773ee5b6f23d9be230fd0a17ffe92" ], "author": { "name": "Dhruv Arya", "email": "dhruv.arya@databricks.com", "time": "Thu Oct 01 08:00:37 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Oct 01 08:00:37 2026 +0000" }, "message": "fix: avoid lost-wakeup hang in get_equality_delete_predicate_for_delete_file_path (#3268)\n\n* fix: avoid lost-wakeup hang in get_equality_delete_predicate_for_delete_file_path\n\nThe equality-delete loader signals completion with Notify::notify_waiters(),\nwhich stores no permit and only wakes Notified futures created before the call.\nget_equality_delete_predicate_for_delete_file_path observed the Loading state,\nreleased the read lock, and only then created the Notified future; a\nnotify_waiters() fired in that window is lost and the await hangs forever.\n\nCreate the Notified (via notified_owned) while still holding the read lock, so\nit is guaranteed to precede any notify_waiters() and be delivered.\n\nSigned-off-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e\n\n* docs: explain why building Notified under the read lock prevents the lost wakeup\n\nState the mechanism rather than the outcome: notified_owned() snapshots tokio\u0027s\nnotify_waiters_calls counter at construction, and holding the read lock orders\nthat snapshot before insert_equality_delete can advance it. This keeps a later\nrefactor from moving notified_owned() out from under the lock unnoticed.\n\nSigned-off-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e\n\n---------\n\nSigned-off-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e\nCo-authored-by: Dhruv Arya \u003caryadhruv@gmail.com\u003e" }, { "commit": "36eae2969b0773ee5b6f23d9be230fd0a17ffe92", "tree": "9cc96e17574250a2e65d073cdad609ad450f3f92", "parents": [ "71e571e531618371499eaf3c995e30c69910e2d9" ], "author": { "name": "Xander", "email": "zander181@googlemail.com", "time": "Wed Sep 30 19:25:21 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 30 19:25:21 2026 +0000" }, "message": "refactor(scan) move test utils to test_utils.rs (#3107)\n\n* refactor(scan) move test utils to test_utils.rs\n\n* move things around\n\n* move\n\n* fix\n\n* comments" }, { "commit": "71e571e531618371499eaf3c995e30c69910e2d9", "tree": "5732db244fa3960e2b08f09ad1c730cecdf884dd", "parents": [ "deab0692f6df211a396d6fc0ad21811f8b6c8491" ], "author": { "name": "kamcheungting-db", "email": "91572897+kamcheungting-db@users.noreply.github.com", "time": "Wed Sep 30 19:24:54 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 30 19:24:54 2026 +0000" }, "message": "refactor(reader): build Arrow reader metadata once (#3242)\n\n* refactor(reader): build Arrow reader metadata once\n\n* refactor(reader): address review feedback on metadata rebuild\n\n- Defer materializing an owned Arrow schema until a rebuild is actually\n needed; the embedded-IDs/no-INT96/no-row-number fast path now returns\n early without cloning.\n- Interpolate missing_field_ids, install_row_number, and the resulting\n schema into the ArrowReaderMetadata error message so failures identify\n which transform was active.\n- Add a regression test covering the combined path: a field-id-less file\n with INT96 timestamps and a _pos projection, asserting coerced\n timestamps and the row-number column together.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* refactor(reader): address remaining review feedback\n\n---------\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e" }, { "commit": "deab0692f6df211a396d6fc0ad21811f8b6c8491", "tree": "2f031cad6158cdc0d81476e02b9644bdc899b9ab", "parents": [ "8cb2adeddb4f8da1ca7bd86ca303337f011c12e2" ], "author": { "name": "Noritaka Sekiyama", "email": "moomindani@gmail.com", "time": "Tue Sep 29 11:49:43 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 29 11:49:43 2026 +0000" }, "message": "refactor(spec): make UnboundPartitionField fields private and carry source-ids (#3267)\n\n* Make UnboundPartitionField fields private and carry source-ids\n\nThe four public fields are replaced by accessors, and the derived builder\u0027s\nvisibility is limited to the crate, so an instance can only be built inside\nthe crate or read out of a partition spec JSON. That is what lets the struct\nguarantee its own invariant: source_ids always holds at least one id.\n\nsource_ids: Vec\u003ci32\u003e replaces source_id: i32. source_id() returns the single\nid, and an error for a multi-argument field rather than quietly handing back\nthe first one. The serde module reads either spelling -- source-id, or\nsource-ids for a v3 multi-argument transform -- and writes back the one that\nmatches the field.\n\nBinding a multi-argument field now fails with a clear error. A bound\nPartitionField still carries a single source_id, so until that struct is\nconverted there is nowhere to put the extra ids, and failing loudly beats\ndropping them on the way into TableCreation or TableUpdate::AddSpec.\n\n* Reject source-id and source-ids together in UnboundPartitionField\n\nPyIceberg already treats the two spellings as mutually exclusive, and no\nwriter emits both, so accepting a matching pair only let the two readers\ndisagree. Also name both spellings in the missing-id error and note why\ncheck_transform_compatibility rejects a multi-argument field.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* Make UnboundPartitionField buildable and pass it to add_partition_field\n\nAddress review: the UnboundPartitionField builder and with_field_id are\npublic, since an unbound field is not expected to be validated yet, and\nUnboundPartitionSpecBuilder::add_partition_field now takes the field\nrather than a single source id, name and transform.\n\nWith the builder public an empty source_ids can be built, so serializing\none no longer panics and both spec builders reject it with a clear error.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n* Validate UnboundPartitionField when it is built\n\nAddress review: the builder\u0027s build() now returns\nResult\u003cUnboundPartitionField\u003e and rejects an empty source_ids, following\nFileScanTask. Deserialization goes through the same builder, so the check\nlives in one place and the spec builders no longer need their own.\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e\n\n---------\n\nCo-authored-by: Isaac \u003cno-reply@databricks.com\u003e" }, { "commit": "8cb2adeddb4f8da1ca7bd86ca303337f011c12e2", "tree": "ed6abce96d8e285e48e6d3099e70be3c4292b358", "parents": [ "5a91091c53a5dfebcc8f85f78f0eeedb269d0b87" ], "author": { "name": "Alex Stephen", "email": "1325798+rambleraptor@users.noreply.github.com", "time": "Mon Sep 28 18:41:04 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 18:41:04 2026 +0000" }, "message": "fix(python): raise instead of panicking in the manifest bindings (#3241)\n\nThere\u0027s some assorted places that we need to raise errors instead\nof panicking." }, { "commit": "5a91091c53a5dfebcc8f85f78f0eeedb269d0b87", "tree": "d7e92c8216d3eca19dbb6a151a28a1099a791c6c", "parents": [ "86d6618804a902eac10f09c09a565aca5dad0d46" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Mon Sep 28 18:08:55 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 18:08:55 2026 +0000" }, "message": "chore: Extending collaborator list to help with review capacity (#3289)\n\nCo-authored-by: ovoievodin \u003co_voievodin@apple.com\u003e" }, { "commit": "86d6618804a902eac10f09c09a565aca5dad0d46", "tree": "6e7458e6267a5e292b6aa88a51bb33ec3b2e844a", "parents": [ "f64575819ec5d61a1d8ffd7740831a2cb64df5cd" ], "author": { "name": "Kevin Liu", "email": "kevinjqliu@users.noreply.github.com", "time": "Mon Sep 28 17:09:53 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 28 17:09:53 2026 +0000" }, "message": "ci: auto-label DataFusion issues and PRs (#3278)\n\nci: auto-label issues and PRs that mention DataFusion\n\nContributors without triage permission cannot add labels. Use\ngithub/issue-labeler to apply the datafusion label when an issue or PR\ntitle or body mentions DataFusion.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "f64575819ec5d61a1d8ffd7740831a2cb64df5cd", "tree": "19bce9e634cba8277e116c044a245fd3a8e1c92c", "parents": [ "615af68da85f2cc6e5139aa180c939214de91d32" ], "author": { "name": "Andrei Tserakhau", "email": "laskoviymishka@gmail.com", "time": "Sun Sep 27 02:03:28 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sun Sep 27 02:03:28 2026 +0000" }, "message": "feat(writer): add PositionDeleteInput builder (#3230)\n\n* feat(writer): add PositionDeleteInput builder\n\nAdd PositionDeleteInputBuilder, a small helper that assembles a\nspec-conforming position delete RecordBatch from (path, pos) rows.\n\nThe batch has exactly the two required position delete columns\n(file_path: Utf8, pos: Int64), both non-null and carrying the reserved\nfield ids, so it passes PositionDeleteFileWriter\u0027s per-batch validation\nand can be handed straight to that writer. An empty builder still\nproduces a valid 0-row batch.\n\nTo keep a single source of truth for the field-id wiring, the Arrow\nschema is no longer hand-wired in the builder. This commit un-gates the\npreviously test-only POSITION_DELETE_ARROW_SCHEMA static and\nposition_delete_arrow_schema() helper in position_delete_writer.rs,\nmaking the helper pub(crate), and the builder now obtains the shared\nArrow projection through it as a cheap Arc clone.\n\nextend accepts any iterator of (Into\u003cString\u003e, i64) rows, and build\ndebug-asserts the paths/positions length invariant. The module docs\nnote that rows are materialized into two heap Vecs before the batch is\nbuilt (so very large inputs should use one builder per write batch),\nthat the optional third row column is not supported yet, and that\nposition delete files are a v2 construct which v3 forbids, so a\nformat-version gate must be applied at the transaction/commit layer.\n\nRelates to #340 and #2218.\n\n* refactor(writer): replace PositionDeleteInput builder with a sorted position_delete_batch fn\n\nPer review, collapse the PositionDeleteInputBuilder surface\n(new/with_capacity/push/extend/len/is_empty/build) into a single free\nfunction, position_delete_batch(rows). A FooBuilder is expected to build a\nFoo, but the builder\u0027s build() only returned a bare RecordBatch, so a plain\nfunction reads more clearly and keeps the API as tight as possible.\n\nThe function collects rows into a BTreeMap\u003cString, RoaringTreemap\u003e keyed by\nfile path. This deduplicates repeated (file_path, pos) pairs and, when\niterated, yields the spec-required ascending (file_path, pos) order for free,\nso callers no longer have to pre-sort their input. A negative pos is rejected\nwith DataInvalid. The batch is still built from the writer\u0027s shared crate\ninternal Arrow schema, and empty input yields a valid 0-row batch.\n\n* refactor(writer): model position deletes as a crate-internal PositionDeletes accumulator\n\nFollowing blackmwk\u0027s review of the redesigned position delete assembly,\nreplace the free position_delete_batch function with a small crate-internal\nPositionDeletes accumulator. Callers build up deletes with insert(path, pos)\nand render them once via to_record_batch().\n\nPositions are now taken as u64: a row position is non-negative by\nconstruction, so the old negative-i64 rejection is gone. The batch\u0027s Int64\npos column is filled with a checked u64-to-i64 conversion that returns\nDataInvalid rather than panicking on a value past i64::MAX. Rows stay keyed\nby a BTreeMap\u003cString, RoaringTreemap\u003e, so iteration yields the spec-required\n(file_path, pos) order with duplicate pairs removed, and the two columns are\nnow built with Arrow StringBuilder and Int64Builder instead of intermediate\nVecs. The batch is assembled against the writer\u0027s shared\nposition_delete_arrow_schema, and an empty accumulator still yields a valid\n0-row batch.\n\nNothing outside the crate consumes this yet, so the whole surface is\npub(crate) and the module carries a dead-code allowance until the write-path\nplumbing lands. The public API drops the position_delete_batch entry\naccordingly.\n\n* refactor(writer): stream PositionDeletes as fixed-size record batches\n\nFollow-up to blackmwk\u0027s review on #3230.\n\n- Add PositionDeletes::to_record_batches(max_rows), a lazy iterator that\n builds each fixed-size batch on demand instead of materializing the whole\n delete set at once, bounding peak memory over large delete sets. It walks\n the paths and positions in sorted (file_path, pos) order, splitting on\n max_rows boundaries; a path whose positions overflow one batch resumes in\n the next. A max_rows of 0 yields a single DataInvalid error rather than\n panicking, and the stream stops after any error.\n- Build the file_path column with StringBuilder::append_value_n, appending\n each path\u0027s run within a batch in one call rather than once per row.\n- Factor a shared private build_batch helper so to_record_batch (single\n batch) and to_record_batches share the column-building logic.\n- Document that duplicate (file_path, pos) inserts collapse silently, which\n is intentional for position deletes.\n\n* refactor(writer): debug_assert against duplicate and oversized position deletes" }, { "commit": "615af68da85f2cc6e5139aa180c939214de91d32", "tree": "7272589cedd2edb66b726c3cc5232f069ac07939", "parents": [ "9a2111a914e9b17fdc039700186056ec14fae606" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Sat Sep 26 09:28:36 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Sat Sep 26 09:28:36 2026 +0000" }, "message": "fix(expr): decode BYTE_ARRAY page-index bounds by field type (#3247)\n\n* fix(expr): decode BYTE_ARRAY page-index bounds by field type\n\nThe page index evaluator decoded every BYTE_ARRAY min/max bound as a\nUTF-8 string via String::from_utf8(..).unwrap(), panicking during scan\npruning whenever a binary (or byte-encoded decimal) column carried a\npredicate. Convert the bound according to the field\u0027s primitive type\ninstead: string, binary, or decimal.\n\n* PR feedback\n\n* Round2 feedback\n\n* Fix trace" }, { "commit": "9a2111a914e9b17fdc039700186056ec14fae606", "tree": "f3b302f77e79cbc6dfa5dbd0bbe3bd37cb48a5e7", "parents": [ "119fad92c891c95a7819d87e4195407b3f7c0578" ], "author": { "name": "Minh Vu", "email": "vuhoangminh97@gmail.com", "time": "Fri Sep 25 21:32:09 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Fri Sep 25 21:32:09 2026 +0000" }, "message": "fix(spec): format pre-epoch timestamptz values (#2892)\n\n* fix(spec): format pre-epoch timestamptz values\n\n* fix(spec): strengthen pre-epoch timestamptz regression\n\n* style: format temporal conversion\n\n* test(spec): match chrono fractional timestamp display\n\n* test(spec): match trimmed timestamp fraction" }, { "commit": "119fad92c891c95a7819d87e4195407b3f7c0578", "tree": "f5d389db10255853983d8b109363ead7c8580e65", "parents": [ "c9a25b23b7645f3af0e9dfd7e85d8cf646901c55" ], "author": { "name": "Oleks V", "email": "comphead@users.noreply.github.com", "time": "Thu Sep 24 18:51:51 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Thu Sep 24 18:51:51 2026 +0000" }, "message": "ci: replace MinIO with RustFS for integration tests (#3273)\n\n* ci: replace MinIO with RustFS for integration tests\n\n`quay.io/minio/minio` no longer allows anonymous pulls, so\n`make docker-up` fails and the integration tests cannot start.\nDataFusion hit the same failure and switched to RustFS in\napache/datafusion#25706. PyIceberg moved to RustFS earlier in\napache/iceberg-python#3928.\n\nRun `rustfs/rustfs:1.0.0` as the shared S3 service instead. It keeps\nthe `admin`/`password` credentials, the 9000/9001 ports, and\nvirtual-hosted-style access through `RUSTFS_SERVER_DOMAINS` and the\nbucket aliases. The healthcheck uses `/health/ready`, which waits for\nstorage and IAM. The `mc` bucket setup becomes `curl --aws-sigv4` run\nfrom the RustFS image, and the public bucket policy is dropped because\nno test reads anonymously.\n\nPoint the REST fixture, HMS, and Spark at `http://rustfs:9000`, and\nrename the MinIO-specific test helpers and the\n`ICEBERG_TEST_MINIO_ENDPOINT` override to RustFS.\n\nCloses #3272\n\n* ci: use vendor-neutral object-store naming for S3 test service\n\nRename the RustFS service, hostname, bucket aliases, test helpers and env\nvar to a generic object-store name (matching PyIceberg), so future\nbackend swaps only need an image change.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* ci: fail fast and gate readiness on test bucket creation\n\nTreat any non-200/409 response as a failure instead of ignoring it, and\nadd a healthcheck so `docker compose up --wait` blocks until the buckets\nexist.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* ci: remove orphan containers on docker-up\n\nRenaming the minio service leaves stale containers holding port 9000 for\nanyone with a previously started stack; clean them up automatically.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n* ci: make create-buckets a one-shot job\n\nRustFS returns 200 when re-creating an existing bucket, so plain chained\ncurl -f calls are idempotent. Replace the status-code parsing, marker-file\nhealthcheck and tail with a one-shot container, and gate spark-iceberg on\nits successful completion.\n\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e\n\n---------\n\nCo-authored-by: ovoievodin \u003co_voievodin@apple.com\u003e\nCo-authored-by: Kevin Liu \u003ckevin.jq.liu@gmail.com\u003e\nCo-authored-by: Copilot App \u003c223556219+Copilot@users.noreply.github.com\u003e" }, { "commit": "c9a25b23b7645f3af0e9dfd7e85d8cf646901c55", "tree": "0b7a7ce54cb6adf90a7d8f001f8b33fe23059345", "parents": [ "830bb886542527f4e8956f03040e7c407b621401" ], "author": { "name": "Xander", "email": "zander181@googlemail.com", "time": "Wed Sep 23 21:20:18 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Wed Sep 23 21:20:18 2026 +0000" }, "message": "fix(encryption) [18/N] wire in ags1 file length for tamper proofing (#3236)\n\n* feat(encryption) [18N] wire in ags1 file length for tamper proofing\n\n* remove debug\n\n* expect\n\n* fix test\n\n* fix opendal\n\n* make more clear\n\n* Add oversized tests\n\n* don\u0027t do unreachable\n\n* import fix\n\n* Assert length in streaming tests\n\n* DRAFT try key_metadata_with_length\n\n* fix(encryption): address key metadata length review" }, { "commit": "830bb886542527f4e8956f03040e7c407b621401", "tree": "e16922c47f98c7e7f9e2048c55261d6d814cd852", "parents": [ "e5a18042b244e61556e8ac7daa33450f3c97cb07" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Tue Sep 22 21:36:02 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 22 21:36:02 2026 +0000" }, "message": "fix(cache): drop format-version from the manifest-list cache key (#3259)\n\n* fix(cache): drop format-version from the manifest-list cache key\n\nThe manifest-list cache key now uses the location alone; format_version\nis no longer part of it.\n\nManifestList::parse_with_version(bytes, version) is a pure function of\nits inputs, the bytes are immutable and uniquely identified by their\nlocation, and an ObjectCache is bound to a single table. So\nformat_version can never disambiguate two lookups at the same location.\n\nThe only case where it could differ for one cache is an in-place format\nupgrade (v1 -\u003e v2 mid-cache-lifetime). Dropping it is more correct there:\nthe cache serves the original correct v1 parse instead of a miss that\nwould reparse the v1 bytes under v2.\n\nAdds test_get_manifest_list_ignores_format_version, asserting two lookups\nat one location under different format versions return the same cached Arc.\n\nFollow-up to the review thread on #3235:\nhttps://github.com/apache/iceberg-rust/pull/3235#discussion_r4024075908\n\n* PR feedback" }, { "commit": "e5a18042b244e61556e8ac7daa33450f3c97cb07", "tree": "6df3adcfa7b517ffa21adeeb32cad08ab92f4926", "parents": [ "e694eea42032a3f0898ccee33f5d3dd0ca9cd24b" ], "author": { "name": "Shawn Chang", "email": "schang@apache.org", "time": "Tue Sep 22 20:55:05 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 22 20:55:05 2026 +0000" }, "message": "RFC: Stateful Transaction Action (#2620)\n\n* rfc only\n\n* update rfc\n\n* make it more readable\n\n* update rfc\n\n* update rfc\n\n* remove default impl for cleanup\n\n* box self" }, { "commit": "e694eea42032a3f0898ccee33f5d3dd0ca9cd24b", "tree": "95ebd53a69572c69ae6fae9813e83ff1a72764fa", "parents": [ "bb1e4a4861f02377489eff818b75138f414c4cb0" ], "author": { "name": "Xander", "email": "zander181@googlemail.com", "time": "Tue Sep 22 19:40:50 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Tue Sep 22 19:40:50 2026 +0000" }, "message": "Introduce invalid data error macro (#2928)\n\n* Introduce invalid data error macro\n\n* fix\n\n* comments\n\n* fix\n\n* fix\n\n---------\n\nCo-authored-by: blackmwk \u003c257669749+blackmwk@users.noreply.github.com\u003e" }, { "commit": "bb1e4a4861f02377489eff818b75138f414c4cb0", "tree": "c5219496b4eee6a2ab714c005351185ebf370a46", "parents": [ "f3e41d5321d0043697041f1353eb47d46f2b4b1d" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Mon Sep 21 21:42:58 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 21 21:42:58 2026 +0000" }, "message": "perf(expr): resolve row group stats once per eq/in/starts_with leaf (#3254)\n\nThe eq, in, and starts_with row-group metric checks each read both the\nlower and upper bound via min_value and max_value, and each of those\nindependently re-runs stats_and_type_for_field_id.\n\nResolve the stats and type once per leaf and decode each bound in place,\nso the lower and upper bounds still decode lazily." }, { "commit": "f3e41d5321d0043697041f1353eb47d46f2b4b1d", "tree": "fec085d21b13b08025e32ff359527dd1c838f054", "parents": [ "53eccc5f0892eb27b4af940e5e10993dc6ab9e4a" ], "author": { "name": "Liang-Chi Hsieh", "email": "viirya@gmail.com", "time": "Mon Sep 21 21:26:11 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 21 21:26:11 2026 +0000" }, "message": "feat(transaction): support transform-based sort orders (#2765)\n\n* feat(transaction): support transform-based sort orders\n\nReplaceSortOrderAction only supported sorting by a column\u0027s raw value:\n`asc`/`desc` hardcoded Transform::Identity and only accepted a column name.\nJava\u0027s SortOrderBuilder.asc/desc accept a Term, which can be a transform\nexpression (bucket[N], year, truncate[W], ...).\n\nAdd asc_with_transform / desc_with_transform. asc/desc are unchanged (same\nsignature and behavior) and now delegate to the new methods with\nTransform::Identity, so no existing caller is affected.\n\nTransform-compatibility with the source column\u0027s type is checked at commit\ntime via the existing SortOrder::builder().build(schema) -\u003e check_compatibility\npath, matching the timing of Java\u0027s SortOrder.Builder.build() -- the spec-level\ntype was already general (SortField.transform, check_compatibility already\nvalidate arbitrary transforms); only the transaction-layer API was narrow.\n\nScope: this only extends the metadata-declaration API. Whether the write path\nsorts data to match a table\u0027s declared sort order -- for any transform,\nincluding the pre-existing identity case -- is a separate, pre-existing gap\nuntouched here.\n\nCloses #2764\n\n* test(transaction): metadata.json round-trip test for transform sort orders\n\nPer review: commit a transform-based sort order through the in-crate memory\ncatalog (which serializes the updated metadata to a metadata.json file via\nTableMetadata::write_to), then reload the table (TableMetadata::read_from\nparses that file) and assert the transforms survive the JSON round-trip.\n\n* docs(transaction): note intended Term convergence; hoist test imports\n\nPer review: document on asc_with_transform/desc_with_transform that once\nTerm becomes transform-carrying (#2665), sort-order declaration is expected\nto converge on Term-based asc/desc (as in Java\u0027s SortOrderBuilder) and the\n_with_transform variants can be deprecated. Also hoist the use statements\nscattered inside test fns to the test module top for consistency.\n\n* fix(transaction): validate sort transforms and preserve input errors\n\n* chore: update public API snapshot for sort transforms\n\n* fix(sort): use consistent validation errors and document void policy" }, { "commit": "53eccc5f0892eb27b4af940e5e10993dc6ab9e4a", "tree": "3aea143cfbdd90766e78cc145983c7bb1cedbe41", "parents": [ "832a4eba7a732baebbbd7f76bb6fd6225f1db1e9" ], "author": { "name": "Anoop Johnson", "email": "anoop@apache.org", "time": "Mon Sep 21 20:18:28 2026 +0000" }, "committer": { "name": "GitHub", "email": "noreply@github.com", "time": "Mon Sep 21 20:18:28 2026 +0000" }, "message": "fix(cache): drop schema-id from the manifest-list cache key (#3235)\n\n* fix(cache): drop schema-id from the manifest-list cache key\n\nObjectCache::get_manifest_list built the cache key from\nsnapshot.schema_id().unwrap(), but schema-id is optional on snapshots in\nevery table version (v1-v3 per spec, and already read as Option\u003cSchemaId\u003e),\nso a snapshot that omits it panicked on every scan (the object cache is on\nby default).\n\nschema-id does not belong in this key: ManifestListReader::load parses the\nmanifest list from its bytes and the table format version only and never\nreads schema-id, and the manifest-list location is already unique per\nsnapshot.\n\n* Fix import\n\n* Harden test" } ], "next": "832a4eba7a732baebbbd7f76bb6fd6225f1db1e9" }