fix: Compare timestamps for partitions metadata (#3603)

* fix: compare timestamps for partitions metadata last_updated_at

`InspectTable._update_partitions_map_from_manifest_entry` aggregates
per-partition stats over manifest entries, which are iterated in manifest
order rather than chronologically. The guard that keeps the most recently
committed snapshot per partition compared the stored `last_updated_snapshot_id`
(a snapshot id) against `snapshot.timestamp_ms` (a commit timestamp). Snapshot
ids are large positive 63-bit integers, so `id < timestamp_ms` is effectively
always false and the row keeps whichever entry was visited first instead of the
newest one, reporting the wrong `last_updated_at` / `last_updated_snapshot_id`
for any partition touched by more than one snapshot.

Compare the stored `last_updated_at` timestamp against the incoming
`snapshot.timestamp_ms`, matching the Java `PartitionsTable.Partition.update`
which uses `snapshotCommitTime > this.lastUpdatedAt`.

* test: reuse data file in partitions regression test

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>

---------

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
2 files changed
tree: 00ce896f6dc8f9a6aa260d6f862f30984e66cfcb
  1. .github/
  2. dev/
  3. mkdocs/
  4. notebooks/
  5. pyiceberg/
  6. tests/
  7. vendor/
  8. .asf.yaml
  9. .codespellrc
  10. .gitignore
  11. .markdownlint.yaml
  12. .pre-commit-config.yaml
  13. AGENTS.md
  14. LICENSE
  15. Makefile
  16. MANIFEST.in
  17. NOTICE
  18. pyproject.toml
  19. README.md
  20. ruff.toml
  21. SECURITY-THREAT-MODEL.md
  22. setup.py
  23. uv.lock
README.md

Iceberg Python

PyIceberg is a Python library for programmatic access to Iceberg table metadata as well as to table data in Iceberg format. It is a Python implementation of the Iceberg table spec.

The documentation is available at https://py.iceberg.apache.org/.

Get in Touch