fix: guard equality delete bounds against missing fields (#857)
## What
`internal::CanContainEqDeletesForFile` crashes with
`std::bad_optional_access` when an equality delete file has lower/upper
bounds for some of its equality fields but not others.
`EqualityDeleteFile::LowerBound`/`UpperBound` return
`Result<std::optional<...>>` (i.e. `std::expected<std::optional<...>,
Error>`), and a field with no bound yields an engaged `expected`
wrapping a disengaged `optional`. The guard checked
`std::expected::has_value()`, which only reports the error state and
does not detect the empty inner `optional`, so a missing bound slipped
past and `->value()` threw at the range check.
The throw is an uncaught exception escaping the `Result`-based call
chain, so it terminates the caller instead of surfacing as an error. It
is reachable during scan planning (`ManifestGroup` →
`DeleteFileIndex::ForEntry`) and commit validation
(`MergingSnapshotUpdate::ValidateNoNewDeletesForDataFiles` →
`ForDataFile`). Equality delete files with bounds for only some fields
are produced legitimately by per-column metrics modes
(`counts`/`none`/`truncate`) and by cross-engine writers, so this is
reachable in normal reader use.
Fixes #856.
## How
Unwrap the `expected` with `ICEBERG_ASSIGN_OR_RAISE` before the guard,
so `has_value()` tests the inner `optional`. This matches the data-side
guard already used a few lines above in the same function, and matches
Java's `DeleteFileIndex.canContainEqDeletesForFile`, which guards all
four bounds uniformly and assumes may-match on any missing bound (it
never prunes on missing statistics).
As a side effect this propagates a genuine bound-deserialization error
upward instead of silently treating it as may-match. That is consistent
with the data-side `Literal::Deserialize` calls in the same function,
which already propagate via `ICEBERG_ASSIGN_OR_RAISE`.
## Testing
Added `DeleteFileIndexTest.TestEqualityDeletePartialFieldBounds`: an
equality delete over fields `{1, 2}` with bounds for field 1 only,
evaluated against a data file that has bounds for both (field 1's range
overlapping so evaluation reaches field 2). Verified fail-without
(throws `std::bad_optional_access`) / pass-with (returns the delete file
as may-match). `null_value_counts` is pinned to 0 for both fields so the
null short-circuits cannot mask the range-check path if the schema
fields later change nullability. Full `manifest_test` passes.C++ implementation of Apache Iceberg™.
Required:
Optional:
git clone https://github.com/apache/iceberg-cpp.git cd iceberg-cpp cmake -S . -B build -G Ninja cmake --build build ctest --test-dir build --output-on-failure
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD_SHARED=ON cmake --build build ctest --test-dir build --output-on-failure cmake --install build
To run a specific test suite:
ctest --test-dir build -R schema_test --output-on-failure
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_BUNDLE=ON cmake --build build ctest --test-dir build --output-on-failure cmake --install build
cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DCMAKE_PREFIX_PATH=/path/to/arrow -DICEBERG_BUILD_BUNDLE=ON cmake --build build ctest --test-dir build --output-on-failure cmake --install build
| Option | Default | Description |
|---|---|---|
ICEBERG_BUILD_STATIC | ON | Build static library |
ICEBERG_BUILD_SHARED | OFF | Build shared library |
ICEBERG_BUILD_TESTS | ON | Build tests |
ICEBERG_BUILD_BUNDLE | ON | Build the battery-included library |
ICEBERG_BUILD_REST | ON | Build REST catalog client |
ICEBERG_BUILD_REST_INTEGRATION_TESTS | OFF | Build REST catalog integration tests |
ICEBERG_BUILD_HIVE | OFF | Build Hive (HMS) catalog client |
ICEBERG_BUILD_SQL_CATALOG | OFF | Build SQL catalog client |
ICEBERG_SQL_SQLITE | OFF | Build the SQLite connector for the SQL catalog |
ICEBERG_SQL_POSTGRESQL | OFF | Build the PostgreSQL connector for the SQL catalog |
ICEBERG_SQL_MYSQL | OFF | Build the MySQL connector for the SQL catalog |
ICEBERG_ENABLE_ASAN | OFF | Enable Address Sanitizer |
ICEBERG_ENABLE_UBSAN | OFF | Enable Undefined Behavior Sanitizer |
meson setup builddir meson compile -C builddir meson test -C builddir --timeout-multiplier 0
Meson provides built-in equivalents for several CMake options:
--default-library=<shared|static|both> instead of ICEBERG_BUILD_STATIC / ICEBERG_BUILD_SHARED-Db_sanitize=address,undefined instead of ICEBERG_ENABLE_ASAN / ICEBERG_ENABLE_UBSAN--libdir, --bindir, --includedir for install directoriesMeson-specific options (configured via -D<option>=<value>):
| Option | Default | Description |
|---|---|---|
rest | enabled | Build REST catalog client |
rest_integration_test | disabled | Build integration test for REST catalog |
tests | enabled | Build tests |
After installing the core libraries, you can build the examples:
cd example cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=/path/to/install cmake --build build
If you are using provided Apache Arrow, include /path/to/arrow in CMAKE_PREFIX_PATH:
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH="/path/to/install;/path/to/arrow"
If you experience network issues when downloading dependencies, you can customize the download URLs using environment variables:
ICEBERG_ARROW_URL: Apache Arrow tarball URLICEBERG_AVRO_URL: Apache Avro tarball URLICEBERG_AVRO_GIT_URL: Apache Avro git repository URLICEBERG_NANOARROW_URL: Nanoarrow tarball URLICEBERG_CROARING_URL: CRoaring tarball URLICEBERG_UTF8PROC_URL: utf8proc tarball URLICEBERG_NLOHMANN_JSON_URL: nlohmann-json tarball URLICEBERG_SPDLOG_URL: spdlog tarball URLICEBERG_CPR_URL: cpr tarball URLExample:
export ICEBERG_ARROW_URL="https://your-mirror.com/apache-arrow-22.0.0.tar.gz" cmake -S . -B build
Apache Iceberg is an active open-source project, governed under the Apache Software Foundation (ASF). Iceberg-cpp is open to people who want to contribute to it. Here are some ways to get involved:
The Apache Iceberg community is built on the principles described in the Apache Way and all who engage with the community are expected to be respectful, open, come with the best interests of the community in mind, and abide by the Apache Foundation Code of Conduct.
In addition, contributors using AI-assisted tools must follow the documented guidelines for AI-assisted contributions available on the Iceberg website: https://iceberg.apache.org/contribute/#guidelines-for-ai-assisted-contributions.
Install the python package pre-commit and run once pre-commit install.
pip install pre-commit pre-commit install
This will setup a git pre-commit-hook that is executed on each commit and will report the linting problems. To run all hooks on all files use pre-commit run -a.
We provide Dev Container configuration file templates.
To use a Dev Container as your development environment, follow the steps below, then select Dev Containers: Reopen in Container from VS Code's Command Palette.
cd .devcontainer cp Dockerfile.template Dockerfile cp devcontainer.json.template devcontainer.json
If you make improvements that could benefit all developers, please update the template files and submit a pull request.
Licensed under the Apache License, Version 2.0