fix: guard equality delete bounds against missing fields (#857)

## What

`internal::CanContainEqDeletesForFile` crashes with
`std::bad_optional_access` when an equality delete file has lower/upper
bounds for some of its equality fields but not others.
`EqualityDeleteFile::LowerBound`/`UpperBound` return
`Result<std::optional<...>>` (i.e. `std::expected<std::optional<...>,
Error>`), and a field with no bound yields an engaged `expected`
wrapping a disengaged `optional`. The guard checked
`std::expected::has_value()`, which only reports the error state and
does not detect the empty inner `optional`, so a missing bound slipped
past and `->value()` threw at the range check.

The throw is an uncaught exception escaping the `Result`-based call
chain, so it terminates the caller instead of surfacing as an error. It
is reachable during scan planning (`ManifestGroup` →
`DeleteFileIndex::ForEntry`) and commit validation
(`MergingSnapshotUpdate::ValidateNoNewDeletesForDataFiles` →
`ForDataFile`). Equality delete files with bounds for only some fields
are produced legitimately by per-column metrics modes
(`counts`/`none`/`truncate`) and by cross-engine writers, so this is
reachable in normal reader use.

Fixes #856.

## How

Unwrap the `expected` with `ICEBERG_ASSIGN_OR_RAISE` before the guard,
so `has_value()` tests the inner `optional`. This matches the data-side
guard already used a few lines above in the same function, and matches
Java's `DeleteFileIndex.canContainEqDeletesForFile`, which guards all
four bounds uniformly and assumes may-match on any missing bound (it
never prunes on missing statistics).

As a side effect this propagates a genuine bound-deserialization error
upward instead of silently treating it as may-match. That is consistent
with the data-side `Literal::Deserialize` calls in the same function,
which already propagate via `ICEBERG_ASSIGN_OR_RAISE`.

## Testing

Added `DeleteFileIndexTest.TestEqualityDeletePartialFieldBounds`: an
equality delete over fields `{1, 2}` with bounds for field 1 only,
evaluated against a data file that has bounds for both (field 1's range
overlapping so evaluation reaches field 2). Verified fail-without
(throws `std::bad_optional_access`) / pass-with (returns the delete file
as may-match). `null_value_counts` is pinned to 0 for both fields so the
null short-circuits cannot mask the range-check path if the schema
fields later change nullability. Full `manifest_test` passes.
2 files changed
tree: 5f26c1b7178be7b97f4541efc0a82a81c42dedfa
  1. .devcontainer/
  2. .github/
  3. ci/
  4. cmake_modules/
  5. dev/
  6. example/
  7. mkdocs/
  8. src/
  9. subprojects/
  10. thirdparty/
  11. .asf.yaml
  12. .clang-format
  13. .clang-tidy
  14. .gitignore
  15. .pre-commit-config.yaml
  16. AGENTS.md
  17. cmake-format.py
  18. CMakeLists.txt
  19. LICENSE
  20. Makefile
  21. meson.build
  22. meson.options
  23. NOTICE
  24. README.md
  25. requirements.txt
  26. SECURITY-THREAT-MODEL.md
README.md

Iceberg

Slack Ask DeepWiki License

Apache Iceberg™ C++

C++ implementation of Apache Iceberg™.

Requirements

Required:

  • C++23 compliant compiler (GCC 14+, Clang 18+, MSVC 2022+)
  • CMake 3.25+ or Meson 1.5+
  • Ninja (recommended build backend)

Optional:

Quick Start

git clone https://github.com/apache/iceberg-cpp.git
cd iceberg-cpp
cmake -S . -B build -G Ninja
cmake --build build
ctest --test-dir build --output-on-failure

Build with CMake

Build, Run Tests and Install Core Libraries

cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD_SHARED=ON
cmake --build build
ctest --test-dir build --output-on-failure
cmake --install build

To run a specific test suite:

ctest --test-dir build -R schema_test --output-on-failure

Build and Install Iceberg Bundle Library

Vendored Apache Arrow (default)

cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_BUNDLE=ON
cmake --build build
ctest --test-dir build --output-on-failure
cmake --install build

Provided Apache Arrow

cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DCMAKE_PREFIX_PATH=/path/to/arrow -DICEBERG_BUILD_BUNDLE=ON
cmake --build build
ctest --test-dir build --output-on-failure
cmake --install build

CMake Build Options

OptionDefaultDescription
ICEBERG_BUILD_STATICONBuild static library
ICEBERG_BUILD_SHAREDOFFBuild shared library
ICEBERG_BUILD_TESTSONBuild tests
ICEBERG_BUILD_BUNDLEONBuild the battery-included library
ICEBERG_BUILD_RESTONBuild REST catalog client
ICEBERG_BUILD_REST_INTEGRATION_TESTSOFFBuild REST catalog integration tests
ICEBERG_BUILD_HIVEOFFBuild Hive (HMS) catalog client
ICEBERG_BUILD_SQL_CATALOGOFFBuild SQL catalog client
ICEBERG_SQL_SQLITEOFFBuild the SQLite connector for the SQL catalog
ICEBERG_SQL_POSTGRESQLOFFBuild the PostgreSQL connector for the SQL catalog
ICEBERG_SQL_MYSQLOFFBuild the MySQL connector for the SQL catalog
ICEBERG_ENABLE_ASANOFFEnable Address Sanitizer
ICEBERG_ENABLE_UBSANOFFEnable Undefined Behavior Sanitizer

Build with Meson

meson setup builddir
meson compile -C builddir
meson test -C builddir --timeout-multiplier 0

Meson provides built-in equivalents for several CMake options:

  • --default-library=<shared|static|both> instead of ICEBERG_BUILD_STATIC / ICEBERG_BUILD_SHARED
  • -Db_sanitize=address,undefined instead of ICEBERG_ENABLE_ASAN / ICEBERG_ENABLE_UBSAN
  • --libdir, --bindir, --includedir for install directories

Meson-specific options (configured via -D<option>=<value>):

OptionDefaultDescription
restenabledBuild REST catalog client
rest_integration_testdisabledBuild integration test for REST catalog
testsenabledBuild tests

Build Examples

After installing the core libraries, you can build the examples:

cd example
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=/path/to/install
cmake --build build

If you are using provided Apache Arrow, include /path/to/arrow in CMAKE_PREFIX_PATH:

cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH="/path/to/install;/path/to/arrow"

Customizing Dependency URLs

If you experience network issues when downloading dependencies, you can customize the download URLs using environment variables:

  • ICEBERG_ARROW_URL: Apache Arrow tarball URL
  • ICEBERG_AVRO_URL: Apache Avro tarball URL
  • ICEBERG_AVRO_GIT_URL: Apache Avro git repository URL
  • ICEBERG_NANOARROW_URL: Nanoarrow tarball URL
  • ICEBERG_CROARING_URL: CRoaring tarball URL
  • ICEBERG_UTF8PROC_URL: utf8proc tarball URL
  • ICEBERG_NLOHMANN_JSON_URL: nlohmann-json tarball URL
  • ICEBERG_SPDLOG_URL: spdlog tarball URL
  • ICEBERG_CPR_URL: cpr tarball URL

Example:

export ICEBERG_ARROW_URL="https://your-mirror.com/apache-arrow-22.0.0.tar.gz"
cmake -S . -B build

Contribute

Apache Iceberg is an active open-source project, governed under the Apache Software Foundation (ASF). Iceberg-cpp is open to people who want to contribute to it. Here are some ways to get involved:

The Apache Iceberg community is built on the principles described in the Apache Way and all who engage with the community are expected to be respectful, open, come with the best interests of the community in mind, and abide by the Apache Foundation Code of Conduct.

In addition, contributors using AI-assisted tools must follow the documented guidelines for AI-assisted contributions available on the Iceberg website: https://iceberg.apache.org/contribute/#guidelines-for-ai-assisted-contributions.

Linting

Install the python package pre-commit and run once pre-commit install.

pip install pre-commit
pre-commit install

This will setup a git pre-commit-hook that is executed on each commit and will report the linting problems. To run all hooks on all files use pre-commit run -a.

Dev Containers

We provide Dev Container configuration file templates.

To use a Dev Container as your development environment, follow the steps below, then select Dev Containers: Reopen in Container from VS Code's Command Palette.

cd .devcontainer
cp Dockerfile.template Dockerfile
cp devcontainer.json.template devcontainer.json

If you make improvements that could benefit all developers, please update the template files and submit a pull request.

License

Licensed under the Apache License, Version 2.0