Release v0.4.0
chore: bump avro to 209a373 to preserve adjust-to-utc on primitives (#954)

Move the avro-cpp pin from 997d50d to 209a373 (current main, 74 commits).
The C++ changes in the range are a std::formatter fix for C++23 builds
(#3815), a JSON decoder fix for the lone low surrogate U+DFFF (#3841),
and AVRO-4351, which makes avro-cpp keep custom attributes on primitive
schema nodes when parsing and printing schema JSON.

AVRO-4351 is what matters here. `GetAdjustToUtc()` reads "adjust-to-utc"
off the primitive node, and the reader takes its file schema from the Avro
file header. With the old pin the attribute was dropped both when the
writer printed the header and when the reader parsed it, so every
timestamp read from a file looked like it had no timezone and projecting
a `timestamptz` or `timestamptz_ns` column out of an Avro file failed with
"Cannot read Iceberg type". No test wrote and re-read a timestamptz
column through a real file, so this went unnoticed.

Add schema-level projection tests that parse "adjust-to-utc" from JSON,
and file-level round trips that write timestamptz columns, assert the
attribute is present in the physical file schema, and read them back.
All of them fail on the old pin and pass on the new one.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
4 files changed
tree: bf2b7afe8897ea7e6ac658f49b5bc43b6849a35e
  1. .devcontainer/
  2. .github/
  3. ci/
  4. cmake_modules/
  5. dev/
  6. example/
  7. mkdocs/
  8. src/
  9. thirdparty/
  10. .asf.yaml
  11. .clang-format
  12. .clang-tidy
  13. .gitignore
  14. .pre-commit-config.yaml
  15. AGENTS.md
  16. cmake-format.py
  17. CMakeLists.txt
  18. LICENSE
  19. Makefile
  20. NOTICE
  21. README.md
  22. requirements.txt
  23. SECURITY-THREAT-MODEL.md
README.md

Iceberg

Slack Ask DeepWiki License

Apache Iceberg™ C++

C++ implementation of Apache Iceberg™.

Requirements

Required:

  • C++23 compliant compiler (GCC 14+, Clang 18+, MSVC 2022+)
  • CMake 3.25+
  • Ninja (recommended build backend)

Optional:

Quick Start

git clone https://github.com/apache/iceberg-cpp.git
cd iceberg-cpp
cmake -S . -B build -G Ninja
cmake --build build
ctest --test-dir build --output-on-failure

Build with CMake

Build, Run Tests and Install Core Libraries

cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_STATIC=ON -DICEBERG_BUILD_SHARED=ON
cmake --build build
ctest --test-dir build --output-on-failure
cmake --install build

To run a specific test suite:

ctest --test-dir build -R schema_test --output-on-failure

Build and Install Iceberg Bundle Library

Vendored Apache Arrow (default)

cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DICEBERG_BUILD_BUNDLE=ON
cmake --build build
ctest --test-dir build --output-on-failure
cmake --install build

Provided Apache Arrow

cmake -S . -B build -G Ninja -DCMAKE_INSTALL_PREFIX=/path/to/install -DCMAKE_PREFIX_PATH=/path/to/arrow -DICEBERG_BUILD_BUNDLE=ON
cmake --build build
ctest --test-dir build --output-on-failure
cmake --install build

CMake Build Options

OptionDefaultDescription
ICEBERG_BUILD_STATICONBuild static library
ICEBERG_BUILD_SHAREDOFFBuild shared library
ICEBERG_BUILD_TESTSONBuild tests
ICEBERG_BUILD_BUNDLEONBuild the battery-included library
ICEBERG_BUILD_RESTONBuild REST catalog client
ICEBERG_BUILD_REST_INTEGRATION_TESTSOFFBuild REST catalog integration tests
ICEBERG_SPDLOGONUse spdlog as the default logging backend
ICEBERG_BUILD_HIVEOFFBuild Hive (HMS) catalog client
ICEBERG_BUILD_SQL_CATALOGOFFBuild SQL catalog client
ICEBERG_SQL_SQLITEOFFBuild the SQLite connector for the SQL catalog
ICEBERG_SQL_POSTGRESQLOFFBuild the PostgreSQL connector for the SQL catalog
ICEBERG_SQL_MYSQLOFFBuild the MySQL connector for the SQL catalog
ICEBERG_ENABLE_ASANOFFEnable Address Sanitizer
ICEBERG_ENABLE_UBSANOFFEnable Undefined Behavior Sanitizer

Build Examples

After installing the core libraries, you can build the examples:

cd example
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=/path/to/install
cmake --build build

If you are using provided Apache Arrow, include /path/to/arrow in CMAKE_PREFIX_PATH:

cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH="/path/to/install;/path/to/arrow"

Customizing Dependency URLs

If you experience network issues when downloading dependencies, you can customize the download URLs using environment variables:

  • ICEBERG_ARROW_URL: Apache Arrow tarball URL
  • ICEBERG_AVRO_URL: Apache Avro tarball URL
  • ICEBERG_AVRO_GIT_URL: Apache Avro git repository URL
  • ICEBERG_NANOARROW_URL: Nanoarrow tarball URL
  • ICEBERG_CROARING_URL: CRoaring tarball URL
  • ICEBERG_UTF8PROC_URL: utf8proc tarball URL
  • ICEBERG_NLOHMANN_JSON_URL: nlohmann-json tarball URL
  • ICEBERG_SPDLOG_URL: spdlog tarball URL
  • ICEBERG_CPR_URL: cpr tarball URL

Example:

export ICEBERG_ARROW_URL="https://your-mirror.com/apache-arrow-22.0.0.tar.gz"
cmake -S . -B build

Contribute

Apache Iceberg is an active open-source project, governed under the Apache Software Foundation (ASF). Iceberg-cpp is open to people who want to contribute to it. Here are some ways to get involved:

The Apache Iceberg community is built on the principles described in the Apache Way and all who engage with the community are expected to be respectful, open, come with the best interests of the community in mind, and abide by the Apache Foundation Code of Conduct.

In addition, contributors using AI-assisted tools must follow the documented guidelines for AI-assisted contributions available on the Iceberg website: https://iceberg.apache.org/contribute/#guidelines-for-ai-assisted-contributions.

Linting

Install the python package pre-commit and run once pre-commit install.

pip install pre-commit
pre-commit install

This will setup a git pre-commit-hook that is executed on each commit and will report the linting problems. To run all hooks on all files use pre-commit run -a.

Dev Containers

We provide Dev Container configuration file templates.

To use a Dev Container as your development environment, follow the steps below, then select Dev Containers: Reopen in Container from VS Code's Command Palette.

cd .devcontainer
cp Dockerfile.template Dockerfile
cp devcontainer.json.template devcontainer.json

If you make improvements that could benefit all developers, please update the template files and submit a pull request.

License

Licensed under the Apache License, Version 2.0