Apache Paimon C++

Paimon C++ is the C++ implementation of Apache Paimon. It provides native, high-performance, and extensible access to the Paimon lake format for C++ engines and services without JVM dependencies.

Background and documentation are available at paimon.apache.org.

Features

Paimon C++ currently provides:

Write: append table and primary key table write support with compaction.
Commit: append table commit support for simple append-only tables.
Scan: batch and stream scan for append tables and primary key tables without changelog.
Read: append table read, primary key table read with deletion vector, and primary key table merge-on-read.
Arrow integration: batch read and write interfaces based on the Arrow Columnar In-Memory Format.
File systems: file system abstraction with built-in local and Jindo file system support.
File formats: file format abstraction with built-in ORC, Parquet, and Avro support.
Runtime utilities: memory pool and thread pool abstractions with default implementations.
AI-Oriented Features: supports RowTracking and DataEvolution mode and provides Global Index capabilities including B-tree index, DiskANN-based vector search with Lumina, and Lucene-based full-text search.
Compatibility: compatibility with Apache Paimon Java format and communication protocols, including commit messages, data splits, and manifests.

Bitmap global index compatibility: Java Paimon now uses a dedicated bitmap global index format instead of the previously shared wrapped bitmap file index format. Paimon C++ therefore currently treats the bitmap global index type as unsupported. The legacy implementation remains in the codebase pending migration to the Java-compatible format.

Note: Only Linux x86_64 builds are currently supported and verified.

Building

git clone https://github.com/apache/paimon-cpp.git
cd paimon-cpp

Build with CMake:

cmake -B build
cmake --build build

Dev Containers

We provide Dev Container configuration file templates.

To use a Dev Container as your development environment, follow the steps below, then select Dev Containers: Reopen in Container from VS Code's Command Palette.

cd .devcontainer
cp Dockerfile.template Dockerfile
cp devcontainer.json.template devcontainer.json

Collaboration

Paimon C++ is an active open-source project and we welcome people who want to contribute or share good ideas! Before contributing, please read the Contributing Guide and the Code Style Guide. You are encouraged to check out our documentation.

License

This project is licensed under the Apache License 2.0.