Apache Paimon C++

Paimon C++ is the C++ implementation of Apache Paimon. It provides native, high-performance, and extensible access to the Paimon lake format for C++ engines and services without JVM dependencies.

Background and documentation are available at paimon.apache.org.

Features

Paimon C++ currently provides:

Write: append table and primary key table write support with compaction.
Commit: append table and primary key table commit support. Direct file-system commits work on non-object-store paths; object-store paths require a catalog-mediated commit.
Scan: batch and stream scan for append tables and primary key tables, with changelog consumption when a changelog producer is configured.
Read: append table read, primary key table read with deletion vector, and primary key table merge-on-read.
Arrow integration: batch read and write interfaces based on the Arrow Columnar In-Memory Format.
File systems: file system abstraction with built-in local, Jindo, OSS, and S3 file system support.
File formats: file format abstraction with built-in Parquet, Blob, ORC, and Avro support, plus optional Lance and Mosaic formats.
Runtime utilities: memory pool and thread pool abstractions with default implementations.
AI-oriented features: row tracking and data evolution modes, plus global index capabilities including B-tree and bitmap indexing, vector search with Lumina, and full-text search with Lucene or Paimon's Tantivy-based implementation.
Compatibility: compatibility with Apache Paimon Java format and communication protocols, including commit messages, data splits, and manifests.

Building

Building requires a C++17-compatible compiler and CMake 3.22 or newer. Linux x86_64 and aarch64 builds are supported and verified in CI. See the build guide for platform support, system requirements, dependency configuration, optional components, and instructions for building and running tests.

git clone https://github.com/apache/paimon-cpp.git
cd paimon-cpp

Build with CMake:

cmake -B build
cmake --build build

Dev Containers

We provide Dev Container configuration file templates.

To use a Dev Container as your development environment, follow the steps below, then select Dev Containers: Reopen in Container from VS Code's Command Palette.

cd .devcontainer
cp Dockerfile.template Dockerfile
cp devcontainer.json.template devcontainer.json

Examples

Start with the read and write example to see how to write, commit, scan, and read table data. The cleanup example demonstrates table maintenance. Both examples have a CMake project that uses installed Paimon and Arrow packages.

Collaboration

Paimon C++ is an active open-source project and we welcome people who want to contribute or share good ideas! Before contributing, please read the Contributing Guide and the Code Style Guide. You are encouraged to check out our documentation.

License

This project is licensed under the Apache License 2.0.