Apache Paimon Vector Index is a pure Rust vector indexing library designed for Apache Paimon and data lake storage such as S3, HDFS, and OSS. Its seek-based readers load only the index pages needed by each query.
The library supports IVF-FLAT, IVF-SQ, IVF-PQ, IVF-RQ, and DiskANN through shared Rust, C, C++, Java/JNI, and Python APIs. IVF-RQ stores deterministic rotated residuals as 1–8 bit planes (4 by default), uses a bounded sign-plane coarse pass before full refinement, and exposes the persisted width as rq_bits metadata. DiskANN combines a Vamana graph, resident PQ codes, paged adjacency/raw vectors, and F16 or F32 reranking for indexes queried from local SSD or object storage. Its parallel builder uses contiguous fixed-capacity adjacency, adaptive dense/sparse visited state, heap frontiers, and reusable prune scratch; query-local adjacency retention is bounded and the shared cold cache is sharded for concurrent hits.
GitHub shows committed HTML files as source. To view the styled documentation, clone the repository and serve docs/ from the repository root:
python3 -m http.server --directory docs 8000
Then open http://localhost:8000/. All documentation assets are local, so no separate build step is required.
Public implementations live in core, ffi, include, jni, java, and python. See the development guide for responsibilities and verification commands.
Submit an issue or ask a question in GitHub Discussions.
Licensed under the Apache License, Version 2.0.