blob: 8203607df5a516838236fa2ce3ac87475f230794 [file] [view]
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
# Apache Paimon Rust
The Rust implementation of [Apache Paimon](https://paimon.apache.org/) — a streaming data lake platform with high-speed data ingestion, changelog tracking, and efficient real-time analytics.
## Overview
Apache Paimon Rust provides native Rust libraries for reading and writing Paimon tables, enabling high-performance data lake access from the Rust ecosystem.
Key features:
- Native Rust reader for Paimon table format
- Support for local filesystem, S3, OSS, COS, Azure, OBS, GCS, and HDFS storage backends
- REST Catalog integration
- Apache DataFusion integration for SQL queries
## Performance
In the TPC-DS SF100 benchmark, Paimon Rust completes the 96-query workload
within 9.3% of DataFusion's native Parquet reader. The gap narrows to 3.1% for
queries whose native-Parquet median is at least five seconds, while Paimon's
ZSTD data files use 23.25% less physical storage than the SNAPPY source files.
[Read the full TPC-DS SF100 performance report](benchmark.md)
## Status
The project is under active development (0.3.0 in development).
## License
Apache Paimon Rust is licensed under the [Apache License 2.0](https://github.com/apache/paimon-rust/blob/main/LICENSE).