Native Rust implementation of Apache Iceberg.
| Catalog Type | Status |
|---|---|
| Rest | Done |
| Hive | Done |
| Sql | In Progress |
| Glue | Done |
| DynamoDB | Not Started |
| FileIO Type | Status |
|---|---|
| S3 | Done |
| Local File | Done |
| GCS | Not Started |
| HDFS | Not Started |
Our FileIO is powered by Apache OpenDAL, so it would be quite easy to expand to other service.
| Feature | Status |
|---|---|
| File based task planning | Done |
| Size based task planning | Not started |
| Filter pushdown(manifest evaluation, partition prunning) | In Progress |
| Apply deletions, including equality and position deletions | Not started |
| Read into arrow record batch | In Progress |
| Parquet file support | Done |
| ORC file support | Not started |
| Feature | Status |
|---|---|
| Data writer | Not started |
| Equality deletion writer | Not started |
| Position deletion writer | Not started |
| Partitioned writer | Not started |
| Upsert writer | Not started |
| Parquet file support | Not started |
| ORC file support | Not started |
| Feature | Status |
|---|---|
| Schema evolution | Not started |
| Update partition spec | Not started |
| Update properties | Not started |
| Replace sort order | Not started |
| Update location | Not started |
| Append files | Not started |
| Rewrite files | Not started |
| Rewrite manifests | Not started |
| Overwrite files | Not started |
| Row level updates | Not started |
| Replace partitions | Not started |
| Snapshot management | Not started |
We will add integrations with other rust based data systems, such as polars, datafusion, etc.
Iceberg is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.
Licensed under the Apache License, Version 2.0