Google Git
Sign in
apache / iceberg-rust / ffb691da60ae37b3eb4b1838e3f006838d2d3807
commitffb691da60ae37b3eb4b1838e3f006838d2d3807[log] [tgz]
authorAlon Agmon <54080741+a-agmon@users.noreply.github.com>Thu May 09 15:45:06 2024 +0300
committerGitHub <noreply@github.com>Thu May 09 06:45:06 2024 -0600
treeb23a23f61aac573451ea8a51ad9ffd9aa2f08f41
parent1c66e5a6f5f5382cac12c561ade8147cb8d6c7cd [diff]
fix (manifest-list): added serde aliases to support both forms conventions (#365)

* added serde aliases to support both forms conventions

* reading manifests without avro schema

* adding avro files of both versions and add a test to deser both

* fixed typo
3 files changed
tree: b23a23f61aac573451ea8a51ad9ffd9aa2f08f41
  1. .cargo/
  2. .devcontainer/
  3. .github/
  4. .idea/
  5. crates/
  6. scripts/
  7. website/
  8. .asf.yaml
  9. .gitattributes
  10. .gitignore
  11. .licenserc.yaml
  12. .typos.toml
  13. Cargo.toml
  14. CHANGELOG.md
  15. CONTRIBUTING.md
  16. deny.toml
  17. LICENSE
  18. Makefile
  19. NOTICE
  20. README.md
  21. rust-toolchain.toml
  22. rustfmt.toml
README.md

Apache Iceberg Rust

Native Rust implementation of Apache Iceberg.

Roadmap

Catalog

Catalog TypeStatus
RestDone
HiveDone
SqlIn Progress
GlueDone
DynamoDBNot Started

FileIO

FileIO TypeStatus
S3Done
Local FileDone
GCSNot Started
HDFSNot Started

Our FileIO is powered by Apache OpenDAL, so it would be quite easy to expand to other service.

Table API

Reader

FeatureStatus
File based task planningDone
Size based task planningNot started
Filter pushdown(manifest evaluation, partition prunning)In Progress
Apply deletions, including equality and position deletionsNot started
Read into arrow record batchIn Progress
Parquet file supportDone
ORC file supportNot started

Writer

FeatureStatus
Data writerNot started
Equality deletion writerNot started
Position deletion writerNot started
Partitioned writerNot started
Upsert writerNot started
Parquet file supportNot started
ORC file supportNot started

Transaction

FeatureStatus
Schema evolutionNot started
Update partition specNot started
Update propertiesNot started
Replace sort orderNot started
Update locationNot started
Append filesNot started
Rewrite filesNot started
Rewrite manifestsNot started
Overwrite filesNot started
Row level updatesNot started
Replace partitionsNot started
Snapshot managementNot started

Integrations

We will add integrations with other rust based data systems, such as polars, datafusion, etc.

Contribute

Iceberg is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.

License

Licensed under the Apache License, Version 2.0