Apache OpenDAL™ YinYang: a cross-platform filesystem foundation.

Clone this repo:
  1. da2f7b7 feat: transfer directories through indexed transactions (#46) by Xuanwo · 5 hours ago main
  2. 9c3b2bc feat: publish indexed filesystem transactions (#45) by Xuanwo · 5 hours ago
  3. 99c937e feat: add authenticated content preparation (#44) by Xuanwo · 22 hours ago
  4. e158d04 docs: specify transactional filesystem publication modes (#47) by Xuanwo · 26 hours ago
  5. 1f00d5d feat(core): persist YinYang through OpenDAL (#43) by Xuanwo · 2 weeks ago

Apache OpenDAL™ YinYang

Build Status Latest Version Crate Downloads MSRV 1.91.0 Apache 2.0 licensed chat

Apache OpenDAL™ YinYang is a cross-platform filesystem foundation.

[!IMPORTANT] Status: active redesign

We are actively working on the design of the next Apache OpenDAL™ YinYang release under RFC-0016. The core provides experimental indexed object-storage transactions, retained snapshots, and authenticated range-verifiable content. The yy CLI publishes and restores directories. Metadata-service publication, Mount, and bidirectional Sync are not implemented yet.

Publish and restore a directory

Build the experimental CLI with cargo build --bin yy. It currently connects to Amazon S3 or MinIO. Create a bucket first, then configure OpenDAL through YINYANG_S3_* environment variables:

export YINYANG_S3_BUCKET=my-bucket
export YINYANG_S3_REGION=us-east-1
export YINYANG_S3_ROOT=my-filesystem
# For MinIO, also set YINYANG_S3_ENDPOINT and YINYANG_STORAGE_PROFILE=minio.
# Supply credentials using YINYANG_S3_ACCESS_KEY_ID and
# YINYANG_S3_SECRET_ACCESS_KEY, or the OpenDAL AWS credential chain.

target/debug/yy create
target/debug/yy publish ./source
target/debug/yy status
target/debug/yy restore ./new-destination

Keep the source directory unchanged during publication. Publishing a later complete snapshot requires publish ./source --replace: remote-only paths are removed. Publication prints a commit UUID; retain it and query yy receipt UUID if the result is uncertain. A missing receipt does not prove failure while an attempt may still complete. --commit-id selects an identity for a new request, not permission to replan an old one. The library exposes a frozen transaction for exact retries without rescanning or reuploading. Concurrent changes within the replaced scope return a conflict without silently erasing them.

Restore requires a destination that does not exist. It verifies and installs files individually without overwriting anything. An interrupted restore can leave a partial directory; inspect it and restore into a new destination to retry. This is one-shot transfer, not a background or bidirectional Sync engine. See the directory transfer contract for supported metadata, failure semantics, and limitations.

Previous releases

The backup branch preserves the implementation released as Apache OpenDAL™ File System. Its Cargo package and command-line executable are both named ofs. This historical implementation predates the YinYang redesign and remains available for reference.

Development

See [CONTRIBUTING.md] for setup instructions. Run the repository checks with:

cargo x lint
cargo x check
cargo x test
cargo x licenses

Branding

Apache OpenDAL™ is the ASF project and umbrella brand. Apache OpenDAL™ YinYang software is a product developed by the Apache OpenDAL project.

Use the full product name, Apache OpenDAL™ YinYang, in titles and first prominent references. Later references may use YinYang when the relationship to the Apache OpenDAL project and the ASF remains clear.

Use yinyang only for the Cargo package and Rust crate, and use yy only for the command-line executable and commands.

For more details, see the Apache Product Name Usage Guide.

License and Trademarks

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Apache OpenDAL, OpenDAL, Apache OpenDAL YinYang, YinYang, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.