core/services/hf/src/docs.md

This service will visit the Hugging Face API to access the Hugging Face File System.

Hugging Face doesn't host official HTTP API docs. Detailed HTTP request API information can be found on the huggingface_hub Source Code.

Storage Backends

This service supports two storage backends:

Git-based repositories (model, dataset, space): Files are versioned in a Git repository. Large files are stored via Xet, Hugging Face's chunk-deduplicated storage backend; writes create new commits. Supports revision for branch/commit targeting.
Object store buckets (bucket): Files are stored in a Hugging Face Bucket (not git-backed). No revisions or commits — all reads and writes use the Xet protocol directly.

Capabilities

This service can be used to:

[ ] create_dir
[x] stat
[x] read
[x] write
[x] delete
[x] list
[ ] copy
[ ] rename
[ ] presign

Configurations

repo_type: The type of the repository. One of model, dataset, space, or bucket.
repo_id: The id of the repository.
revision: The revision of the repository. Only applicable for git-based repo types (model, dataset, space).
root: Set the work directory for backend.
token: The token for accessing the repository. Required for write operations.
endpoint: The Hub base URL. Default is https://huggingface.co. Can also be set via the HF_ENDPOINT environment variable.

Refer to [HfBuilder]'s public API docs for more information.

Examples

Via Builder (Git-based dataset)

use opendal_core::Operator;
use opendal_core::Result;
use opendal_service_hf::Hf;

#[tokio::main]
async fn main() -> Result<()> {
    let builder = Hf::default()
        .repo_type("dataset")
        .repo_id("username/my-dataset")
        .revision("main")
        .root("/path/to/dir")
        .token("access_token");

    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

Via Builder (Object store bucket)

use opendal_core::Operator;
use opendal_core::Result;
use opendal_service_hf::Hf;

#[tokio::main]
async fn main() -> Result<()> {
    let builder = Hf::default()
        .repo_type("bucket")
        .repo_id("username/my-bucket")
        .token("access_token");

    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

Via URI

use opendal_core::Operator;
use opendal_core::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Git-based dataset
    let op = Operator::from_uri((
        "hf://datasets/username/my-dataset@main",
        vec![("token", "access_token")],
    ))?;

    // Object store bucket
    let op = Operator::from_uri((
        "hf://buckets/username/my-bucket",
        vec![("token", "access_token")],
    ))?;

    Ok(())
}