PyPaimon Rust

This project builds the Rust-powered core for PyPaimon while also providing DataFusion integration for querying Paimon tables.

Install via PyPI:

pip install pypaimon-rust

If you want to use the native Python DataFusion SessionContext, install datafusion as well.

Query Paimon Tables with DataFusion

The recommended way to query Paimon tables is through SQLContext, which supports multi-catalog registration, DDL, DML, and all Paimon-specific SQL extensions:

from pypaimon_rust.datafusion import SQLContext

ctx = SQLContext()
ctx.register_catalog("paimon", {
    "warehouse": "/path/to/warehouse",
})

# DDL
ctx.sql("CREATE SCHEMA paimon.my_db")
ctx.sql("CREATE TABLE paimon.my_db.users (id INT, name STRING, PRIMARY KEY (id))")

# DML
ctx.sql("INSERT INTO paimon.my_db.users VALUES (1, 'alice'), (2, 'bob')")

# Query tables via SQL (catalog.database.table)
batches = ctx.sql("SELECT * FROM paimon.my_db.users")

Alternatively, if you want to use the native Python DataFusion SessionContext, install datafusion and register a PaimonCatalog:

from datafusion import SessionContext
from pypaimon_rust.datafusion import PaimonCatalog

catalog = PaimonCatalog({
    "warehouse": "/path/to/warehouse",
})

ctx = SessionContext()
ctx.register_catalog_provider("paimon", catalog)

# Query tables via SQL (catalog.database.table)
df = ctx.sql("SELECT * FROM paimon.default.my_table LIMIT 10")
df.show()

REST Catalog

from datafusion import SessionContext
from pypaimon_rust.datafusion import PaimonCatalog

catalog = PaimonCatalog({
    "metastore": "rest",
    "uri": "http://localhost:8080",
    "warehouse": "my_warehouse",
})

ctx = SessionContext()
ctx.register_catalog_provider("paimon", catalog)

Time travel queries are not supported in the Python binding at this time.