DataFusion Documentation

This folder contains the source content of the Python API. This is published to https://datafusion.apache.org/python by a GitHub action when changes are merged to the main branch.

Dependencies

It's recommended to install build dependencies and build the documentation inside a Python venv using uv.

To prepare building the documentation run the following on the root level of the project:

# Set up a virtual environment with the documentation dependencies
uv sync --dev --group docs --no-install-package datafusion

Build & Preview

Run the provided script to build the HTML pages.

# Build the repository
uv run --no-project maturin develop --uv
# Build the documentation
uv run --no-project docs/build.sh

The HTML will be generated into a build directory in docs.

Preview the site on Linux by running this command.

firefox docs/build/html/index.html

Release Process

This documentation is hosted at https://datafusion.apache.org/python

When the PR is merged to the main branch of the DataFusion repository, a github workflow which:

1. Builds the html content
2. Pushes the html content to the asf-site branch in this repository.

The Apache Software Foundation provides https://arrow.apache.org/, which serves content based on the configuration in .asf.yaml, which specifies the target as https://datafusion.apache.org/python.