Contributing to hudi-rs

Welcome to the Apache Hudi community! We appreciate your interest in contributing to this open-source data lake platform. This guide will walk you through the process of making your first contribution.

Starter issues

If you are new to the project, we recommend starting with issues listed in https://github.com/apache/hudi-rs/contribute.

File an issue

Testing and reporting bugs are also valueable contributions. Please follow the issue template to file bug reports.

Issue tracking

All issues tagged for a release can be found in the corresponding milestone page, see https://github.com/apache/hudi-rs/milestones.

Features, bugs, and p0 issues that are targeting the next release can be found in this project view. Pull requests won't be tracked in the project view, instead, they will be linked to the corresponding issues.

Prepare for development

  • Install Rust, e.g. as described here
  • Install uv, the fast Python package manager, as described here
  • Have a compatible Python version installed (check python/pyproject.toml for current requirement)

Pre-commit hooks (recommended)

This project provides pre-commit hooks to catch issues locally before they reach CI. After setting up the virtual environment:

make setup-pre-commit

The hooks are split into two stages for a fast development cycle:

On every commit (~5 seconds):

  • File hygiene: trailing whitespace, end-of-file fixer, YAML validation, large file check, merge conflict markers
  • Rust formatting: auto-formats via cargo fmt (same as make format-rust)
  • Python formatting: auto-formats via ruff format (same as make format-python)

On every push (thorough, ~1-5 minutes):

  • Rust checks: cargo clippy and format verification (same as make check-rust)
  • Python checks: ruff linting and mypy type checking (same as make check-python)

These are the same checks that run in CI. To skip hooks temporarily: git commit --no-verify or git push --no-verify.

Commonly used dev commands

For most of the time, use dev commands specified in the Makefile.

[!NOTE] This project uses uv as the Python package manager for faster dependency resolution and installation. All Python-related commands in the Makefile have been configured to use uv.

To setup python virtual env, run

make setup-venv

[!NOTE] This will use uv to set up the virtual environment in .venv/. Activate the virtual environment by running source .venv/bin/activate for example.

Once a virtual environment is activated, build the project for development by

make develop

This will install hudi dependency built from your local repo to the virtual env.

Run tests locally

For Rust,

# For all tests
make test-rust
# or
cargo test --workspace

# For all tests in a crate / package
cargo test -p hudi-core

# For a specific test case
cargo test -p hudi-core table::tests::hudi_table_get_schema

For Python,

# For all tests
make test-python
# or
pytest -s python/tests

# For a specific test case
pytest python/tests/test_table_read.py -s -k "test_read_table_has_correct_schema"

Before creating a pull request

Run the below command and fix issues if any:

make format check test

Create a pull request

When submitting a pull request, please follow these guidelines:

  1. Title Format: The pull request title must follow the format outlined in the conventional commits spec. This is a standardized format for commit messages, and also allows us to auto-generate change logs and release notes. Since only the main branch requires this format, and we always squash commits and then merge the PR, incremental commits' messages do not need to conform to it.
  2. Line Count: A general guideline is to keep the PR's diff, i.e., max(added lines, deleted lines), less than 1000 lines. Keeping PRs concise makes it easier for reviewers to thoroughly examine changes without experiencing fatigue. If your changes exceed this limit, consider breaking them down into smaller, logical PRs that address specific aspects of the feature or bug fix.
  3. Coverage Requirements: All new features and bug fixes must include appropriate unit tests to ensure functionality and prevent regressions. Tests should cover both typical use cases and edge cases. Ensure that new tests pass locally before submitting the PR.
  4. Code Comments: Properly designed APIs and code should be self-explanatory and make in-code comments redundant. In case that complex logic or non-obvious implementations are absolutely unavoidable, please add comments to explain the code's purpose and behavior.

Code coverage

We use codecov to generate code coverage report and enforce code coverage rate. See codecov.yml for the configuration.

Learning

To help with contributing to the project, please explore Hudi's documentation for further learning.

Code of Conduct

We expect all community members to follow our Code of Conduct.