This documentation explains the release process for Apache DataFusion Comet. Some preparation tasks can be performed by any contributor, while certain release tasks can only be performed by a DataFusion Project Management Committee (PMC) member.
The following is a quick-reference checklist for the full release process. See the detailed sections below for instructions on each step.
.asf.yamlBefore starting the release process, review the user guide to ensure it accurately reflects the current state of the project:
It is also recommended to run benchmarks (such as TPC-H and TPC-DS) comparing performance against the previous release to check for regressions. See the Comet Benchmarking Guide for instructions.
These are tasks where agentic coding tools can be particularly helpful — for example, scanning the codebase for newly registered expressions and cross-referencing them against the documented list, or generating test queries to verify expression support status.
Any issues found should be addressed before creating the release branch.
This part of the process can be performed by any committer.
Here are the steps, using the 0.13.0 release as an example.
This document assumes that GitHub remotes are set up as follows:
$ git remote -v apache git@github.com:apache/datafusion-comet.git (fetch) apache git@github.com:apache/datafusion-comet.git (push) origin git@github.com:yourgithubid/datafusion-comet.git (fetch) origin git@github.com:yourgithubid/datafusion-comet.git (push)
Create a release branch from the latest commit in main and push to the apache repo:
git fetch apache git checkout main git reset --hard apache/main git checkout -b branch-0.13 git push apache branch-0.13
Creating the branch is the only direct push to it. After protecting the branch (next step), all later changes to the release branch (documentation, version bump, changelog) go through pull requests targeting it.
Add the new branch to protected_branches in .asf.yaml so it requires a pull request and review, the same as main. ASF applies .asf.yaml from the default branch, so this is a PR against main:
protected_branches: main: required_pull_request_reviews: required_approving_review_count: 1 branch-0.13: required_pull_request_reviews: required_approving_review_count: 1
All release branches stay protected, including older ones, so released code cannot be pushed to directly.
The docs on main contain only template markers; CI fills them at publish time. A release branch instead commits the generated content so the archived docs for the release render real tables. Run the script to generate the config reference, refresh the Implementation column of the Spark Expression Support page, and produce the per-Spark-version compatibility pages, freezing them onto the branch:
./dev/generate-release-docs.sh git add docs/source/user-guide/latest/ git commit -m "Generate docs for 0.13.0 release"
The script runs one Maven build per supported Spark version, so it takes a while. Open a PR with this commit targeting the release branch.
Open a PR targeting the release branch that changes the Maven version from 0.13.0-SNAPSHOT to 0.13.0 in:
pom.xml files: pom.xml, common/pom.xml, spark/pom.xml, and spark-integration/pom.xml<comet.version> property in the Spark test diffs under dev/diffscomet version in the Iceberg test diffs under dev/diffs/iceberg, which use a Gradle version catalog entry (comet = "0.13.0") rather than a Maven propertyThere is no need to update the Rust crate versions because they will already be 0.13.0.
Once the version is bumped, verify that no references to the old version remain:
git grep -n '0.13.0-SNAPSHOT' -- . ':!docs/source/changelog'
Any hit outside the change log is a place that will break on the release branch. Note that dropping the -SNAPSHOT qualifier changes the built artifact file names, so anything that locates the jar by glob must match a bare version too. Prefer patterns such as comet-spark-spark3.5_2.12-*.jar over comet-spark-spark3.5_2.12-*-SNAPSHOT.jar, and prefer resolving the jar by glob over hardcoding a version. The release branch runs the same CI workflows as main, including the PyArrow UDF tests, so a -SNAPSHOT-only glob in a test harness or script fails only after the release branch is cut.
Create a PR against the main branch to prepare for developing the next release:
0.14.0.0.14.0-SNAPSHOT in the same set of files listed above (the pom.xml files, the Spark test diffs under dev/diffs, and the Iceberg test diffs under dev/diffs/iceberg).Generate a change log to cover changes between the previous release and the release branch HEAD by running the provided dev/release/generate-changelog.py.
It is recommended that you set up a virtual Python environment and then install the dependencies:
cd dev/release python3 -m venv venv source venv/bin/activate pip3 install -r requirements.txt
To generate the changelog, set the GITHUB_TOKEN environment variable to a valid token and then run the script providing two commit ids or tags followed by the version number of the release being created. The following example generates a change log of all changes between the previous version and the current release branch HEAD revision.
export GITHUB_TOKEN=<your-token-here> python3 generate-changelog.py 0.12.0 HEAD 0.13.0 > ../../docs/source/changelog/0.13.0.md
Open a PR adding this change log targeting the release branch. Generate it late, once backports to the release branch are complete; if more changes land on the release branch before the release candidate is tagged, regenerate it and update the PR. After the release is approved and tagged, open a separate PR to bring the same change log file into main.
The common/pom.xml resource configuration unconditionally bundles native/target/{x86_64,aarch64}-apple-darwin/release/libcomet.dylib into the common jar when those files exist on disk. Maven‘s clean removes common/target but does not touch Cargo’s native/target directory, so a stale dylib left over from a prior local make release or make release-linux on the release manager's workstation can silently end up in a release jar (see #2232 for the incident in 0.9.1).
The build-release-comet.sh script now runs cargo clean for you, but as a defensive measure, prefer running the release build from a fresh clone of the repository rather than your day-to-day working tree.
The build process requires Docker. Download the latest Docker Desktop from https://www.docker.com/products/docker-desktop/. If you have multiple docker contexts running switch to the context of the Docker Desktop. For example -
$ docker context ls NAME DESCRIPTION DOCKER ENDPOINT ERROR default Current DOCKER_HOST based configuration unix:///var/run/docker.sock desktop-linux Docker Desktop unix:///Users/parth/.docker/run/docker.sock my_custom_context * tcp://192.168.64.2:2376 $ docker context use desktop-linux
The build-release-comet.sh script will create a docker image for each architecture and use the image to build the platform specific binaries. These builder images are created every time this script is run. The script optionally allows overriding of the repository and branch to build the binaries from (Note that the local git repo is not used in the building of the binaries, but it is used to build the final uber jar).
Usage: build-release-comet.sh [options] This script builds comet native binaries inside a docker image. The image is named "comet-rm" and will be generated by this script Options are: -r [repo] : git repo (default: https://github.com/apache/datafusion-comet.git) -b [branch] : git branch (default: release) -t [tag] : tag for the spark-rm docker image to use for building (default: "latest").
Example:
cd dev/release && ./build-release-comet.sh && cd ../..
The build output is installed to a temporary local maven repository. The build script will print the name of the repository location at the end. This location will be required at the time of deploying the artifacts to a staging repository
Ensure that the Maven version update and change log have been merged to the release branch before tagging.
Tag the release branch with 0.13.0-rc1 and push to the apache repo
git fetch apache git checkout branch-0.13 git reset --hard apache/branch-0.13 git tag 0.13.0-rc1 git push apache 0.13.0-rc1
Note that pushing a release candidate tag will trigger a GitHub workflow that will build a Docker image and publish it to GitHub Container Registry at https://github.com/apache/datafusion-comet/pkgs/container/datafusion-comet
In docs directory:
docs/source/index.rst and add a new navigation menu link for the new release in the section _toc.user-guide-links-versionedbuild.sh to delete the locally cloned comet-* branch for the new release e.g. comet-0.13generate-versions.py:latest_released_version = "0.13.0" previous_versions = ["0.11.0", "0.12.0"]
Test the documentation build locally, following the instructions in docs/README.md.
Once verified, create a PR against the main branch with these documentation changes. After merging, the docs will be deployed to https://datafusion.apache.org/comet/ by the documentation publishing workflow.
Note that the download links in the installation guide will not work until the release is finalized, but having the documentation available could be useful for anyone testing out the release candidate during the voting period.
This part of the process can mostly only be performed by a PMC member.
Setting up your project in the ASF Nexus Repository from here: https://infra.apache.org/publishing-maven-artifacts.html
Set up your development environment from here: https://infra.apache.org/publishing-maven-artifacts.html
The script publish-to-maven.sh will publish the artifacts created by the build-release-comet.sh script. The artifacts will be signed using the gpg key of the release manager and uploaded to the maven staging repository.
Note that installed GPG keys can be listed with gpg --list-keys. The gpg key is a 40 character hex string.
Note: This script needs xmllint to be installed. On macOS xmllint is available by default.
On Ubuntu apt-get install -y libxml2-utils
On RedHat yum install -y xmlstarlet
./dev/release/publish-to-maven.sh -h usage: publish-to-maven.sh options Publish signed artifacts to Maven. Options -u ASF_USERNAME - Username of ASF committer account -r LOCAL_REPO - path to temporary local maven repo (created and written to by 'build-release-comet.sh') The following will be prompted for - ASF_PASSWORD - Password of ASF committer account GPG_KEY - GPG key used to sign release artifacts GPG_PASSPHRASE - Passphrase for GPG key
example
./dev/release/publish-to-maven.sh -u release_manager_asf_id -r /tmp/comet-staging-repo-VsYOX ASF Password : GPG Key (Optional): GPG Passphrase : Creating Nexus staging repository ...
In the Nexus repository UI (https://repository.apache.org/) locate and verify the artifacts in staging (https://central.sonatype.org/publish/release/#locate-and-examine-your-staging-repository).
The script closes the staging repository but does not release it. Releasing to Maven Central is a manual step performed only after the vote passes (see Publishing Maven Artifacts below).
Note that the Maven artifacts are always published under the final release version (e.g. 0.13.0), not the RC version — the -rc1 / -rc2 suffix only appears in the git tag and the source tarball in SVN. Because the script creates a new staging repository on each run, re-staging the same version for a subsequent RC is supported as long as no staging repository for that version has been released to Maven Central.
The create-tarball.sh script creates a signed source tarball and uploads it to the dev subversion repository.
Before running this script, ensure you have:
requests Python package installed (pip3 install requests)Run the create-tarball script on the release candidate tag (0.13.0-rc1):
./dev/release/create-tarball.sh 0.13.0 1
This will generate an email template for starting the vote.
Send the email that is generated in the previous step to dev@datafusion.apache.org.
The verification procedure for voters is documented in Verifying Release Candidates. Voters can also use the dev/release/verify-release-candidate.sh script to assist with verification:
./dev/release/verify-release-candidate.sh 0.13.0 1
If the vote does not pass, address the issues raised, increment the release candidate number, and repeat from the Tag the Release Candidate step. For example, the next attempt would be tagged 0.13.0-rc2.
Before staging the next RC, drop the previous RC's staging repository in the Nexus UI by selecting it and clicking “Drop”. This avoids leaving multiple closed staging repositories for the same version and prevents accidentally releasing the wrong one when the vote eventually passes. The Maven version (e.g. 0.13.0) is shared across all RCs, so each run of publish-to-maven.sh creates a new staging repository for the same GAV — only one of them should ever be released to Maven Central.
Once the vote passes, we can publish the source and binary releases.
Run the release-tarball script to move the tarball to the release subversion repository.
./dev/release/release-tarball.sh 0.13.0 1
Go to https://github.com/apache/datafusion-comet/releases and create a release for the release tag, and paste the changelog in the description.
Promote the Maven artifacts from staging to production by visiting https://repository.apache.org/#stagingRepositories and selecting the staging repository and then clicking the “release” button.
Push a release tag (0.13.0) to the apache repository.
git fetch apache git checkout 0.13.0-rc1 git tag 0.13.0 git push apache 0.13.0
Note that pushing a release tag will trigger a GitHub workflow that will build a Docker image and publish it to GitHub Container Registry at https://github.com/apache/datafusion-comet/pkgs/container/datafusion-comet
Reply to the vote thread to close the vote and announce the release. The announcement email should include:
Register the release with the Apache Reporter Service using a version such as COMET-0.13.0.
See the ASF documentation on when to archive for more information.
dev svnRelease candidates should be deleted once the release is published.
Get a list of DataFusion Comet release candidates:
svn ls https://dist.apache.org/repos/dist/dev/datafusion | grep comet
Delete a release candidate:
svn delete -m "delete old DataFusion Comet RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-comet-0.13.0-rc1/
release svnOnly the latest release should be available. Delete old releases after publishing the new release.
Get a list of DataFusion releases:
svn ls https://dist.apache.org/repos/dist/release/datafusion | grep comet
Delete a release:
svn delete -m "delete old DataFusion Comet release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-comet-0.12.0
Writing a blog post about the release is a great way to generate more interest in the project. We typically create a Google document where the community can collaborate on a blog post. Once the content is agreed then a PR can be created against the datafusion-site repository to add the blog post. Any contributor can drive this process.