Google Git
Sign in
apache/datafusion-java/refs/heads/main/./dev/release
tree: 0ffc0c0c5294545b15e581290b0a6caaadcbce24
  1. datafusion-java-rm/
  2. build-release.sh
  3. check-rat-report.py
  4. create-tarball.sh
  5. generate-changelog.py
  6. publish-to-maven.sh
  7. rat_exclude_files.txt
  8. README.md
  9. release-tarball.sh
  10. requirements.txt
  11. verify-release-candidate.sh
dev/release/README.md

Release Process

This guide is for maintainers creating release candidates of Apache DataFusion Java and running the release process.

Release Prerequisites

Add a git remote for the apache repo

The instructions below assume the upstream git repo git@github.com:apache/datafusion-java.git is configured as the remote named apache:

git remote add apache git@github.com:apache/datafusion-java.git

Add your GPG public key to the SVN KEYS file

If you will be releasing the final tarball, your GPG public key must be present in:

See https://infra.apache.org/release-signing.html#generate for generating keys.

Committers can add signing keys via Subversion using their ASF account:

svn co https://dist.apache.org/repos/dist/dev/datafusion
cd datafusion
(gpg --list-sigs "Your Name" && gpg --armor --export "Your Name") >> KEYS
svn commit KEYS -m "Add key for Your Name"

Release Process: Step by Step

As part of the Apache governance model, official releases consist of signed source tarballs approved by the PMC.

1. Pick a Release Candidate (RC) number

Pick numbers in sequential order, with 1 for rc1, 2 for rc2, and so on.

2. Create the Git tag for the Release Candidate

Release tags look like 0.1.0, and release candidate tags look like 0.1.0-rc1.

git fetch apache
git tag 0.1.0-rc1 apache/main
git push apache 0.1.0-rc1

3. Create, sign, and upload the tarball

Run create-tarball.sh with the version and RC number:

./dev/release/create-tarball.sh 0.1.0 1

The script:

  1. Creates and uploads the release candidate artifacts to the dist/dev/datafusion location on the Apache SVN distribution server, under apache-datafusion-java-<version>-rc<rc>/.
  2. Prints an email template to send to dev@datafusion.apache.org for the release vote.

4. Vote on the Release Candidate

Send the email printed by the script to dev@datafusion.apache.org.

The vote must remain open for at least 72 hours and requires at least three PMC +1 votes and no -1 votes to pass.

Verifying Release Candidates

dev/release/verify-release-candidate.sh downloads the candidate tarball, verifies its signature and checksums, and builds and tests the source distribution:

./dev/release/verify-release-candidate.sh 0.1.0 1

The script requires curl, git, gpg, a C toolchain, java, and either shasum or sha256sum/sha512sum to be on the PATH. It will install a temporary Rust toolchain into the sandbox directory.

5. If the Vote Passes: Promote the Tarball

Move artifacts from the dev SVN area to the release area using release-tarball.sh:

./dev/release/release-tarball.sh 0.1.0 1

6. Create the Release Git Tag

Tag the same release candidate commit with the final release tag:

git checkout 0.1.0-rc1
git tag 0.1.0
git push apache 0.1.0

7. Add the Release to Apache Reporter

Add the release at https://reporter.apache.org/addrelease.html?datafusion so it appears in the project's board report.

8. Delete Old RCs and Releases

See the ASF documentation on when to archive.

Release candidates should be deleted once the release is published:

svn ls https://dist.apache.org/repos/dist/dev/datafusion | grep datafusion-java
svn delete -m "delete old DataFusion Java RC" \
  https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-java-0.1.0-rc1/

Only the latest release should remain in the release SVN area. Delete older releases after publishing a new one:

svn ls https://dist.apache.org/repos/dist/release/datafusion | grep datafusion-java
svn delete -m "delete old DataFusion Java release" \
  https://dist.apache.org/repos/dist/release/datafusion/datafusion-java-0.1.0

Binary Release: Multi-Platform JAR

Source tarballs are the official Apache release artifact, but consumers also expect a published JAR on Maven Central that bundles native libs for the common platforms. This section covers building that JAR.

Prerequisites (release manager machine)

  • macOS host (Apple Silicon or Intel)
  • Docker Desktop running with BuildKit enabled
  • Java 17+
  • Rust toolchain via rustup
  • gpg configured with a key listed in the ASF KEYS file
  • xmllint on PATH (pre-installed on macOS; libxml2-utils on Debian/Ubuntu)

Build the multi-platform JAR

build-release.sh clones the repo into two Linux Docker containers (one for linux/amd64, one for linux/arm64), builds the native .so libraries inside each, then builds the two macOS .dylib libraries directly on the host. All four libraries are placed in the JAR's resource tree at org/apache/datafusion/<os>/<arch>/lib<datafusion_jni>.<ext>, and the JAR is installed into a temporary local Maven repository.

./dev/release/build-release.sh

The script prints the local Maven repo path at the end. Inspect the JAR to verify all four native libraries are bundled:

unzip -l "$JAR" | grep org/apache/datafusion/

Publish to Apache Nexus staging

Once the local Maven repo from build-release.sh looks correct, sign and upload to Apache Nexus staging using publish-to-maven.sh:

./dev/release/publish-to-maven.sh -u <asf-username> -r <local-maven-repo-path>

The script prompts for the ASF password and GPG passphrase, creates a staging repository on repository.apache.org, signs every artifact, uploads it, and closes the staging repository. Verify in the Nexus UI that the staged artifacts look correct before promoting to release.