| <!--- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| --> |
| |
| # Release Process |
| |
| This guide is for maintainers creating release candidates of Apache |
| DataFusion Java and running the release process. |
| |
| ## Release Prerequisites |
| |
| ### Add a git remote for the `apache` repo |
| |
| The instructions below assume the upstream git repo |
| `git@github.com:apache/datafusion-java.git` is configured as the remote |
| named `apache`: |
| |
| ```shell |
| git remote add apache git@github.com:apache/datafusion-java.git |
| ``` |
| |
| ### Add your GPG public key to the SVN `KEYS` file |
| |
| If you will be releasing the final tarball, your GPG public key must be |
| present in: |
| |
| - https://dist.apache.org/repos/dist/dev/datafusion/KEYS |
| - https://dist.apache.org/repos/dist/release/datafusion/KEYS |
| |
| See https://infra.apache.org/release-signing.html#generate for generating |
| keys. |
| |
| Committers can add signing keys via Subversion using their ASF account: |
| |
| ```shell |
| svn co https://dist.apache.org/repos/dist/dev/datafusion |
| cd datafusion |
| (gpg --list-sigs "Your Name" && gpg --armor --export "Your Name") >> KEYS |
| svn commit KEYS -m "Add key for Your Name" |
| ``` |
| |
| ## Release Process: Step by Step |
| |
| As part of the Apache governance model, official releases consist of |
| signed source tarballs approved by the PMC. |
| |
| ### 1. Pick a Release Candidate (RC) number |
| |
| Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, and |
| so on. |
| |
| ### 2. Create the Git tag for the Release Candidate |
| |
| Release tags look like `0.1.0`, and release candidate tags look like |
| `0.1.0-rc1`. |
| |
| ```shell |
| git fetch apache |
| git tag 0.1.0-rc1 apache/main |
| git push apache 0.1.0-rc1 |
| ``` |
| |
| ### 3. Create, sign, and upload the tarball |
| |
| Run `create-tarball.sh` with the version and RC number: |
| |
| ```shell |
| ./dev/release/create-tarball.sh 0.1.0 1 |
| ``` |
| |
| The script: |
| |
| 1. Creates and uploads the release candidate artifacts to the |
| [`dist/dev/datafusion`](https://dist.apache.org/repos/dist/dev/datafusion) |
| location on the Apache SVN distribution server, under |
| `apache-datafusion-java-<version>-rc<rc>/`. |
| 2. Prints an email template to send to `dev@datafusion.apache.org` for |
| the release vote. |
| |
| ### 4. Vote on the Release Candidate |
| |
| Send the email printed by the script to `dev@datafusion.apache.org`. |
| |
| The vote must remain open for at least 72 hours and requires at least |
| three PMC `+1` votes and no `-1` votes to pass. |
| |
| ### Verifying Release Candidates |
| |
| `dev/release/verify-release-candidate.sh` downloads the candidate |
| tarball, verifies its signature and checksums, and builds and tests the |
| source distribution: |
| |
| ```shell |
| ./dev/release/verify-release-candidate.sh 0.1.0 1 |
| ``` |
| |
| The script requires `curl`, `git`, `gpg`, a C toolchain, `java`, and |
| either `shasum` or `sha256sum`/`sha512sum` to be on the `PATH`. It will |
| install a temporary Rust toolchain into the sandbox directory. |
| |
| ### 5. If the Vote Passes: Promote the Tarball |
| |
| Move artifacts from the `dev` SVN area to the `release` area using |
| `release-tarball.sh`: |
| |
| ```shell |
| ./dev/release/release-tarball.sh 0.1.0 1 |
| ``` |
| |
| ### 6. Create the Release Git Tag |
| |
| Tag the same release candidate commit with the final release tag: |
| |
| ```shell |
| git checkout 0.1.0-rc1 |
| git tag 0.1.0 |
| git push apache 0.1.0 |
| ``` |
| |
| ### 7. Add the Release to Apache Reporter |
| |
| Add the release at https://reporter.apache.org/addrelease.html?datafusion |
| so it appears in the project's board report. |
| |
| ### 8. Delete Old RCs and Releases |
| |
| See the ASF documentation on [when to |
| archive](https://www.apache.org/legal/release-policy.html#when-to-archive). |
| |
| Release candidates should be deleted once the release is published: |
| |
| ```shell |
| svn ls https://dist.apache.org/repos/dist/dev/datafusion | grep datafusion-java |
| svn delete -m "delete old DataFusion Java RC" \ |
| https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-java-0.1.0-rc1/ |
| ``` |
| |
| Only the latest release should remain in the `release` SVN area. Delete |
| older releases after publishing a new one: |
| |
| ```shell |
| svn ls https://dist.apache.org/repos/dist/release/datafusion | grep datafusion-java |
| svn delete -m "delete old DataFusion Java release" \ |
| https://dist.apache.org/repos/dist/release/datafusion/datafusion-java-0.1.0 |
| ``` |
| |
| ## Binary Release: Multi-Platform JAR |
| |
| Source tarballs are the official Apache release artifact, but consumers |
| also expect a published JAR on Maven Central that bundles native libs |
| for the common platforms. This section covers building that JAR. |
| |
| ### Prerequisites (release manager machine) |
| |
| - macOS host (Apple Silicon or Intel) |
| - Docker Desktop running with BuildKit enabled |
| - Java 17+ |
| - Rust toolchain via rustup |
| - `gpg` configured with a key listed in the ASF KEYS file |
| - `xmllint` on `PATH` (pre-installed on macOS; `libxml2-utils` on Debian/Ubuntu) |
| |
| ### Build the multi-platform JAR |
| |
| `build-release.sh` clones the repo into two Linux Docker containers |
| (one for `linux/amd64`, one for `linux/arm64`), builds the native |
| `.so` libraries inside each, then builds the two macOS `.dylib` |
| libraries directly on the host. All four libraries are placed in the |
| JAR's resource tree at |
| `org/apache/datafusion/<os>/<arch>/lib<datafusion_jni>.<ext>`, and the |
| JAR is installed into a temporary local Maven repository. |
| |
| ```shell |
| ./dev/release/build-release.sh |
| ``` |
| |
| The script prints the local Maven repo path at the end. Inspect the JAR |
| to verify all four native libraries are bundled: |
| |
| ```shell |
| unzip -l "$JAR" | grep org/apache/datafusion/ |
| ``` |
| |
| ### Publish to Apache Nexus staging |
| |
| Once the local Maven repo from `build-release.sh` looks correct, sign |
| and upload to Apache Nexus staging using `publish-to-maven.sh`: |
| |
| ```shell |
| ./dev/release/publish-to-maven.sh -u <asf-username> -r <local-maven-repo-path> |
| ``` |
| |
| The script prompts for the ASF password and GPG passphrase, creates a |
| staging repository on `repository.apache.org`, signs every artifact, |
| uploads it, and closes the staging repository. Verify in the Nexus UI |
| that the staged artifacts look correct before promoting to release. |