layout: doc_page

Release Process For Java Components

NOTES:

  • This process covers major and minor releases only. Bug-fix releases, which increment the third digit, are performed on a A.B.X branch and not on master, but otherwise are similar.
  • Some of these operations can be performed either on the Command-Line or in your IDE, whatever you prefer.

Preparation

  • Confirm correctness for

    • LICENSE
    • NOTICE -- check for copyright dates
    • README.md
    • .travis.yml
    • .gitignore
    • pom.xml

  • From Command Line or IDE:

    • Run Unit tests
    • Run Code Coverage > 90%
    • Run SpotBugs checks (is it properly configured?)
    • Run Checkstyle (is it properly configured?)
    • Confirm that all temporary branches are checked into master and/or deleted, both local and remote.
    • Confirm any new bug fixes have corresponding tests

  • From Command Line at Component root:

    • Confirm GPG is running (check this every time you open a new Terminal):
      • $ env | grep GPG # you should see something like: GPG_TTY=/dev/ttys000
      • To start GPG if GPG Agent is not running:
        • $ eval $(gpg-agent --daemon)
    • Confirm GitHub repository is current and git status is clean:
      • $ git status
        • “nothing to commit, working tree clean”
    • At major version releases, search for deprecated code and remove at Major Versions only.
      • $ find . -name “*.java” -type f -print | xargs grep -i -n -s -A0 “deprecated”
    • Check Maven Versions:
      • $ mvn versions:display-plugin-updates
    • Maven Tests:
      • $ mvn apache-rat:check
      • $ mvn clean test
      • $ mvn clean test -P strict
      • $ mvn clean javadoc:javadoc
      • $ mvn clean install -DskipTests=true
      • Check that the /target/ directory has 5 jars:
        • -javadoc.jar
        • -sources.jar
        • -test-sources.jar
        • -tests.jar
        • -.jar
      • Check your local Maven repository
        • ~/.m2/repository/org/apache/datasketches/datasketches-<component>/A.B.0-SNAPSHOT/
        • It should have 5 new jars and a .pom file.

Create Permanent Release Branch & POM Version Preparation

  • Assume current master POM version = A.B.0-SNAPSHOT
  • From IDE or Command Line:
    • Switch from Master to new Permanent Branch: “A.B.X”
    • Edit pom.xml version to A.B.0 (remove -SNAPSHOT, do not change A or B)
    • Commit the change. DO NOT PUSH!
    • Create Annotated TAG: A.B.0-RC1 (or RCn)
    • Write down the Git hash : example: 40c6f4f
    • Now Push Branch “A.B.X” with edited pom.xml to origin
    • DO NOT MERGE THIS PERMANENT BRANCH INTO MASTER
  • From IDE or Command-line:
    • Do explicit push of tags on branch “A.B.X” to origin:
      • $ git push origin --tags
  • From a web browser at origin web site: github.com/apache/datasketches-<component>
    • Select the A.B.X branch
    • Confirm that the tag: A.B.0-RC1 exists and that the tag is on the latest commit and with the correct Git hash.
    • DO NOT CREATE PR OR MERGE THIS PERMANENT BRANCH INTO MASTER
  • From IDE or Command Line:
    • Confirm that the tag A.B.0-RC1 and the branch A.B.X, and HEAD coincide with the correct Git hash.
    • Confirm that there are no unstaged or staged changes.
    • Return to master branch
    • Edit master pom.xml to A'.B'.0-SNAPSHOT where A' or B' will be incremented by 1. (Bug fix releases will change the 3rd digit)
    • Commit and Push this change to origin/master with the comment “Release Process: Change pom version to A'.B'.0-SNAPSHOT.”
    • Return to release branch A.B.X
    • You may minimize your IDE, pointing at the release branch.

Create and/or Checkout Local dist/dev directories on your system

  • If you have not already, on your system create the two directory structures that mirror the dist.apache.org/repos/ directories:
    • mkdir dist/dev/datasketches/
    • mkdir dist/release/datasketches/
  • Checkout both “dev” and “release” directories

Create the Candidate Apache Release Distribution on dist/dev

Create primary zip files & signatures

  • You will need the following arguments:
    • Absolute path of target project.basedir on your system
    • Project.artifactId : datasketches-<component> where component is e.g., java, pig, hive,...
    • GitHub Tag: A.B.0-RC1 (or RCn)
    • Have your GPG passphrase handy -- you have only a few seconds to enter it!
  • Start a new terminal in the above dist/dev/datasketches/scripts directory on your system:
    • Confirm GPG is running: $ env | grep GPG
      • If not: $ eval $(gpg-agent --daemon)
    • Run something like:
      • $ ./bashDeployToDist.sh /Users/<name>/dev/git/Apache/datasketches-<component> datasketches-<component> A.B.0-RC1
      • Follow the instructions.
      • NOTE: if you get the error “gpg: signing failed: No pinentry”:
        • open .gnupg/gpg-agent.conf
        • change to: pinentry-program /usr/local/bin/pinentry-tty
        • reload the gpg agent in the terminal: gpg-connect-agent reloadagent /bye
        • restart the ./bashDeployToDist script
      • Close the terminal
  • Check and grab the web URL ~ https://dist.apache.org/repos/dist/dev/datasketches/<component>/A.B.0-RC1/
    • There should be 3 files: *-src.zip, *-src.zip.asc, *-src.zip.sha512

Java: Push Jars to Nexus (Maven Central) Staging

  • Return to original terminal at the project.basedir
  • If starting new terminal make sure GPG is running: $ env | grep GPG
    • If not: $ eval $(gpg-agent --daemon)
  • $ git status # make sure you are still on the release branch: _A.B.X
  • TRIAL-RUN:
    • $ mvn clean install -Pnexus-jars -DskipTests=true
      • Check that jars & pom have .asc signatures
  • DEPLOY
    • $ mvn clean deploy -Pnexus-jars -DskipTests=true
      • Login to repository.apache.org / Staging Repositories for orgapachedatasketches-XXXX
      • Click Content and search to the end. Each jar & pom should have .asc, .md5, .sha1 signatures
      • [CLOSE] the Staging Repository with a comment: “<component> A.B.0”
      • Confirm its existance under Repositories/Staging web-site : org/apache/datasketches-<component>/A.B.0
      • Grab its URL while there. You will need it for the Vote Letter.
      • Check your local Maven repository
        • ~/.m2/repository/org/apache/datasketches/datasketches-<component>/A.B.0/
        • It should have 5 new jars and a .pom file each with .asc, .md5, and .sha1 signatures

Create Copy of External Artifact Distributions

Prepare & Send [VOTE] Letter to dev@

  • See VoteTemplates directory for a recent example
  • If vote is not successful, fix the problem and repeat above steps.
  • After a successful vote return to this point and continue ...

Prepare & Send [VOTE-RESULT] Letter to dev@

  • See VoteTemplates directory for a recent example
  • Declare that the vote is closed.
  • Summarize vote results

Move files from dev/staging to release

Move primary zip files dist/dev to dist/release

  • In local dist/dev/datasketches/
  • In local dist/release/datasketches/
    • Open Terminal #2
    • Using local file system copy files
      • From ... /dist/dev/datasketches/<component>/version-RCnn/*
      • To ... /dist/release/datasketches/<component>/version (no RCnn)/*
    • Using Terminal #2 at ... /dist/release/datasketches directory:
      • svn add . --force
      • svn ci -m “Release A.B.0”
      • Remove the prior release...
      • svn remove <component>/X.Y.0
      • svn ci -m “Remove Prior release”
      • svn status # should be empty
    • Using local file system
      • Delete the prior X.Y.0 directory

Move External Artifact Distributions dist/dev to dist/release

Java: Release Jars on Nexus Staging

  • On Nexus repository.apache.org click on Staging Repositories
  • Select “orgapachedatasketches-XXXX” (If more than one make sure you select the right one!)
  • At the top of the window, select “Release”
  • Confirm that the attributes have moved to the “Releases” repository under “Repositories”
    • Browse to *Releases/org/apache/datasketches/...

Create & Document Release Tag on GitHub

  • Open your IDE and switch to the recently created Release Branch A.B.X
  • Find the recently created A.B.0-RCn tag in that branch
  • At that same GitHub ID hash, create a new tag A.B.0 (without the RCn).
  • From the Command Line: Push the new tag to origin:
    • $ git push origin --tags
  • On the GitHub component site document the release

Update Website Downloads.md “Latest Source Zip Files” Table

  • This script assumes that the remote .../dist/release/datasketches/... directories are up-to-date with no old releases.
  • Start a new terminal in the ../dist/dev/datasketches/scripts directory on your system:
  • Make sure you local website directory is pointing to master and up-to-date.
  • Run the following with the argument specifying the location of your local website directory:
    • $ ./createDownloadsInclude.sh /Users/<name>/ ... /datasketches-website
  • When this is done, be sure to commit the changes to the website.

Update Javadocs (or Equivalent) on Website

Update Website Documentation (if new functionality)

Prepare Announce Letter to dev@

  • ASF requests that you wait 24 hours to publish Announce letter to allow the propagation to mirrors.
  • Use recent template
  • Summarize vote results

Update These Instructions

  • If you have updated this file or any of the scripts, please check it in using SVN using your local dist/dev directory copy:
    • $ svn status
    • $ svn add . --force # if adding a file for the first time
    • $ svn ci -m “update Release Steps”