// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements.  See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership.  The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License.  You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied.  See the License for the
// specific language governing permissions and limitations
// under the License.

= RELEASING KUDU

== Overview

This document gives the steps required to perform a Kudu release, and is a
resource for Kudu release managers. Make sure to read and understand the
link:http://www.apache.org/legal/release-policy.html[ASF policy on software
releases] prior to conducting the release management activity. Pay attention to
the link:https://www.apache.org/legal/resolved.html[3rd-party license policy],
making sure the release doesn't include anything from the
link:https://www.apache.org/legal/resolved.html#category-x[Category X].
The command-line snippets in this document are for `bash` (Bourne-again shell).
To edit or update this document, edit `RELEASING.adoc` in `master`.

== Stating Intent to Release

NOTE: All email messages must be sent from your apache.org email address, otherwise with
company or personal email addresses sometimes emails are not delivered.
If using gmail.com service for email correspondence, follow
link:https://support.google.com/mail/answer/22370?hl=en[this guide] to add your apache.org address
to be used as `From` address when sending email messages.

. *A week before branching*: send an email to dev@kudu.apache.org to
  announce that the branch will be happening, including a deadline for when new
  commits will need gatekeeper approval. Start
  <<_compile_the_release_notes,compiling the release notes>>.

. *A day before branching*: send another email to dev@kudu.apache.org
  to warn about it.

=== Compile the release notes

Start a Google doc for the release notes that is editable by anyone, then use
https://s.apache.org to generate a short URL like kudu-1.y-rn and share it on
dev@kudu.apache.org asking for help. The author of a commit probably knows best
if a change belongs to the release notes, and how to succintly describe the
change. Of course, there will be contributors who are less active and maybe not
even monitoring the dev@ list, so you'll still have to write release notes for
their changes.

Use `git shortlog <prev-version>..HEAD` to get a list of contributions and
add it to the doc as a summary of all commits. Note the commit hash at `HEAD`, as
you'll need to generate another list between this commit and the tip when
finally branching.

Example doc: https://s.apache.org/kudu1.16rn

== Creating the Branch

. Create a new branch from `master`:
+
----
  git checkout master
  git pull
  git checkout -b branch-1.x.y
----

. Make a note of the SHA1 for the tip of the new branch, which is the first
  field of the result of this command:
+
----
  git log --oneline -n1
----

. Push the branch to public remote https://gitbox.apache.org/repos/asf/kudu.git.
  The following example assumes it's called `apache`.
+
----
  git push apache branch-1.x.y
----

. Create a new branch on Gerrit. Go to
  http://gerrit.cloudera.org:8080/#/admin/projects/kudu,branches and create a
  new branch with the same name and the previously-noted SHA1.

. Ask someone with permissions to fix the gerrit.cloudera.org mirroring
  configuration. Cloudera hosts the Gerrit server and a Cloudera employee will
  have to perform this step because SSH access is behind a firewall. The steps
  are as follows:
  1. Ensure your public SSH key is in `~gerrit/.ssh/authorized_keys` on
     gerrit.cloudera.org
  2. From behind the firewall, `ssh gerrit@gerrit.cloudera.org` to log in.
  3. Back up the existing replication configuration file by executing
     `cp ~/etc/replication.config ~/etc/replication.config.bak.`date '+%Y%m%d.%H%M%S'``
  4. Edit `~/etc/replication.config` to add a line for the new branch, such as
     `branch-1.x.y`
  5. Send email to the dev lists for Kudu and Impala (dev@kudu.apache.org and
     dev@impala.apache.org) indicating that you are going to restart Gerrit
     (link:https://s.apache.org/2Wj7[example]). It is best to do the restart at
     some time of day when you don't expect many people to be using the system,
     since Gerrit can take a few minutes to restart.
  6. Restart Gerrit: `~/bin/gerrit.sh restart`
  7. Make sure Gerrit has been successfully restarted: after a few minutes,
     try to open the link:https://gerrit.cloudera.org/#/admin/projects[project list]
     page in your favorite browser.

. As needed, patches can be cherry-picked to the new branch.

== Write the release notes

. Copy the release notes from the shared doc to docs/release_notes.adoc.

. Edit and format the release notes.

. Get the number of contributors and the list of new contributors using the
  following command, and add them to the contributors section (see
  link:https://kudu.apache.org/releases/1.16.0/docs/release_notes.html#rn_1.16.0_contributors[previous
  release notes] for examples):
+
----
  export PREV="1.15.0" # the previous release
  git shortlog -sn $PREV.. | wc -l
  diff <(git shortlog -sn ${PREV} | cut -f 2 | sort) \
    <(git shortlog -sn ${PREV}.. | cut -f 2 | sort) \
    | grep -E "^>" | sed 's/>/*/'
----
+
. Commit the release notes to the newly created branch and submit it to Gerrit.
  Use `git shortlog -sn $PREV..` to list the contributors and add them to
  the reviewers to give everyone a chance to comment on the release notes of
  their changes.

== Updating Versions in Master

. Check out the `master` branch and bump the version in `version.txt`. Don't
  update `kudu-version` in `examples/java/java-example/pom.xml` yet: it should
  be updated later on when release artifacts are published (see below).

. Bump the version numbers in docs/release_notes.adoc.

. Commit and push that change to Gerrit.

. Notify dev@kudu.apache.org that the new branch is available (see
  link:https://lists.apache.org/thread.html/de58960366583943391c00bd6b75dbd1fab3bc9067af05dd7b817a90@%3Cdev.kudu.apache.org%3E[here]
  for an example).

== Preparing a Release Candidate

. Before building a release candidate, make sure you have followed the
  Apache committer guide for
  link:https://www.apache.org/dev/new-committers-guide.html#set-up-security-and-pgp-keys[
  setting up your PGP keys]. In addition to the link:http://pgp.mit.edu/[MIT
  PGP public key server] referenced in the committer guide, consider adding
  your keys to one of link:http://pool.sks-keyservers.net:11371[SKS OpenPGP
  keyservers] or link:http://keyserver.ubuntu.com:11371[Ubuntu OpenPGP
  keyserver]. The MIT keyserver sometimes is not available, and it takes some
  time to propagate key updates anyways: the latter two servers are used by the
  link:https://repository.apache.org[Apache Maven repo server] to verify the
  signature of the uploaded maven artifacts as of October 2019.

. If building and signing on a remote/shared machine, consider
  link:https://wiki.gnupg.org/AgentForwarding[forwarding GPG agent via SSH].
  That means you can keep your secret keys on a local machine even when signing
  the artifacts to be released
  (works even for a hardware token like a smartcard, etc.).

. Out of the Kudu git workspace, checkout the
  link:https://dist.apache.org/repos/dist/release/kudu/[release SVN repository].
  Later on, the officially released and signed artifacts will be put into
  this repository. At this point, just add your PGP key to the `KEYS` file
  (if it's not there yet), making it available for the signature verification:
+
----
  svn co https://dist.apache.org/repos/dist/release/kudu/ kudu-dist-release
  cd kudu-dist-release
  (gpg --list-sigs <your-email-address> && gpg --armor --export <your-email-address>) >> KEYS
  svn commit --username=<your_apache_username> -m "Adding my key to the KEYS file"
----

. When close to building a release candidate, try building a source tarball
  (on a supported platform):
+
----
  ./build-support/build_source_release.py
----

. Fix any issues it finds, such as RAT.

. Make sure `kudu-binary` JAR artifact can be successfully built both on Linux
  and macOS:
+
----
  ./build-support/mini-cluster/build_mini_cluster_binaries.sh
----

. Test the full Java build. This will sign and build everything without
  deploying any artifacts:
+
----
  # Run a gpg-agent if you don't normally.
  gpg-agent --daemon
  # List keys with identifiers in the traditional 8-character key ID format.
  # Take a note of the identifier of the key you want to use for signing.
  gpg --list-secret-keys --keyid-format=short
  cd java
  ./gradlew clean install -PforceSigning -Psigning.gnupg.keyName=<8-character-pgp-key-id>
----

. Create a new version update commit which removes the -SNAPSHOT suffix (same
  process as above).

. In the newly created branch `branch-1.x.y` of the project git repo, update
  version-related information under the `examples` sub-directory (e.g.,
  `kudu-version` in `examples/java/java-example/pom.xml`, `version` in
  `examples/java/insert-loadgen/pom.xml`, etc.).
  The idea is making all examples use the artifacts of the newly released
  `1.x.y` version of Kudu. Commit and send out the changes for review on Gerrit.

. When ready, create a new lightweight tag and push it to the Apache Git repository.
+
----
  git tag 1.x.y-RC1
  git push apache 1.x.y-RC1
----

. Build a source tarball against the RC tag.

. Out of the Kudu git workspace, checkout the
  link:https://dist.apache.org/repos/dist/dev/kudu/[dev Subversion (SVN) repository].
  Create a new sub-directory named correspondingly. Copy the artifacts to this
  sub-directory and commit.
+
----
  svn co --depth=immediates https://dist.apache.org/repos/dist/dev/kudu/ kudu-dev-release
  cd kudu-dev-release
  mkdir 1.x.y-RC1
  cp <path_to_kudu_git_workspace>/build/apache-kudu-1.x.y.tar.* 1.x.y-RC1
  svn add 1.x.y-RC1
  svn commit --username=<your_apache_username> -m "Adding Kudu 1.x.y RC1"
----

. Create a Maven staging repository for the release candidate Java artifacts.
+
----
  # Run a gpg-agent if you don't normally
  gpg-agent --daemon
  cd java
  ./gradlew clean assemble
  # Turn off bash history: this is to avoid exposing the credentials
  # via .bash_history file.
  set +o history
  ./gradlew --no-parallel uploadArchives \
      -Psigning.gnupg.keyName=<8-character-pgp-key-id> \
      -PmavenUsername='<APACHE-LDAP-USERNAME>' \
      -PmavenPassword='<APACHE-LDAP-PASSWORD>'
  # Turn on bash history.
  set -o history
----
+
NOTE: If `uploadArchives` is executed without `--no-parallel`, uploading a
number of artifacts fails with "peer not authenticated" errors.

. Build and deploy new binary test JARs for the RC on macOS and Linux. Build
  the Linux JAR on a CentOS 7 image, and build the macOS JAR on macOS Big Sur
  if possible (see
  link:https://issues.apache.org/jira/browse/KUDU-2724[KUDU-2724] to remove the
  need to build on an old version of macOS).
+
----
  # Build a binary JAR for the local operating system. Make sure the thirdparty
  # components were built to match the source code that the RC is being built
  # with. It's a good idea to clone the Kudu git repo into a dedicated
  # workspace, rebuilding the thirdparty compoments from scratch for particular
  # release. The resulting JAR is output into the build/mini-cluster directory.
  ./build-support/mini-cluster/build_mini_cluster_binaries.sh
  # Sign and publish all matching kudu-binary artifacts from the
  # build/mini-cluster directory to the Maven staging repository that hosts
  # the Java artifacts of the Apache Kudu project (see above).
  # Turn off bash history: this is to avoid exposing the credentials persisted
  # in .bash_history file.
  set +o history
  ./build-support/mini-cluster/publish_mini_cluster_binaries.sh -a=deploy \
      -u='<APACHE-LDAP-USERNAME>' -p='<APACHE-LDAP-PASSWORD>'
  # Turn bash history back on.
  set -o history
----
+
NOTE: If the binary test JAR artifacts are deployed by the same person and from the
same machine as the Java Maven artifacts, they should appear in the same
staging repository. Otherwise, they will create a separate Maven staging
repository, which is not a problem.
+
TIP: To publish an artifact outside of the `build/mini-cluster` directory, e.g.
if JAR was built by someone else or for another OS, pass the
`-j=directory_name` argument to the publishing script to specify the location
of the JAR file to sign and publish.

. Close the Maven staging repository (or repositories).
+
Go to the link:https://repository.apache.org/[repository manager] and log
into the repository server using your Apache credentials. Make sure to enable
Adobe Flash in your browser for this Web site. Now, go the
link:https://repository.apache.org/\#stagingRepositories[staging repository]
and look for ‘orgapachekudu-####’ in the staging repositories list. You can
check the `Content` tab at the bottom to make sure you have all of the
expected stuff (client, various integrations, etc.). Hit the checkbox next to
your new staging repo and hit `Close`. Enter something similar to
"Apache Kudu 1.x.y-RC1" into the description box and confirm. Wait a minute
or two and hit `Refresh`, and your staging repo should now have a URL shown
in its summary tab
(e.g. `https://repository.apache.org/content/repositories/orgapachekudu-1005`)

== Initiating a Vote for an RC

. Send an email to dev@kudu.apache.org to start the RC process, using this
  link:https://lists.apache.org/thread/bl22kdwl0ybgty4p427p3vfm08vxqpqw[example]
  as a template.

. Reminder that voting on a release requires a
  link:https://www.apache.org/foundation/glossary.html#MajorityApproval[Majority Approval]
  by the PMC.

. Cycle through as many RCs as required.

. Always send an email with a different subject to indicate the result. For
  link:https://lists.apache.org/thread/w4ztrk821vhr3g2y2ko8yxv2f9dbndvh[example].

. After the vote passes, send an email to dev@kudu.apache.org indicating the
  result.

== Release

. For a release to be made official, the result release candidate must be put
  in the release SVN repository. Create a new sub-directory in the release SVN
  repository for the new release and copy the files from the dev repository:
+
----
  cd kudu-dist-release
  mkdir 1.x.y
  cp <path_to_kudu-dev-release>/1.x.y-RC1/* 1.x.y
  svn add 1.x.y
  svn commit --username=<your_apache_username> -m "Adding files for Kudu 1.x.y"
----

. In the Kudu git repo, create a signed tag from the RC's tag, verify the
  signature has been applied and verifiable, and push it to the
  Apache Git repository:
+
----
  gpg --list-secret-keys --keyid-format=short
  git tag -u <gpg_key_id> -m 'Release Apache Kudu 1.x.y' 1.x.y 1.x.y-RC1
  git tag -v 1.x.y
  git push apache 1.x.y
----

. Release the staged Java artifacts. Select the release candidate staging
  repository in link:https://repository.apache.org/#stagingRepositories[Nexus],
  and click `Release`. You should shortly be able to see the artifacts in
  link:https://search.maven.org/search?q=g:org.apache.kudu[Maven Central].

. Release the Python artifacts. You will need to setup an account on link:https://PyPi.org[PyPi.org]
  and ask to be added to the kudu-python PyPi project if you have not done this before.
+
----
  # Prepare and sign the python source distribution.
  cd python
  rm -rf dist/*
  python setup.py sdist
  gpg --detach-sign -a dist/kudu-python-1.x.y.tar.gz
  # Upload the distribution to PyPi using twine.
  pip install twine
  twine upload dist/*
----
Note: You can upload to the test PyPi by adding
`--repository-url https://test.pypi.org/legacy/` to the twine command.

. Generate the version-specific documentation from that branch following these
  <<README.adoc#updating-the-site,instructions>>.
+

WARNING: The site *MUST NOT* be built on Mac. See the
<<README.adoc#building-docs,documentation building instructions>> for details.

+
. Update the `index.md` file in the `releases` directory. Add a line about
  the newly released version into the `Latest release` section, move line
  about the previous one into the `Previous releases`. Overall, update
  the `Previous releases` and `Archived releases` to reflect the new contents
  of the
  link:https://dist.apache.org/repos/dist/release/kudu/[release SVN repository].
  See the last item of this section to get the idea on what the new contents
  of the release SVN repository is going to be.

. A new sub-directory named after the release version should be added into the
  `releases` directory: it should contain `apidocs`, `cpp-client-api`,
  `docs` sub-dirs and `index.md` file. Add the newly added sub-directory into
   the git staging:
+
----
  # Assuming current working directory is the root of the git workspace.
  git add releases/1.x.y
----

. Take a quick look at the auto-generated `releases/1.x.y/index.md` file
  to make sure the download links meet the current
  link:https://www.apache.org/dev/release-download-pages.html[criteria]. The
  criteria keep changing and the announcement will be rejected if our release
  page doesn't meet the criteria.

. Commit the changes:
+
----
  git commit -a -m "update website for 1.x.y release"
----

. Submit these changes to the `gh-pages` Gerrit branch and get them reviewed.

. Add a release blog post about the new release and send it out for review, similar to
  link:https://kudu.apache.org/2019/07/09/apache-kudu-1-10-0-release.html[this].

. Once the reviews are finished and the commits are pushed to `gh-pages` branch,
  update the website following these
  <<README.adoc#deploying-changes-to-the-apache-kudu-web-site,instructions>>.

. Build and push the Docker images (run the below command from the project
  root).
+
----
 ./docker/docker-build.py --action push --platforms linux/amd64 linux/arm64
----
+
To push the images to Dockerhub, you need to be granted permissions by the ASF
infra team, which you can request via an Apache JIRA ticket similar to
link:https://issues.apache.org/jira/browse/INFRA-23166[INFRA-23166].

. About 24 hours after all artifacts have been published, send an email to
  user@kudu.apache.org, dev@kudu.apache.org, and announce@apache.org to
  announce the new release. The email should be similar to
  link:https://s.apache.org/pduz[this]. The announcement email must be sent
  from your apache.org address, otherwise apache.org mailer will not deliver
  the message to the addressees of the announce@ group at least.  If using
  gmail.com service for email correspondence, follow
  link:https://support.google.com/mail/answer/22370?hl=en[this guide] to add
  your apache.org address to be used as `From` address when sending email
  messages. Also post the announcement to the blog and tweet it with the
  link:https://twitter.com/ApacheKudu[@ApacheKudu] Twitter handle.

. Update the version number on the branch you released from back to a SNAPSHOT
  for the next patch release, such as `1.6.1-SNAPSHOT` after the `1.6.0` release.

. In the `master` branch of the project git repo, update version-related
  information under the `examples` sub-directory to make all examples using the
  fresh artifacts of latest released version.  This should be done only after
  publishing the officially released artifacts, so they are available while
  building the examples.

. In the `master` branch of the project git repo, add the previous release notes to the
  `prior_release_notes.adoc` and send out the change for review on Gerrit.

. About another 24 hours later, clean up the SVN. If releasing a new minor
  version, delete the oldest minor version branch in the release repo (e.g. if
  `1.7.1`, `1.8.0`, and `1.9.0` exist and you just released `1.10.0`, delete
  `1.7.1`). If releasing a maintenance version, delete the previous maintenance
  branch (e.g. if you released `1.2.1`, delete `1.2.0`). Also delete any
  release candidates from the dev SVN.

. Set the release date for the just released version in
  link:https://issues.apache.org[Apache JIRA] or ask someone with permission
  to do so. After logging in, in the left sidebar switch to the `Releases` item.
  Once at the `Releases` page, apply the `Unreleased` filter. In the row
  corresponding to the new release, click at the ellipsis in the `Action`
  column and select the `Release` item, setting the appropriate release date.