This guide is for Apache Kylin Committers only.
Shell commands are on Mac OS X as sample.
For people in China, please aware using proxy to avoid potential firewall issue.
Make sure you have avaliable account and privilege for following applications:
Follow instructions at http://www.apache.org/dev/release-signing to create a key pair
Install gpg (On Mac OS X as sample):brew install gpg
Generate gpg key:
Reference: https://www.gnupg.org/gph/en/manual/c14.html
All new RSA keys generated should be at least 4096 bits. Do not generate new DSA keysgpg --full-generate-key
Verify your key:gpg --list-sigs YOUR_NAME
Get the fingerprint of your key: gpg --fingerprint YOUR_NAME
It will display the fingerprint like “Key fingerprint = XXXX XXXX ...”, then add the fingerprint to your apache account at https://id.apache.org/ in “OpenPGP Public Key Primary Fingerprint” field; wait for a few hours the key will added to https://people.apache.org/keys/, for example: https://people.apache.org/keys/committer/lukehan.asc
Generate ASCII Amromed Key:gpg -a --export YOUR_MAIL_ADDRESS > YOUR_NAME.asc &
Upload key to public server:gpg --send-keys YOUR_KEY_HASH
or Submit key via web:
Open and Submit to http://pool.sks-keyservers.net:11371 (you can pickup any available public key server)
Once your key submitted to server, you can verify using following command:gpg --recv-keys YOUR_KEY_HASH for example:gpg --recv-keys 027DC364
Add your public key to the KEYS file by following instructions in the KEYS file.:
KEYS file location: ${kylin}/KEYS
For example:(gpg --list-sigs YOURNAME && gpg --armor --export YOURNAME) >> KEYS
Commit your changes.
Before you start:
Configure Apache repository server in Maven If you‘re the first time to do release, you need update the server authentication information in ~/.m2/settings.xml; If this file doesn’t exist, copy a template from $M2_HOME/conf/settings.xml;
In the “servers” section, make sure the following servers be added, and replace #YOUR_APACHE_ID#, #YOUR_APACHE_PWD#, #YOUR_GPG_PASSPHRASE# with your ID, password, and passphrase: {% highlight bash %} apache.snapshots.https #YOUR_APACHE_ID# #YOUR_APACHE_PWD# apache.releases.https #YOUR_APACHE_ID# #YOUR_APACHE_PWD#
<!-- To publish a website of some part of Maven --> <server> <id>apache.website</id> <username>#YOUR_APACHE_ID#</username> <password>#YOUR_APACHE_PWD#</password> <!-- Either <privateKey>...</privateKey> --> <filePermissions>664</filePermissions> <directoryPermissions>775</directoryPermissions> </server> <!-- To stage a website of some part of Maven --> <server> <id>stagingSite</id> <!-- must match hard-coded repository identifier in site:stage-deploy --> <username>#YOUR_APACHE_ID#</username> <filePermissions>664</filePermissions> <directoryPermissions>775</directoryPermissions> </server> <server> <id>gpg.passphrase</id> <passphrase>#YOUR_GPG_PASSPHRASE#</passphrase> </server>
Fix license issues {% highlight bash %} Set passphrase variable without putting it into shell history $ read -s GPG_PASSPHRASE
Make sure that there are no junk files in the sandbox $ git clean -xf $ mvn clean
Make sure all unit tests are passed $ mvn test
Check the org.apache.kylin.common.KylinVersion class, ensure the value of CURRENT_KYLIN_VERSION is the release version.
Fix any license issues as reported by target/rat.txt $ mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_PASSPHRASE} install {% endhighlight %}
Optionally, when the dry-run has succeeded, change install to deploy: {% highlight bash %} $ mvn -Papache-release -DskipTests -Dgpg.passphrase=${GPG_PASSPHRASE} deploy {% endhighlight %}
Prepare
Make sure your can ssh connection to github: {% highlight bash %} ssh -T git@github.com {% endhighlight %}
Create a branch for release work from your current development branch, named with this release version, e.g. v2.5.0-release (should not be the same name with the tag name), and then push it to Apache.
{% highlight bash %} $ git checkout -b vX.Y.Z-release $ git push -u origin vX.Y.Z-release {% endhighlight %}
If any of the steps fail, clean up (see below), fix the problem, and start again from the top.
{% highlight bash %} Set passphrase variable without putting it into shell history $ read -s GPG_PASSPHRASE
Make sure that there are no junk files in the sandbox $ git clean -xf $ mvn clean
Optionally, do a dry run of the release:prepare step, which sets version numbers. e.g. releaseVersion=2.5.0, developmentVersion=2.5.1-SNAPSHOT, use default tag kylin-2.5.0 $ mvn -DdryRun=true -DskipTests -DreleaseVersion=X.Y.Z -DdevelopmentVersion=(X.Y.Z+1)-SNAPSHOT -Papache-release -Darguments=“-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests” release:prepare 2>&1 | tee /tmp/prepare-dry.log {% endhighlight %}
Check the dry run output:
target directory should be these 8 files, among others:apache-kylin-..zip, check that all files belong to a directory called apache-kylin-X.Y.Z-SNAPSHOT.NOTICE, LICENSE, README.mdRun real release: Now, run the release for real.
{% highlight bash %}
$ mvn -DskipTests -DreleaseVersion=X.Y.Z -DdevelopmentVersion=(X.Y.Z+1)-SNAPSHOT -Papache-release -Darguments=“-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests” release:prepare
$ mvn -DskipTests -Papache-release -Darguments=“-Dgpg.passphrase=${GPG_PASSPHRASE} -DskipTests” release:perform {% endhighlight %}
Close the staged artifacts in the Nexus repository:
Build Promotion, click Staging RepositoriesStaging Repositories tab there should be a line with profile org.apache.kylinUpload to staging area:
Upload the artifacts via subversion to a staging area, https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-X.Y.Z-rcN: {% highlight bash %}
$ mkdir -p ~/dist/dev $ pushd ~/dist/dev $ svn co https://dist.apache.org/repos/dist/dev/kylin $ popd
$ cd target $ mkdir ~/dist/dev/kylin/apache-kylin-X.Y.Z-rcN $ mv apache-kylin-* ~/dist/dev/kylin/apache-kylin-X.Y.Z-rcN
$ rm ~/dist/dev/kylin/apache-kylin-X.Y.Z-rcN/apache-kylin-X.Y.Z-SNAPSHOT-source-release.zip.asc.sha256
$ cd ~/dist/dev/kylin $ svn add apache-kylin-X.Y.Z-rcN $ svn commit -m ‘Upload release artifacts to staging’ --username <YOUR_APACHE_ID> {% endhighlight %}
Cleaning up after a failed release attempt: {% highlight bash %}
$ git tag
$ git tag -d kylin-X.Y.Z $ git push origin :refs/tags/kylin-X.Y.Z
$ mvn release:clean
$ git status $ git reset --hard HEAD {% endhighlight %}
{% highlight bash %}
$ mvn test
$ gpg --recv-keys key
$ curl -O https://dist.apache.org/repos/dist/release/kylin/KEYS
function checkHash() { cd “$1” for i in *.{pom,gz}; do if [ ! -f $i ]; then continue fi if [ -f $i.sha256 ]; then if [ “$(cat $i.sha256)” = “$(shasum -a 256 $i)” ]; then echo $i.sha256 present and correct else echo $i.sha256 does not match fi else shasum -a 256 $i > $i.sha256 echo $i.sha256 created fi done }; $ checkHash apache-kylin-X.Y.Z-rcN {% endhighlight %}
Vote on Apache Kylin dev mailing list
Release vote on dev list, use the commit id that generated by Maven release plugin, whose message looks like “[maven-release-plugin] prepare release kylin-x.x.x”:
{% highlight text %} To: dev@kylin.apache.org Subject: [VOTE] Release apache-kylin-X.Y.Z (RC[N])
Hi all,
I have created a build for Apache Kylin X.Y.Z, release candidate N.
Changes highlights: ...
Thanks to everyone who has contributed to this release. Here’s release notes: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316121&version=12343540
The commit to be voted upon:
https://github.com/apache/kylin/commit/xxx
Its hash is xxx.
The artifacts to be voted on are located here: https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-X.Y.Z-rcN/
The hash of the artifact is as follows: apache-kylin-X.Y.Z-source-release.zip.sha256 xxx
A staged Maven repository is available for review at: https://repository.apache.org/content/repositories/orgapachekylin-XXXX/
Release artifacts are signed with the following key: https://people.apache.org/keys/committer/lukehan.asc
Please vote on releasing this package as Apache Kylin X.Y.Z.
The vote is open for the next 72 hours and passes if a majority of at least three +1 PMC votes are cast.
[ ] +1 Release this package as Apache Kylin X.Y.Z [ ] 0 I don‘t feel strongly about it, but I’m okay with the release [ ] -1 Do not release this package because...
Here is my vote:
+1 (binding)
{% endhighlight %}
After vote finishes, send out the result:
{% highlight text %} Subject: [RESULT][VOTE] Release apache-kylin-X.Y.Z (RC[N]) To: dev@kylin.apache.org
Thanks to everyone who has tested the release candidate and given their comments and votes.
The tally is as follows.
N binding +1s:
N non-binding +1s:
No 0s or -1s.
Therefore I am delighted to announce that the proposal to release Apache-Kylin-X.Y.Z has passed.
{% endhighlight %}
After a successful release vote, we need to push the release out to mirrors, and other tasks.
In JIRA, search for all issues resolved in this release, and do a bulk update changing their status to “Closed”, with a change comment “Resolved in release X.Y.Z (YYYY-MM-DD)” (fill in release number and date appropriately).
Uncheck “Send mail for this update”.
Mark the version released in JIRA system, Manage Versions.
Promote the staged nexus artifacts.
Check the artifacts into svn. {% highlight bash %}
$ mkdir -p ~/dist/dev $ cd ~/dist/dev $ svn co https://dist.apache.org/repos/dist/dev/kylin
$ mkdir -p ~/dist/release $ cd ~/dist/release $ svn co https://dist.apache.org/repos/dist/release/kylin $ cd kylin $ mkdir apache-kylin-X.Y.Z $ cp -rp ../../dev/kylin/apache-kylin-X.Y.Z-rcN/apache-kylin* apache-kylin-X.Y.Z/ $ svn add apache-kylin-X.Y.Z
svn commit -m ‘checkin release artifacts’ {% endhighlight %}
Svnpubsub will publish to https://dist.apache.org/repos/dist/release/kylin and propagate to http://www.apache.org/dyn/closer.cgi/kylin within 24 hours.
If there are more than 2 releases, clear out the oldest ones:
{% highlight bash %} cd ~/dist/release/kylin svn rm apache-kylin-X.Y.Z svn commit -m ‘Remove old release’ {% endhighlight %}
The old releases will remain available in the release archive.
Release same version in JIRA, check Change Log.
After publish the release, you need generate the binary packages and then put them to the svn release repository;
Do git fetch --all --prune --tags to sync your local repo with remote.
Git checkout the tag for current release;
Make a binary package by refering to this doc;
Sign the generated binary package with gpg, e.g,: {% highlight bash %} gpg --armor --output apache-kylin-2.5.0-bin.tar.gz.asc --detach-sig apache-kylin-2.5.0-bin.tar.gz {% endhighlight %}
Generate the sha256 file for the binary package, e.g,: {% highlight bash %} shasum -a 256 apache-kylin-2.5.0-bin.tar.gz > apache-kylin-2.5.0-bin.tar.gz.sha256
on Linux: openssl sha256 apache-kylin-2.5.0-bin.tar.gz > apache-kylin-2.5.0-bin.tar.gz.sha256
{% endhighlight %}
Push the binary package, the signature file and the sha256 file to the svn dev repo, then run svn mv <files-in-dev> <files-in-release> to move them to svn release repo.
For different Hadoop/HBase version, you may need repeat the above steps;
Add the files and then commit the svn changes.
After publish the release, you need to do update some source code:
Merge the release branch, e.g. v2.5.0-release, to the current development branch, e.g. 2.5.x, for preparing the next development iteration.
Manually update KylinVersion class, change value of CURRENT_KYLIN_VERSION to current development version.
Refer to How to document for more detail.
Send one mail with subject like “[Announce] Apache Kylin X.Y.Z released” to following list:
Here is a sample of announcement email (by studying Kafka's):
{% highlight text %} The Apache Kylin team is pleased to announce the immediate availability of the 2.5.0 release.
This is a major release after 2.4, with more than 100 bug fixes and enhancements; All of the changes in this release can be found in: https://kylin.apache.org/docs/release_notes.html
You can download the source release and binary packages from Apache Kylin's download page: https://kylin.apache.org/download/
Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Apache Hadoop, supporting extremely large datasets.
Apache Kylin lets you query massive dataset at sub-second latency in 3 steps:
Thanks to everyone who has contributed to the 2.5.0 release.
We welcome your help and feedback. For more information on how to report problems, and to get involved, visit the project website at https://kylin.apache.org/
{% endhighlight %}
This guide drafted with reference from Apache Calcite Howto doc, Thank you very much.