Here's some miscellaneous documentation about using Avatica.
Prerequisites are maven (3.2.1 or later) and Java (JDK 8 or later) on your path.
Unpack the source distribution .tar.gz file, cd to the root directory of the unpacked source, then build using maven:
{% highlight bash %} $ tar xvfz apache-calcite-avatica-1.12.0-src.tar.gz $ cd apache-calcite-avatica-1.12.0-src $ mvn install {% endhighlight %}
Running tests describes how to run more or fewer tests.
Prerequisites are git, maven (3.2.1 or later) and Java (JDK 8 or later) on your path.
Create a local copy of the github repository, cd to its root directory, then build using maven:
{% highlight bash %} $ git clone git://github.com/apache/calcite-avatica.git avatica $ cd avatica $ mvn install {% endhighlight %}
Running tests describes how to run more or fewer tests.
The test suite will run by default when you build, unless you specify -DskipTests:
{% highlight bash %} $ mvn clean verify -Dcheckstyle.skip {% endhighlight %}
By default, invoking the verify Maven lifecycle phase will also cause checkstyle rules to be run. It is expected that contributions pass the checkstyle rules; however, it is common to ignore these while working on a feature/bug and fix them at the end.
See the [developers guide]({{ site.baseurl }}/develop/#contributing).
See the [developers guide]({{ site.baseurl }}/develop/#getting-started).
The following sections might be of interest if you are adding features to particular parts of the code base. You don't need to understand these topics if you are just building from source and running tests.
Calcite's Avatica Server component supports RPC serialization using Protocol Buffers. In the context of Avatica, Protocol Buffers can generate a collection of messages defined by a schema. The library itself can parse old serialized messages using a new schema. This is highly desirable in an environment where the client and server are not guaranteed to have the same version of objects.
Typically, the code generated by the Protocol Buffers library doesn't need to be re-generated only every build, only when the schema changes.
First, install Protobuf 3.1:
{% highlight bash %} $ wget https://github.com/google/protobuf/releases/download/v3.1.0/protobuf-java-3.1.0.tar.gz $ tar xf protobuf-java-3.1.0.tar.gz && cd protobuf-3.1.0 $ ./configure $ make $ sudo make install {% endhighlight %}
Then, re-generate the compiled code:
{% highlight bash %} $ cd avatica/core $ ./src/main/scripts/generate-protobuf.sh {% endhighlight %}
The following sections are of interest to Calcite committers and in particular release managers.
Follow instructions here to create a key pair. (On Mac OS X, I did brew install gpg and gpg --gen-key.)
Add your public key to the KEYS file by following instructions in the KEYS file. (The KEYS file is not present in the git repo or in a release tar ball because that would be redundant.)
By default, Maven plugins which require you to unlock a GPG secret key will prompt you in the terminal. To prevent you from having to enter this password numerous times, it is highly recommended to install and run gpg-agent.
This can be started automatically via an ~/.xsession on Linux or some scripting in your shell's configuration script of choice (e.g. ~/.bashrc or ~/.zshrc)
{% highlight bash %} GPG_AGENT=$(which gpg-agent) GPG_TTY=tty export GPG_TTY if [[ -f “$GPG_AGENT” ]]; then envfile=“${HOME}/.gnupg/gpg-agent.env”
if test -f “$envfile” && kill -0 $(grep GPG_AGENT_INFO “$envfile” | cut -d: -f 2) 2>/dev/null; then source “$envfile” else eval “$(gpg-agent --daemon --log-file=~/.gpg/gpg.log --write-env-file “$envfile”)” fi export GPG_AGENT_INFO # the env file does not contain the export statement fi {% endhighlight %}
Also, ensure that default-cache-ttl 6000 is set in ~/.gnupg/gpg-agent.conf to guarantee that your credentials will be cached for the duration of the build.
Before you start:
{% highlight bash %}
git clean -xn
mvn -Papache-release clean install {% endhighlight %}
When the dry-run has succeeded, change install to deploy.
Before you start:
README, site/_docs/howto.md, site/_docs/docker_images.md, and docker/src/main/dockerhub/Dockerfile have the correct version number.NOTICE has the current copyright year.version.major and version.minor in pom.xml.site/_docs/history.md. Include the commit history, and say which versions of Java, Guava and operating systems the release is tested against.Create a release branch named after the release, e.g. branch-avatica-1.9, and push it to Apache.
{% highlight bash %} $ git checkout -b branch-avatica-X.Y $ git push -u origin branch-avatica-X.Y {% endhighlight %}
We will use the branch for the entire the release process. Meanwhile, we do not allow commits to the master branch. After the release is final, we can use git merge --ff-only to append the changes on the release branch onto the master branch. (Apache does not allow reverts to the master branch, which makes it difficult to clean up the kind of messy commits that inevitably happen while you are trying to finalize a release.)
Now, set up your environment and do a dry run. The dry run will not commit any changes back to git and gives you the opportunity to verify that the release process will complete as expected.
If any of the steps fail, clean up (see below), fix the problem, and start again from the top.
{% highlight bash %}
git clean -xn
patch -p1 <<EOF diff --git a/docker/src/main/dockerhub/Dockerfile b/docker/src/main/dockerhub/Dockerfile index 4617a4e..4ccd97f 100644 --- a/docker/src/main/dockerhub/Dockerfile +++ b/docker/src/main/dockerhub/Dockerfile @@ -23,3 +23,3 @@ RUN mkdir -p /home/avatica/classpath
-ARG AVATICA_VERSION=“1.12.0” +ARG AVATICA_VERSION=“1.12.0-SNAPSHOT” EOF
mvn -DdryRun=true -DreleaseVersion=X.Y.Z -DdevelopmentVersion=X2.Y2.Z2-SNAPSHOT -Dtag=avatica-X.Y.Z-rcN -Papache-release -Duser.name=${asf.username} release:prepare {% endhighlight %}
Check the artifacts:
target directory should be these 6 files, among others:apache-calcite-avatica-..tar.gz (currently there is no binary distro), check that all files belong to a directory called apache-calcite-avatica-X.Y.Z-src.NOTICE, LICENSE, README, README.mdREADME is correctKEYS file in the source distroscore/target/avatica-core-X.Y.Z.jar and server/target/avatica-server-X.Y.Z-sources.jar), verify that the META-INF directory contains the correct contents for DEPENDENCIES, LICENSE and NOTICE per the source/classes contained. Refer to the ASF licensing documentation on what is required.If something is not correct, you can invoke the release:clean mojo to remove the generated files from your workspace:
{% highlight bash %} mvn release:clean {% endhighlight %}
If successful, remove the -DdryRun flag and run the release for real.
{% highlight bash %}
mvn -DreleaseVersion=X.Y.Z -DdevelopmentVersion=X2.Y2.Z2-SNAPSHOT -Dtag=avatica-X.Y.Z-rcN -Papache-release -Duser.name=${asf.username} release:prepare
mvn -Papache-release -Duser.name=${asf.username} release:perform -Darguments=“-DskipTests” {% endhighlight %}
Verify the staged artifacts in the Nexus repository:
Build Promotion, click Staging RepositoriesStaging Repositories tab there should be a line with profile org.apache.calciteUpload the artifacts via subversion to a staging area, https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-avatica-X.Y.Z-rcN:
{% highlight bash %}
mkdir -p ~/dist/dev pushd ~/dist/dev svn co https://dist.apache.org/repos/dist/dev/calcite popd
cd target mkdir ~/dist/dev/calcite/apache-calcite-avatica-X.Y.Z-rcN mv apache-calcite-avatica-* ~/dist/dev/calcite/apache-calcite-avatica-X.Y.Z-rcN
cd ~/dist/dev/calcite svn add apache-calcite-avatica-X.Y.Z-rcN svn ci {% endhighlight %}
{% highlight bash %}
git tag
git tag -d avatica-X.Y.Z git push origin :refs/tags/avatica-X.Y.Z
mvn release:clean
git status git reset --hard HEAD {% endhighlight %}
{% highlight bash %}
gpg --recv-keys key
curl -O https://dist.apache.org/repos/dist/release/calcite/KEYS
function checkHash() { cd “$1” for i in *.{pom,gz}; do if [ ! -f $i ]; then continue fi if [ -f $i.sha256 ]; then if [ “$(cat $i.sha256)” = “$(shasum -a 256 $i)” ]; then echo $i.sha256 present and correct else echo $i.sha256 does not match fi else shasum -a 256 $i > $i.sha256 echo $i.sha256 created fi done } checkHash apache-calcite-avatica-X.Y.Z-rcN {% endhighlight %}
Release vote on dev list
{% highlight text %} To: dev@calcite.apache.org Subject: [VOTE] Release apache-calcite-avatica-X.Y.Z (release candidate N)
Hi all,
I have created a build for Apache Calcite Avatica X.Y.Z, release candidate N.
Thanks to everyone who has contributed to this release. You can read the release notes here: https://github.com/apache/calcite-avatica/blob/XXXX/site/_docs/history.md
The commit to be voted upon: http://git-wip-us.apache.org/repos/asf/calcite-avatica/commit/NNNNNN
Its hash is XXXX.
The artifacts to be voted on are located here: https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-avatica-X.Y.Z-rcN/
The hashes of the artifacts are as follows: src.tar.gz.sha256 XXXX
A staged Maven repository is available for review at: https://repository.apache.org/content/repositories/orgapachecalcite-NNNN
Release artifacts are signed with the following key: https://people.apache.org/keys/committer/jhyde.asc
Please vote on releasing this package as Apache Calcite Avatica X.Y.Z.
The vote is open for the next 72 hours and passes if a majority of at least three +1 PMC votes are cast.
[ ] +1 Release this package as Apache Calcite X.Y.Z [ ] 0 I don‘t feel strongly about it, but I’m okay with the release [ ] -1 Do not release this package because...
Here is my vote:
+1 (binding)
Julian {% endhighlight %}
After vote finishes, send out the result:
{% highlight text %} Subject: [RESULT] [VOTE] Release apache-calcite-avatica-X.Y.Z (release candidate N) To: dev@calcite.apache.org
Thanks to everyone who has tested the release candidate and given their comments and votes.
The tally is as follows.
N binding +1s:
N non-binding +1s:
No 0s or -1s.
Therefore I am delighted to announce that the proposal to release Apache Calcite Avatica X.Y.Z has passed.
Thanks everyone. We’ll now roll the release out to the mirrors.
There was some feedback during voting. I shall open a separate thread to discuss.
Julian {% endhighlight %}
Use the Apache URL shortener to generate shortened URLs for the vote proposal and result emails. Examples: s.apache.org/calcite-1.2-vote and s.apache.org/calcite-1.2-result.
After a successful release vote, we need to push the release out to mirrors, and other tasks.
Choose a release date. This is based on the time when you expect to announce the release. This is usually a day after the vote closes. Remember that UTC date changes at 4pm Pacific time.
In JIRA, search for all issues resolved in this release, and do a bulk update changing their status to “Closed”, with a change comment “Resolved in release X.Y.Z (YYYY-MM-DD)” (fill in release number and date appropriately). Uncheck “Send mail for this update”.
Promote the staged nexus artifacts.
Copy the Git tag:
{% highlight bash %} git tag rel/avatica-X.Y.X avatica-X.Y.Z-rcN git push origin rel/avatica-X.Y.Z {% endhighlight %}
Check the artifacts into svn.
{% highlight bash %}
mkdir -p ~/dist/dev cd ~/dist/dev svn co https://dist.apache.org/repos/dist/dev/calcite
mkdir -p ~/dist/release cd ~/dist/release svn co https://dist.apache.org/repos/dist/release/calcite cd calcite cp -rp ../../dev/calcite/apache-calcite-avatica-X.Y.Z-rcN apache-calcite-avatica-X.Y.Z svn add apache-calcite-avatica-X.Y.Z
svn ci {% endhighlight %}
Svnpubsub will publish to the release repo and propagate to the mirrors within 24 hours.
If there are now more than 2 releases, clear out the oldest ones:
{% highlight bash %} cd ~/dist/release/calcite svn rm apache-calcite-avatica-X.Y.Z svn ci {% endhighlight %}
The old releases will remain available in the release archive.
Publish the Docker images.
Add a release note by copying [site/_posts/2016-11-01-release-1.9.0.md]({{ site.sourceRoot }}/site/_posts/2016-11-01-release-1.9.0.md), generate the javadoc and copy to site/target/avatica/apidocs and site/target/avatica/testapidocs, publish the site, and check that it appears in the contents in news.
After 24 hours, announce the release by sending an email to announce@apache.org. You can use the 1.8.0 announcement as a template. Be sure to include a brief description of the project.
{: #publish-the-web-site}
See instructions in [site/README.md]({{ site.sourceRoot }}/site/README.md).