Instructions for preparing and distributing a release of Apache Thrift are fairly complex. These procedures are documented here, and we're working to automate as much of this as possible. There are few projects like ours that integrate with 28 programming languages. Given the extreme number of package management systems that Apache Thrift integrates with (compared to perhaps any), part of the burden of releasing Apache Thrift is to manually package and upload some of these language-specific packages.
It is important to note here that Apache Thrift is designed for version interoperability, so one can use a version 0.7.0 client with a 0.12.0 server. A particular version number does not make any guarantees as to the features available in any given language. See the Language Feature Matrix to learn more.
Apache Thrift and the vast majority of package management systems out there conform to the SemVer 2.0 version numbering specification. Apache Thrift uses the following versioning rules:
Further, if there are only packaging changes for a single third-party distribution point to correct an issue, the major.minor.patch may remain the same while adding a suffix compatible with that distribution point, for example, “0.12.0.1” for nuget, or “0.12.0-1” for maven.
It is common to have language-specific critical defects or packaging errors that need to be resolved between releases of Apache Thrift. The project handles these on a case-by-case basis for languages that have their own package management systems. When a language-specific patch is made, the patch level of the distribution pushed to the external package manager is bumped.
As such, there may be cases between Apache Thrift releases where there are (for example) a 0.12.1 and 0.12.2 version of a Haskell Hackage package, and perhaps also a 0.12.3 version of a dlang dub package. You will not find a tag or an official project release in these cases, however, the code changes will be reflected in the release branch and in master. In these cases we would not release a version of Apache Thrift nor would we refresh all the external language packages.
The master branch will always contain the next anticipated release version. When a release cycle begins, a branch is cut from master. The release branch will already have all of the correct versions, and therefore release branches can be easily merged back into master. (This was not true of releases before 0.12.0).
The authoritative repository for Apache Thrift is stored in GitHub. It is mirrored by GitBox.
All code (submitted via pull request or direct push) is committed to the master branch. Until version 1.0 of Apache Thrift each release branch was named <version>, for example in version 0.12.0 there is a branch named the same. For version 1.0 releases any beyond, releases will have a branch named release/<version>.
Up to version 0.12.0 each release of Apache Thrift was tagged with a <version> tag. Starting with the 0.12.0 release, each release of Apache Thrift will be tagged with a v<version> tag to satisfy external package management tools (such as ones for dlang and golang). For example the tag of version 0.12.0 is v0.12.0.
Apache Thrift has no official release schedule, however the project aims to release at least twice per year.
A complete release cycle will take about 1 week to complete, if things go well, with half of that time waiting for a vote.
Before a release cycle begins, someone must nominate themselves on the development mailing list as the release manager for that release. In order to be a release manager you must meet the following criteria:
dd if=/dev/sda of=/dev/zerorng-tools package.All Apache Thrift releases go through a 72-hour final release candidate voting procedure. Votes from members of the Apache Thrift PMC are binding, and all others are non-binding. For these examples, the master branch is at version 1.0.0 and that is the next release.
Scrub the Apache Jira backlog. There are a couple things to do:
Open Issues without a Component - make sure everything has an assigned component, as the release notes are grouped together by language.
Open Issues with a Fix Version - these will be issues that someone placed a fixVersion on in Jira, but have not been resolved or closed yet. They are likely stale somehow. Resolutions for these issues include resolving or closing the issue in Jira, or simply removing the fixVersion if the issue hasn't been fixed.
Open Blocking Issues - blocking issues should block a release. Scrub the list to see if they are really blocking the release, and if not change their priority.
Open Critical Issues - this list will end up in the known critical issues list in the changes file. Scrub it to make sure everything is actually critical.
It is healthy to scrub these periodically, whether or not you are making a new release.
Check that the version number in the master branch matches the version number of the upcoming release. To check the master branch version, run:
thrift$ grep AC_INIT configure.ac | cut -d'[' -f3 | cut -d']' -f1 1.0.0
If it does not match (this should be extremely rare), you need to submit a pull request setting the master branch to the desired version of the upcoming release. In the following example, we prepare to commit a branch where the version number is changed from 1.0.0 to 1.1.0:
thrift$ git checkout -b fix-version-for-release thrift$ build/veralign.sh 1.0.0 1.1.0 # check to see if any of the manually modified files needs changes thrift$ git push ... # make a pull request
Create a release branch for the release, in this example 1.0.0:
thrift$ git checkout master thrift$ git pull thrift$ git checkout -b "release/1.0.0" thrift$ git push
Now there is a release/1.0.0 branch in GitHub for Apache Thrift.
By creating a release branch we allow work to continue on the master branch for the next release while we finalize this one. Note that release/1.0.0 and master in this example are now identical, and therefore it is possible to merge the release branch back into master at the end of the release!
Modify these files manually, inserting the release into them at the appropriate location. Follow existing patterns in each file:
doap.rdfdebian/changelogGenerate the content for CHANGES.md - this is one of the most time-consuming parts of the release cycle. It is a lot of work, but the result is well worth it to the consumers of Apache Thrift:
Find all Issues Fixed but not Closed in 1.0.0 (adjust the version in the link to suit your needs).
Export the list of issues to a CSV (Current Fields) and open in Excel (or a similar spreadsheet).
Hide all columns except for the issue id (i.e. THRIFT-nnnn), the component (first one), and the summary.
Sort by component ascending and then by id ascending.
Create a fourth column that will contain the contents of each line that goes into the release notes. Once you have the formula working in one cell paste it into the other rows to populate them. Use a formula to get the column to look like this:
Issue Component Summary RelNote THRIFT-123 C++ - Library Drop C++03 [THRIFT-123](https://issues.apache.org/jira/browse/THRIFT-3978) - Drop C++03
For example, if the row above was row “B” in EXCEL it would look something like:
=CONCAT("[", B1, "]", "https://issues.apache.org/jira/browse/", B1, " - ", B3)
Create a level 3 section in CHANGES.md under the release for each component and copy the items from the RelNote column into the changes file.
Find all Open Critical Issues and add them to CHANGES.md in the list of known critical issues for the release.
Commit all changes to the release branch.
Generate the source tarball.
On a linux system get a clean copy of the release branch, for example:
~$ git clone -b "release/1.0.0" git@github.com:apache/thrift.git thrift-1.0.0-src
In the clean copy of the release branch, start a docker build container and run make dist:
~$ cd thrift-1.0.0-src ~/thrift-1.0.0-src$ docker run -v $(pwd):/thrift/src:rw \ -it thrift/thrift-build:ubuntu-bionic /bin/bash root@8b4101188aa2:/thrift/src# ./bootstrap.sh && ./configure && make dist
The result will be a file named thrift-1.0.0.tar.gz. Check the size and make sure it is roughly 4MB. It could get larger over time, but it shouldn't jump by orders of magnitude. Once satisfied you can exit the docker container with exit.
Generate signatures and checksums for the tarball:
gpg --armor --output thrift-1.0.0.tar.gz.asc --detach-sig thrift-1.0.0.tar.gz md5sum thrift-1.0.0.tar.gz > thrift-1.0.0.tar.gz.md5 sha1sum thrift-1.0.0.tar.gz > thrift-1.0.0.tar.gz.sha1 sha256sum thrift-1.0.0.tar.gz > thrift-1.0.0.tar.gz.sha256
Generate the Windows Thrift Compiler. This is a statically linked compiler that is portable and folks find it useful to be able to download one, especially if they are using third-party distributed runtime libraries for interpreted languages on Windows. There are two ways to generate this:
Using a Development VM
C:\build> cmake ..\thrift -DBISON_EXECUTABLE=c:\ProgramData\chocolatey\lib\winflexbison\tools\win_bison.exe -DFLEX_EXECUTABLE=c:\ProgramData\chocolatey\lib\winflexbison\tools\win_flex.exe -DWITH_MT=ON -DWITH_SHARED_LIB=OFF -DWITH_CPP=OFF -DWITH_JAVA=OFF -DWITH_HASKELL=OFF -DWITH_PYTHON=OFF -DWITH_C_GLIB=OFF -DBUILD_TESTING=OFF -DBUILD_TUTORIALS=OFF -DBUILD_COMPILER=ON C:\build> cmake --build . --config Release
Using Docker for Windows, follow the instructions for building the compiler.
In both cases:
thrift.exe to your linux system where the signed tarball lives and rename it to thrift-1.0.0.exe (substitute the correct version, of course).Upload the release artifacts to the Apache Dist/Dev site. This requires subversion:
~$ mkdir -p dist/dev ~$ cd dist/dev ~/dist/dev$ svn co "https://dist.apache.org/repos/dist/dev/thrift" thrift ~/dist/dev$ cd thrift
Copy the tarball, windows compiler executable, and 8 additional signing files into a new directory for the release:
~/dist/dev/thrift$ mkdir 1.0.0-rc0 # copy the files into the directory ~/dist/dev/thrift$ svn add 1.0.0-rc0
The layout of the files should match the current release. Once done, add the release candidate and check it in:
~/dist/dev/thrift$ svn status # verify everything is correct ~/dist/dev/thrift$ svn commit -m "Apache Thrift 1.0.0-rc0 in dist dev" \ --username <apache-username> --password <apache-password>
Verify the release candidate artifacts are available at:
Send a voting announcement message to dev@thrift.apache.org following this template as a guide:
To: dev@thrift.apache.org Subject: [VOTE] Apache Thrift 1.0.0-rc0 release candidate --- All, I propose that we accept the following release candidate as the official Apache Thrift 1.0.0 release: https://dist.apache.org/repos/dist/dev/thrift/1.0.0-rc0/thrift-1.0.0-rc0.tar.gz The release candidate was created from the release/1.0.0 branch and can be cloned using: git clone -b release/1.0.0 https://github.com/apache/thrift.git The release candidates GPG signature can be found at: https://dist.apache.org/repos/dist/dev/thrift/1.0.0-rc0/thrift-1.0.0-rc0.tar.gz.asc The release candidates checksums are: md5: sha1: sha256: A prebuilt statically-linked Windows compiler is available at: https://dist.apache.org/repos/dist/dev/thrift/1.0.0-rc0/thrift-1.0.0-rc0.exe Prebuilt statically-linked Windows compiler GPG signature: https://dist.apache.org/repos/dist/dev/thrift/1.0.0-rc0/thrift-1.0.0-rc0.exe.asc Prebuilt statically-linked Windows compiler checksums are: md5: sha1: sha256: The source tree as ZIP file to be published via Github releases: https://dist.apache.org/repos/dist/dev/thrift/1.0.0-rc0/thrift-1.0.0-rc0.zip ZIP source tree GPG signature: https://dist.apache.org/repos/dist/dev/thrift/1.0.0-rc0/thrift-1.0.0-rc0.zip.asc ZIP source tree checksums are: md5: sha1: sha256: The CHANGES list for this release is available at: https://github.com/apache/thrift/blob/release/1.0.0/CHANGES.md Please download, verify sig/sum, install and test the libraries and languages of your choice. This vote will close in 72 hours on 2019-07-06 21:00 UTC [ ] +1 Release this as Apache Thrift 1.0.0 [ ] +0 [ ] -1 Do not release this as Apache Thrift 1.0.0 because...
If any issues are brought up with the release candidate, you will need to package another and reset the voting clock.
Voting on the development mailing list provides additional benefits (wisdom from Christopher Tubbs):
Send a message to dev@thrift.apache.org with the voting results. Use this template as a guide:
To: dev~thrift.apache.org Subject: [VOTE][RESULT] Release Apache Thrift 1.0.0 --- All, Including my own vote of +1 we have N binding +1 and no -1. The vote for the Apache Thrift 1.0.0 release is ***successful***. Thank you to all who helped test and verify.
Use svn to checkout the release part of thrift (similar to dev) and copy the files over from dev, matching the previous release structure:
~$ mkdir -p dist/release ~$ cd dist/release ~/dist/release$ svn co "https://dist.apache.org/repos/dist/release/thrift" thrift ~/dist/release$ cd thrift ~/dist/release/thrift$ mkdir 1.0.0 ~/dist/release/thrift$ cp -p ../../dev/thrift/1.0.0-rc0/* 1.0.0/ ~/dist/release/thrift$ svn status # verify everything is correct ~/dist/release/thrift$ svn commit -m "Apache Thrift 1.0.0 official release" \ --username <apache-username> --password <apache-password>
NOTE Once you check-in, you need to wait about a day for all the mirrors to update. You cannot send the announcement email or update the web site until the mirrors are updated.
Create and push a tag for the release, for example “v1.0.0”.
NOTE: All new releases must have the “v” prefix to satisfy third-party package managers (dlang dub, golang, etc..)
NOTE: You should sign the release tag. Since you already have a GPG signing key for publishing the Apache Release, you want to upload that key to your GitHub account. Once the key is known by GitHub you can sign the tag.
~/thrift$ # make sure you are on the release branch ~/thrift$ git checkout release/1.0.0 ~/thrift$ git pull ~/thrift$ git tag -s v1.0.0 -m "Version 1.0.0" ~/thrift$ git push --tags
NOTE: If you get the error “gpg failed to sign the data” when tagging, try this fix: export GPG_TTY=$(tty). Alternatively, it may be necessary to specify the -u <keyid> as an additional argument.
Create a new release from the GitHub Tags Page. Attach the statically built Windows thrift compiler as a binary here.
Merge the release branch into master. This ensures all changes made to fix up the release are in master.
~/thrift$ git checkout master ~/thrift$ git pull ~/thrift$ git merge release/1.0.0
The merge of 1.0.0 into master should proceed as a fast-forward since the 1.0.0 release branch. If there are discrepancies the best thing to do is resolve them and then submit a pull request. This pull request must be MERGED and not REBASED after the CI build is successful. You may want to do this yourself and mark the pull request as [DO NOT MERGE].
Update the ASF CMS content for thrift to include the new release. Note over time we will retire this in favor of including all documentation in the GitHub repository. The page with the variables that are important like the current release or distribution links is in trunk/lib/path.pm in the ASF CMS for thrift.
trunk.lib.path.pm.current_release and current_release_date to reflect the correct information.Make an announcement on the dev@ and user@ mailing lists of the release. There's no template to follow, but you can point folks to the official web site at https://thrift.apache.org, and to the GitHub site at https://github.org/apache.thrift.
Visit https://reporter.apache.org/addrelease.html?thrift and register it. You will get an automated reminder as the one who committed into dist. This informs the Apache Board of Directors of releases through project reports.
Create a local branch to bump the release number to the next anticipated release:
~/thrift$ git checkout -b bump-master ~/thrift$ build/veralign.sh 1.0.0 1.1.0
The veralign script will set the version number in all of the language packaging files and headers. You do not need to worry about the manually modified files at this time. You should however ensure everything is correct by looking at the diff.
Create a pull request to advance master to the next anticipated release.
In Apache Jira, select all tickets where the fix version is the release and the status is not closed (example) and use the bulk editing tool to close them.
FIXME Ask someone with admin access to Apache Jira to change the fixVersion in question from unreleased to released, for example: https://issues.apache.org/jira/browse/THRIFT-4686
Ensure that the Jira release page for the version has the same number of issues in the version as issues done, and that there are no issues in progress and no issues to do, and no warnings. Finally, mark it as released and set the date of the release.
security@apache.org if you are not sure if there are any CVEs to report.See https://thrift.apache.org/lib/ for the current status of each external package manager's distribution. The information below is from the 0.12.0 release:
This section needs to be updated with detailed instructions for each language, or pointers to the README.md files in each language directory with detailed release instructions for the given package management system.
lib/perl run the script build-cpan-dist.sh.If you have any questions email dev@thrift.apache.org.