blob: 8110885cc10c659e99144a20789b831d4489b43c [file] [view]
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Apache Flume: How to Release
=============================
This document describes how to make a release of Flume. It is a work in
progress and should be refined by the [Release
Manager](http://incubator.apache.org/guides/releasemanagement.html#glossary-release-manager)
(RM) as they come across aspects of the release process not yet
documented here.
NOTE: For the purpose of illustration, this document assumes that the
version being released is 1.0.0, and the following development version
will become 1.1.0.
Prerequisites
-------------
### Policy documents
The policy on releasing artifacts from an incubating Apache project is
stated in the [Guide to Release Management During
Incubation](http://incubator.apache.org/guides/releasemanagement.html#glossary-release-manager).
While Flume is now a top-level project, that guide is still a good
resource for new release managers since it contains what are considered
to be best practices.
The Release Manager (RM) must go through the policy document to
understand all the tasks and responsibilities of running a release.
### Give a heads up
The RM should first create an umbrella issue and then setup a timeline
for release branch point. The time for the day the umbrella issue is
created to the release branch point must be at least two weeks in order
to give the community a chance to prioritize and commit any last minute
features and issues they would like to see in the upcoming release.
The RM should then send the pointer to the umbrella issue along with the
tentative timeline for branch point to the user and developer lists. Any
work identified as release related that needs to be completed should be
added as a subtask of the umbrella issue to allow users to see the
overall release progress in one place.
Send out an e-mail like this:
To: dev@flume.apache.org
Subject: [DISCUSS] Flume 1.7 release plan
We have almost 100 commits since 1.6.0, and a bunch of new features
and improvements including a Taildir Source, many upgrades,
performance increases, plus a lot of fixes and documentation
additions.
It seems to me that it's time to cut a 1.7 release soon. I would be
happy to volunteer to RM.
Below is the list of tickets that have patches submitted and are
scheduled for 1.7.0:
https://issues.apache.org/jira/browse/FLUME-2998?jql=project%20%3D%20FLUME%20AND%20status%20%3D%20%22Patch%20Available%22%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%20v1.7.0 (18 tickets)
There are more tickets that have no patch submitted:
https://issues.apache.org/jira/browse/FLUME-2998?jql=project%20%3D%20FLUME%20AND%20issuetype%20in%20standardIssueTypes()%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%20v1.7.0 (30 tickets)
I suggest tracking the release process using the JIRA at https://issues.apache.org/jira/browse/FLUME-2924
If this all sounds OK, I'd like to suggest targeting the week of
October 10 for a first RC. (Also, anything that is non-trivial should
get in by October 7. Any feature or big change that is not committed
by then would be scheduled for the next release.)
That should leave enough time for people to get moving on patches and
reviews for their documentation improvements, critical bug fixes, and
low-risk enhancements, etc.
Also, I would like to propose October 7 (one week from today) for the
branch date.
Please let me know your thoughts.
### JIRA cleanup
1\. Before a release is done, make sure that any issues that are fixed
have their fixVersion setup correctly. Run the following JIRA query to
see which resolved issues do not have their fix version set up
correctly:
**`JIRA: project = Flume and Resolution = Fixed and fixVersion is EMPTY`**
The result of the above query should be empty. If some issues do show up
in this query that have been fixed since the last release, please
bulk-edit them to set the fix version to the version being released.
2\. Since the JIRA release note tool will list all JIRAs associated with
a particular fixVersion regardless of their Resolution, you may want to
remove fixVersion on issues declared as "Duplicate", "Won't Fix", "Works
As Expected", etc. You can use this query to find those issues, then
just bulk edit them and leave the fixVersion field blank when editing
them. (Note that you will need to replace "v1.4.0" below with the
version you are trying to release.)
**`JIRA: project = Flume AND (Status != Resolved OR Resolution != Fixed) AND fixVersion = v1.6.0`**
3\. Finally, check out the output of the **[JIRA release note
tool](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311321)**
to see which JIRAs are included in the release, in order to do a sanity
check.
### Monitor active issues
It is important that between the time that the umbrella issue is filed
to the time when the release branch is created, no experimental or
potentially destabilizing work is checked into the trunk. While it is
acceptable to introduce major changes, they must be thoroughly reviewed
and have good test coverage to ensure that the release branch does not
start of being unstable.
If necessary the RM can discuss if certain issues should be fixed on the
trunk in this time, and if so what is the gating criteria for accepting
them.
Creating Release Artifacts
--------------------------
### Communicate with the community
1\. Send an email to dev@flume.apache.org to
Notify that you are about to branch.
Ask to hold off any commits until this is finished.
2\. Send another email after branching is done.
### Update the LICENSE file
The release manager is responsible for updating the LICENSE file to
provide accurate license information for all binary artifacts contained
in the codebase. This is a tedious and painstaking process, and must be
performed for each release that includes a binary artifact.
### Prepare branches and create tag
In this section, the release is X.Y.Z (e.g. 1.3.0)
1\. Checkout the trunk.
git clone http://git-wip-us.apache.org/repos/asf/flume.git flume
2\. Update CHANGELOG in the trunk to indicate the changes going into the
new version.
The change list can be swiped from the **[JIRA release note
tool](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311321)**
(choose the "text" format for the change log). See JIRA Cleanup above to
ensure that the release notes generated by this tool are what you are
expecting. Additionally, in the last two releases we have re-ordered the
issues to be New Features, Improvements, Bugs, etc as by default the
least important items are at the top.
3\. Update the "version" value in RELEASE-NOTES in trunk to X.(Y+1).0:
vim RELEASE-NOTES
git add RELEASE-NOTES
git commit RELEASE-NOTES -m "Updating Flume version in RELEASE-NOTES for release X.Y.Z"
4\. Create a branch for the X.(Y+1) release series
E.g. flume-1.5 should be branched off of trunk
git checkout trunk
git checkout -b flume-X.(Y+1)
4\. Update the "version" value of all pom.xml and documentation files in
trunk to X.(Y+1).0-SNAPSHOT
git checkout trunk
find . -name pom.xml | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.$(Y+1).0-SNAPSHOT/"
find flume-ng-doc/ -name "*.rst" | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.$(Y+1).0-SNAPSHOT/"
git add .
git commit -m "Updating trunk version to X.$(Y+1).0-SNAPSHOT for Flume X.Y.Z release"
5\. Checkout the release branch and remove -SNAPSHOT from the release
branch poms and docs and commit:
git checkout flume-X.Y
find . -name pom.xml | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.Y.0/"
find flume-ng-doc/ -name "*.rst" | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.Y.0/"
git add .
git commit -m "FLUME-XXXX: Removing -SNAPSHOT from X.Y branch"
6\. Ensure RELEASE-NOTES has the appropriate version and description of
the release.
7\. Push the branching changes upstream
git push -u origin trunk:trunk
git push -u origin flume-X.Y:flume-X.Y
8\. Tag a release candidate (in the example below, RC1):
git tag -a release-X.Y.Z-rc1 -m "Apache Flume X.Y.Z RC1"
git push origin release-X.Y.Z-rc1
If an rc2, rc3 etc is needed, simply create a new rc tag:
git tag -d release-X.Y.Z-rc2
git push origin release-X.Y.Z-rc2
### Preparing to sign the artifacts
All artifacts must be signed and checksummed. In order to sign a release
you will need a PGP key. You should get your key signed by a few other
people. You will also need to recv their keys from a public key server.
See the [Apache release signing](https://www.apache.org/dev/release-signing)
page for more details.
1\. Add your key to the
[KEYS](https://dist.apache.org/repos/dist/release/flume/KEYS) file:
(gpg --list-sigs <your-email> && gpg --armor --export <your-email>) >> KEYS
And commit the changes.
### Generating and signing the source artifacts
Generation of the source artifact is created by the maven-assembly-plugin in the package phase.
Signing is done by the maven-gpg-plugin using the `sign` profile in the verify phase
Checksum generation for the source and binary artifacts is done by checksum-maven-plugin
in the verify phase, an SHA512 checksum is generated
mvn clean package -Psign -DskipTests
The generated artifacts can be found in the `flume-ng-dist/target` directory
If you would like to create it manually, the previous process is:
There is a script in the Flume source tree for generating and signing the Flume
source artifacts. Once the release candidate is tagged, generate the source
release with the following steps.
1\. From the top of the Flume source tree, create a directory for the artifacts
and then generate them:
mkdir ./source-artifacts
./dev-support/generate-source-release.sh X.Y.Z release-X.Y.Z-rc1 ./source-artifacts/
The artifacts will be placed in the directory you specify (in this case,
`./source-artifacts`)
### Testing the source tarball
1\. Unpack the source tarball
tar xzvf apache-flume-X.Y.Z-src.tar.gz
2\. Do a full build inside the source tarball. Allow all unit tests &
integration tests to run and also include the docs.
cd apache-flume-X.Y.Z-src
export LC_ALL=C.UTF-8 # Required to build the javadocs on some platforms and in some locales
mvn clean install -Psite -DskipTests
3\. Verify that the HTML docs that should have been generated inside the
binary artifact under /docs are there and do not have rendering errors.
### Generating, signing, and deploying the binary artifacts
Maven is configured to generate, sign, and deploy the binary artifacts
automatically. Use the following steps to do that:
1\. Create and sign the artifacts, including site docs. This pushes the
signed artifacts to the ASF staging repository.
In order to do this, you will need a settings.xml file with your
username and password for the ASF staging repository. Typically this is
placed in \~/.m2/settings.xml and might look something like this:
<settings>
  <servers>
    <server>
      <id>apache.staging.https</id>
      <username>your_user_id</username>
      <password>your_password</password>
    </server>
  </servers>
</settings>
2\. Once your settings.xml file is correct, run the following from the
Flume source directory to generate and deploy the artifacts:
mvn clean deploy -Psite -Psign -DskipTests
This will sign, checksum, and upload each artifact to Nexus.
Note: the checksum files will not be mirrored; They should be downloaded
from the main apache dist site.
Do the same for javadoc and source artifacts:
mvn javadoc:jar gpg:sign deploy:deploy
mvn source:jar gpg:sign deploy:deploy
3\. Publish Staging repository
Login to <https://repository.apache.org> and select Staging Repositories
on the left under Build Promotion.
Select org.apache.flume from the list of repositories, verify it looks
OK, and then click Close using "Apache Flume X.Y.Z" as the description
to allow others to see the repository. Note that the staging repository
will have a numeric id associated with it that will be used later
4\. Copy the source artifacts you built locally to people.apache.org
$ rsync -e ssh -av source-artifacts/apache-flume-X.Y.Z-src.tar.gz* people.apache.org:public_html/apache-flume-X-Y.Z-rcN/
5\. Copy the binary artifacts you deployed via Maven to people.apache.org
$ ssh people.apache.org
$ cd public_html
$ mkdir apache-flume-X.Y.Z-rcN
$ cd apache-flume-X.Y.Z-rcN
$ wget --no-check-certificate https://repository.apache.org/content/repositories/orgapacheflume-XXXX/org/apache/flume/flume-ng-dist/X.Y.Z/flume-ng-dist-X.Y.Z-bin.tar.gz{,.{asc,sha512}}
$ for file in flume-ng-dist-*; do mv $file $(echo $file | sed -e "s/flume-ng-dist/apache-flume/g");done
Running the vote
----------------
### Call for dev list votes
Send an email to dev@flume.apache.org list. For example,
To: dev@flume.apache.org
Subject: [VOTE] Release Apache Flume version X.Y.Z RC1
This is the XXXXX release for Apache Flume as a top-level project,
version X.Y.Z. We are voting on release candidate RC1.
It fixes the following issues:
<Link-to-CHANGELOG-in-the-tag>
*** Please cast your vote within the next 72 hours ***
The tarball (*.tar.gz), signature (*.asc), and checksums (*.sha512)
for the source and binary artifacts can be found here:
https://people.apache.org/~mpercy/flume/apache-flume-X.Y.Z-RC1/
Maven staging repo:
https://repository.apache.org/content/repositories/orgapacheflume-XXXXX/
The tag to be voted on:
https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=<commit-hash-of-the-tag>
Flume's KEYS file containing PGP keys we use to sign the release:
https://svn.apache.org/repos/asf/flume/dist/KEYS
You need 3 +1 votes from Flume PMC members for a release.
When the vote gets cancelled or passes, send out an e-mail like:
This vote is cancelled.
I'm about to send an e-mail about RC2.
or (if the vote passes):
This vote is now closed.
I will send out the results in a separate email.
Thanks to all who voted!
### Send results of a successful vote
After a successful RC vote has passed, send an email to dev@flume.apache.org list. For example,
To: dev@flume.apache.org
Subject: [RESULT] Flume 1.7.0 release vote
The release vote for Apache Flume 1.7.0 has been completed in this
thread: https://lists.apache.org/thread.html/3cf114125dbea6e662b69d1312c34184690af327900e8dcccea5edde@%3Cdev.flume.apache.org%3E
The vote has received 3 binding +1 votes from the following PMC members:
Mike Percy
Hari Shreedharan
Brock Noland
Four non-binding +1 votes were received from:
Denes Arvay
Bessenyei Balázs Donát
Lior Zeno
Attila Simon
No +0 or -1 votes were received.
Since three +1 votes were received from the PMC with no -1 votes, the
vote passes and Flume 1.7.0 RC2 will be promoted to the Flume 1.7.0
release.
Thanks to all who voted!
Rolling out the Release
-----------------------
### Upload the artifacts
#### Source and convenience artifacts
svn checkout https://dist.apache.org/repos/dist/release/flume dist-flume
cd dist-flume
mkdir 1.0.0
cp <all release artifacts including asc/checksum files> 1.0.0/
svn rm stable # remove older version link
ln -s 1.0.0 stable # update stable version link
svn add stable 1.0.0
svn commit
It may take up to 24 hours for all mirrors to sync up.
#### Deploy Maven artifacts
General instructions on how to deploy the poms and jars to Maven Central
can be found at the [Apache page on publishing maven
artifacts](https://www.apache.org/dev/publishing-maven-artifacts.html).
**Flume-specific instructions:**
After you get your `settings.xml` file configured correctly (see above
link) then you can run:
`mvn clean deploy -Psite -DskipTests -Papache-release` to push the
artifacts to the staging repo.\
You will need to go to the Apache Maven Repository @
<http://repository.apache.org/> and log in with your Apache LDAP
credentials\
Once you log in you will see Build Promotion &gt; Staging Repositories
on the left hand side\
You will want to edit the Flume artifacts that you don't want to push to
Maven (you can delete stuff like the release tarball)\
Click Close to make the atrifacts available on the Staging repository.\
To push to Central, you click Release.
### Announce the release
Send an email to announce@apache.org (the from: address must be
@apache.org). For example,
To: announce@apache.org, user@flume.apache.org, dev@flume.apache.org
Subject: [ANNOUNCE] Apache Flume 1.2.0 released
The Apache Flume team is pleased to announce the release of Flume
version 1.2.0.
Flume is a distributed, reliable, and available service for efficiently
collecting, aggregating, and moving large amounts of log data.
This release can be downloaded from the Flume download page at:
http://flume.apache.org/download.html
The change log and documentation are available on the 1.2.0 release page:
http://flume.apache.org/releases/1.2.0.html
Your help and feedback is more than welcome. For more information on how
to report problems and to get involved, visit the project website at
http://flume.apache.org/
The Apache Flume Team
### Update the website
1. Checkout <https://svn.apache.org/repos/asf/flume/site/trunk>
2. Add a page to the `content/sphinx/releases` directory for the new
release, with the Changelog and links to the documentation (refer
previous release pages for details. The documentation should simply
use the same paths as the previous releases with correct versions -
the documentation will be checked in directly to the production
website as mentioned below).
3. Update `content/sphinx/releases/index.rst` to update the pointer to
the latest release.
4. Update `content/sphinx/download.rst` to also point to the
latest release.
5. Update `content/sphinx/index.rst` as necessary to add a News item to
the home page.
6. Copy the release version of the FlumeUserGuide.rst and
FlumeDeveloperGuide.rst to the Documentation directory.
7. Commit the changes to svn. Go to <https://cms.apache.org/flume/> -
Stage and publish the changes.
8. Checkout
<https://svn.apache.org/repos/infra/websites/production/flume>.
9. Create a directory with the current release's directory name under
content/releases/content (e.g., content/releases/content/1.3.1)
10. Copy both the HTML and PDF versions of the user guide and developer
guide (manually generate the PDF from the HTML files), and the
javadocs (apidocs directory) into this directory.
11. Commit the changes to svn. Done!