blob: f0e920e51d0091f868bceefbf5fabc0c605ca9df [file] [log] [blame]
SPAMASSASSIN DEVELOPMENT SNAPSHOT PROCEDURE
===========================================
- cd to the directory for the codebase you want the devel tree to
come from
ssh spamassassin.zones.apache.org
cd [checkedoutdir]
NOTE: SpamAssassin.Zones is a crashed box replaced by SpamAssassin-vm.
A copy of the sabuildtools is in
/root/zonestorage-restore/spamassassin/export/home/kmcgrail/sabuildtools
- ensure the required code and data is available for the build scripts:
$HOME/sabuildtools
All can be copied (or symlinked) from ~jm on spamassassin.zones.apache.org if
required.
- ensure your PATH is correct:
PATH=$HOME/sabuildtools/bin:$PATH
- run "./build/update_devel" to build the tar.gz files.
- next run "./build/update_devel_rules" to build the rules tarball,
or run "./build/repackage_latest_update_rules" to repackage the
latest file on updates.spamassassin.org.
- by default, they're written to ~/public_html/devel/ .
Copy them to wherever you want, yourself.
- KAM has included all of these on one of his devel systems with perl
5.8.6 (the earliest supported version of Perl as of 4/28/2015). One
note is that you need webmake 2.4 from
http://webmake.taint.org/devel/HTML-WebMake-2.4.tar.gz
Also, a copy is mirrored at www.pccc.com/downloads/SpamAssassin
SPAMASSASSIN RELEASE PROCEDURE
==============================
- First off, log in to spamassassin.zones.apache.org for some preliminary tool
checks.
ssh spamassassin.zones.apache.org
- ensure the required code and data is available for the build scripts:
see above.
- ensure your PATH is correct:
PATH=$HOME/sabuildtools/bin:$PATH
- ensure ExtUtils::MakeMaker is version 6.46 or later, to create
a detailed-enough META.yml (TODO: this should not be required.)
perl -MExtUtils::MakeMaker -e 'print $ExtUtils::MakeMaker::VERSION'
- In your own, writable, checkout (this may be on another host instead
of spamassassin.zones.apache.org, such as your home machine):
Run "svn status" to ensure you're building/releasing from a pristine
checkout:
cd [checkedoutdir]
svn status
The output should look like this:
? rules/SIQ.pm
? rules/sandbox-hstern.pm
? rules/sandbox-felicity.pm
(ie., no "M" or "C" files; any files marked "M" have been locally
modified, and should be "svn revert"ed before you continue.)
- Consider updating the TLD list in 20_aux_tlds.cf. As of 4/6/2015,
this is not automated but bug 7165 is open for this purpose.
- edit lib/Mail/SpamAssassin.pm and comment the $IS_DEVEL_BUILD
line. Ensure the correct version number is present in $VERSION
and @EXTRA_VERSION.
Prereleases: formatted like this: "pre4" -- in $EXTRA_VERSION, and
$IS_DEVEL_BUILD is left as 1 instead of commented. When referred to in
full, it's "3.1.0-pre4".
Release candidates: formatted like this: "rc2" -- in $EXTRA_VERSION, and
$IS_DEVEL_BUILD is commented. When referred to in full, it's "3.1.0-rc2".
Full releases: comment out @EXTRA_VERSION
[NOTE: when editing files in these instructions, you may have to
use another checkout somewhere else, check in the changes there,
then 'svn update' in the release account -- since I think it
builds from a read-only checkout.]
- Ensure the new version number takes hold:
make clean ; perl Makefile.PL < /dev/null ; make
./spamassassin -L -t < sample-nonspam.txt | grep X-Spam-
Verify that the X-Spam-* headers use the correct version string,
without an SVN revision number (those are only for dev and pre builds).
NOTE: If the date is old, don't worry, it will change later with the
relbuild checkout.
- create the Changes file. Do this step on your own checkout, as
the release checkout is read-only.
MAKE CERTAIN THIS IS RUN IN THE ROOT OF YOUR CHECKOUT!
There are two options here:
- For releases on a maintainance branch (e.g. 3.0.1, .2, etc.):
TZ=UTC svn log --non-interactive --stop-on-copy > Changes
This will output all of the changes since the .0 release in the
current branch (the last copy -- note, if a copy was done
afterwards (move between repositories, etc, you'll need to
modify that command).
- For releases on the trunk (e.g. a .0 release):
TZ=UTC svn log -r HEAD:1567124 --non-interactive > Changes
r497472 was the release of 3.2.0-pre1; r233108 was the start of 3.2.0,
just after the 3.1.0 branch was created; r46030 was the start of the
3.1.0 work (created 3.0 branch); replace with the correct rev number
for the point you want to start at if different. I used r1149751 to
produce the first 3.4.0-pre release, for example. r1567124 is the
3.4.0 release tag revision. r1676613 is the 3.4.1 release tag
revision.
To find the release tag revision, use the tags website, i.e.:
https://svn.apache.org/repos/asf/spamassassin/tags/
NOTE: 3.4.1 is currently on trunk and will be branched to 3.4 because
3.4 branch was not used.
- Exclude some automated changes from the logs:
cat Changes | perl -ne ' BEGIN{$/=("-"x72)."\n"}
chomp; @ln=split/^/;
next if @ln==3 && $ln[2]=~/^\s*(updated scores for revision.*|promotions validated|auto-generated rules|)\s*$/;
print "@ln$/"' > Changes.tmp
- Word wrap the Changes file with something like
fold -sw 74 Changes.tmp > Changes && rm -f Changes.tmp
- run "make distcheck" to ensure all files are included in the
distribution that should be, and to ensure all files that are listed
in the MANIFEST also exist in SVN.
make distcheck
- run "make disttest" to ensure all default tests pass once the
distribution is fully packaged. (this'll take a minute or two.)
make disttest < /dev/null
- run the release test suite, which forces all tests to run and also
applies an additional, more stringent set of tests. This may require
"sudo" privileges on your test box, and unusual CPAN modules like
"Devel::SawAmpersand", so you might have to skip some of this or ignore
some spurious failures. For example, I don't have re2c or dcc installed
on my devel system so I fail xt/50_dcc & xt/50_sa_compile.
Additionally, this will run tests such as the spamd prefork which will
kill all running spamd processes. This release test suite is intended
for a development environment.
make test TEST_FILES="xt/*.t"
Note: As of 2014-02-04, all of these tests can pass test
with no tests skipped except dcc.t which doesn't
properly test if dcc is not available - Bug 6944.
- Check in the updated Changes file, and SpamAssassin.pm.
svn commit -m "preparing to release X.Y.Z"
(If you are privately preparing a security release, and don't wish to
perform commits to public SVN repos, you can defer this step until
later.)
- SVN tag the release files. This is done using "svn copy".
- For a maintainance release (x.y.1, x.y.2):
vers=3_4_2
Then run:
repo=https://svn.apache.org/repos/asf/spamassassin
svn delete -m "replaced old tag" $repo/tags/spamassassin_release_$vers
svn copy -m "creating tag for release $vers" \
$repo/branches/3.4 $repo/tags/spamassassin_release_$vers
- For a trunk release (x.y.0):
vers=3_4_1
repo=https://svn.apache.org/repos/asf/spamassassin
svn delete -m "replaced old tag" $repo/tags/spamassassin_release_$vers
svn copy -m "creating tag for release $vers" \
$repo/trunk $repo/tags/spamassassin_release_$vers
- For a trunk release candidate (x.y.0-rc1), pre-release (x.y.0-preZ) or alpha (x.y.0-alphaZ):
vers=3_4_1_rc_2
Then run:
repo=https://svn.apache.org/repos/asf/spamassassin
svn delete -m "replaced old tag" $repo/tags/spamassassin_release_$vers
svn copy -m "creating tag for release $vers" \
$repo/trunk $repo/tags/spamassassin_release_$vers
This will do a completely server-side tagging (which is the same as
a branch really) of whatever the latest branch revision to be the new
base of the tag release.
(If you are privately preparing a security release, and don't wish to
perform commits to public SVN repos, you can defer this step until
later.)
- In your checkout on spamassassin.zones.apache.org (may be read-only):
Check out the code from the tag you just made:
#These should be set from the previous command unless you are on a
# different server
#vers=3_4_1
#repo=https://svn.apache.org/repos/asf/spamassassin
rm -rf ~/relbuild
svn co $repo/tags/spamassassin_release_$vers ~/relbuild
cd ~/relbuild
(If you are privately preparing a security release, you will need
to ensure that the code in this build directory matches up using
some other mechanism, since SVN is public.)
- run "./build/update_stable" to build the tar.gz files.
PATH=$HOME/sabuildtools/bin:$PATH
./build/update_stable
WARNING: this will use the most recently-created "sa-update" tarball for
the rules tgz, so if changes have been committed to the "rules" dir in
SVN since then, they will not be released!
if you get "gpg-agent" errors about not connecting to the PIN entry
module, you didn't update your $PATH.
- run "./build/repackage_latest_update_rules" to repackage the
latest rules file for the purpose of publishing a starter rule file
for people who can't run sa-update:
PATH=$HOME/sabuildtools/bin:$PATH
./build/repackage_latest_update_rules
- test the tar.gz and zip files! redo until they work!! ;)
- Write the release announcement mail! This is a simple matter of copying
the previous release's announcement, updating the version numbers and
links, fixing the MD5 and SHA1 checksums in this mail, and summarising
the important changes from the Changes file.
cp build/announcements/3.4.0.txt build/announcements/3.4.1.txt
svn add !$
vi !$
NOTE: Here's a quick example to concat the MD5 or SHA1 checksums:
ls *3.4.1.* | grep md5 | xargs cat --
ls *3.4.1.* | grep sha1 | xargs cat --
- If there are any issues to note, make sure to edit the UPGRADE file.
Every x.x.0 release at a minimum should include a section.
- Check the README file for any items to change.
- (for any rc, prerelease, or full release) Place the tarballs in a
discreet location (discreet means not linked from the "downloads" page
of the website, but included in the vote mail) and request a vote on the
development mailing list to make the release. Post the URL,
md5sums/sha1sums, and proposed release announcement mail to the dev
list. The default location -- ~/public_html/devel/ , or
http://people.apache.org/~jm/devel/ , qualifies as "discreet".
If you build on the zone, you will need to scp them over:
scp ~/public_html/devel/Mail-SpamAssassin-* people.apache.org:public_html/devel/
While doing this, also upload a copy of the proposed release
announcement to the website.
As of 2014-02-04, this is done by adding announcement to the devel
directory for the website and running webmake. See main.wmk in site svn.
Pre-releases and RCs require just lazy consensus -- ie. no objections.
- (for a pre or rc release), - upload .tar.gz (not .bz2) tarball to CPAN
at http://pause.cpan.org/:
https://pause.perl.org/pause/authenquery?ACTION=add_uri
See notes below for issues on indexing but the tar should be named in a
manner that CPAN will process it as a Developer Release per
http://www.cpan.org/modules/04pause.html#developerreleases
As of 2014-02-05, adding -TRIAL to the filename should work. For example:
Mail-SpamAssassin-3.4.1-rc2-TRIAL.tar.gz
- (for a full release) 3 +1's votes from PMC members are required to
proceed with an official (not pre-release) ASF software release.
- If you've been doing some steps in private, for a security release,
now's the time to perform them publicly; tagging etc.
- Edit lib/Mail/SpamAssassin.pm, and uncomment the $IS_DEVEL_BUILD line.
SVN commit it (and the release announcement file from above, if
you made one):
svn commit -m "X.Y.Z RELEASED"
- [X.Y.0 RELEASES ONLY]: Now, start the new development codebase. For
minor updates of a 2.x tree (e.g. 2.x.1, 2.x.2), you don't need to
branch; for major updates (2.x.0) you need to create a new development
branch, off the trunk.
repo=https://svn.apache.org/repos/asf/spamassassin
svn copy $repo/tags/spamassassin_release_3_4_1 $repo/branches/3.4 \
-m 'Creating 3.4 branch based on 3.4.1 so that trunk can go to 4.0'
"trunk" is SVN's concept of head. Typically, our branches are named
for their minor version number. In the example above, 3.4 is the
branch for the stable 3.4.x releases.
- In the new development codebase, edit lib/Mail/SpamAssassin.pm
and bump the $VERSION line to the correct version.
- then, commit the new version number to the new branch:
svn commit -m "X.Y.N devel cycle started"
(where X.Y.N is the new version number)
- [X.Y.0 RELEASES ONLY]: in trunk, bump the version to the correct
version in the following files (just as above for the branch):
- lib/Mail/SpamAssassin.pm
- In build/mkupdates/run_part2 change versions="X.Y.Z" to the new
development version (A.B.C).
- [X.Y.0 RELEASES ONLY]: NOTE: This may change, make sure this info is
still accurate before changing this (preferrably find out BEFORE doing
an X.Y.0 release. Update the mkupdates stuff:
# - Create an empty file for the new version number in
# /var/named/updates.dev.spamassassin.org.d/ on the zone and chown to
# updatesd and chgrp to others.
#
# NOTE: I don't believe this does anything KAM: 2014-02-11 and the
# directory doesn't exist.
- Add "$INCLUDE /var/named/updates.spamassassin.org.d/X.Y+1.0" to the
/var/named/spamassassin.org zone file OR use a CNAME (see below)
NOTE: Per bug 6644, using a CNAME for 3.4.1, 3.4.2, 4.0.0 to 3.3.2
- commit the changes, update the zone (if not doing above on the zone -
NOTE: See /var/named/README) and tick the zone file using build/mkupdates/tick_zone_serial
- !WARNING! After the next step, the version number will be considered
"burned". The number is locked for this particular code. The same
number cannot be used for a future different release. So make sure
you're happy before you go on!
If you need to redo something, re-comment the $IS_DEVEL_BUILD line,
revert the $VERSION bump, and go back to 'Ensure the new version number
takes hold'. You can retag with the normal 'svn copy' command used in
'SVN tag the release files', even if that tag already existed; but be
sure to check in another commit message to note what happened, e.g.:
svn commit -m "oops, had to redo: THIS IS THE REAL X.Y.Z RELEASE"
- Release a new rules update matching the released code:
#2014-02-11 switched to su - instead of sudo because of .ssh and .svn
stored config items
ssh spamassassin.zones.apache.org
su - updatesd
cd /home/updatesd/svn/spamassassin/build/mkupdates
./update-rules-3.3 3.4
NOTE: Is this needed with our rule update process? not convinced it
is... 2015-04-28
- publish the tarballs
ASF bylaws prohibit the distribution of files claiming to be
releases from the website, without 3 PMC +1 votes, so you need to
make it clear that this is an unofficial "test build" by placing it
in your public_html dir:
http://people.apache.org/~kmcgrail/devel/ for example
NOTE: It's also feasible that you could publish via SVNPubSub as noted
below but using the SVN repository at:
https://dist.apache.org/repos/dist/dev/SpamAssassin
The choice is up to the Release Manager for these test builds.
For full release builds, you need to publish using SVNPubSub.
This is a simple matter of committing the new artifacts and signature
files to https://dist.apache.org/repos/dist/spamassassin once a
release vote passes.
[note: Symlinks may not work on all mirrors even though there may be
documentation to the contrary.]
As documented at http://www.apache.org/dev/release.html, SpamAssassin
as of 2014-02-26 publishes tarballs by manipulating the repo on svn.
svn co https://dist.apache.org/repos/dist/release/spamassassin/ release
- Manipulate the files to add the new version artifacts to the source dir
- Remove tar-balls for now-obsolete versions from dist:
Only the most recent version of each supported release line should be
stored. The archives will auto update.
See http://www.apache.org/dev/release#archived.
- Commit
- Wait 24 hours before you announce to allow mirrors to update
(NOTE: it may be better to wait a while before doing this; it appears to
take a while for the spamassassin.apache.org website mirror script to
run, so the old downloads page will stick around for a while before it's
safe to do so.)
- (for full release builds) update the main website "downloads" page:
ssh spamassassin.zones.apache.org
cd /var/www/buildbot.spamassassin.org/staging/website
vi main.wmk
See the comment at the top of the file; you'll need to change at
least one variable ("relversion") to the new version number, and
possibly more, depending on if this is the first release of a new
release line.
When changing the timestamp in the downloads.cgi link URL, ensure you
use a time 2 hours from now. The ASF websites are synced internally to
the public HTTP servers on a bi-hourly basis, so it may take up to this
long for a file-download link to become valid and not a 404.
- rebuild the SpamAssassin website with webmake:
PATH=$HOME/sabuildtools/bin:$PATH
cd /var/www/buildbot.spamassassin.org/staging/website
webmake -F
- update the 'doc' tree in the SpamAssassin website
WARNING: if you're moving to a new major release, x.y.0, you need to edit
'build/update_website_docs' beforehand and set the "vers" line. And if
you are running on a server other than buildbot (NOTE: this works on the
spamassassin-vm.apache.org box), the PERL and WEBDIR
vars may need tweaking:
cd /var/www/buildbot.spamassassin.org/staging/website
svn up
svn delete --force full/3.4.x
svn commit -m "removing old doc tree from website" full
#Checkout the current release
cd /tmp
svn checkout http://svn.apache.org/repos/asf/spamassassin/tags/spamassassin_release_3_4_1/ release
cd /tmp/release
build/update_website_docs
cd /var/www/buildbot.spamassassin.org/staging/website
svn add full/3.4.x
svn commit -m "updating new doc tree on website" full
- and check the entire website into SVN (see bug 4998 for reasons why).
Note that you may need to resolve conflicts and ensure the contents of
'full/3.3.x' are all added successfully:
svn up
svn commit -m "added new release to website"
- update the tag used to point to "current release":
repo=https://svn.apache.org/repos/asf/spamassassin
svn delete -m "updating for new release" $repo/tags/spamassassin_current_release_3.4.x
svn copy -m "updating for new release" $repo/tags/spamassassin_release_3_4_1 $repo/tags/spamassassin_current_release_3.4.x
- upload release .tar.gz (not .bz2) tarball to CPAN at http://pause.cpan.org/:
https://pause.perl.org/pause/authenquery?ACTION=add_uri
(Note that recently, PAUSE has started indexing sub-modules under
Mail::SpamAssassin::, and it can't deal with our multi-maintainer
setup. You may receive mail indicating that "indexing failed" after
the upload; as long as the main Mail::SpamAssassin module was indexed
correctly, this is fine. However, it would help if you could visit
https://pause.perl.org/pause/authenquery?ACTION=share_perms
select 3.1 ("Make someone else co-maintainer"), and ensure that the
other releasers (JMASON, DOS, FELICITY, others?) all have permissions as
'co-maintainer' on the full set of your listed modules in the
"Mail::SpamAssassin::" namespace.
- announce on the users, dev, and announce mailing lists using the
previously-prepared release announcement.
NOTE: you must send this mail with a "From:" address @apache.org,
otherwise it'll be bounced by the ASF's custom spam filtering
rules. See [1] below for more requirements for the announce email.
# IGNORING 2014-02-11 - NOT BOTHERING WITH FRESHMEAT - TOO OUT OF DATE AND
# NOW FREECODE
#
#- Before doing the next step, run through the release summary mail, and
# write up a really short summary of the important changes in
# human-readable format. This should be less than 600 chars to fit into
# Freshmeat's format, and to be easily understandable.
#- announce to Freshmeat at http://freshmeat.net/:
#
# http://freshmeat.net/projects/spamassassin/releases/new
- SourceForge - We have an account with SF and haven't published to it in
some time. Need to consider doing so.
- Approve the posting to the announce list (the list admins will get a
mail indicating how to do this.)
- Add the new version to the Bugzilla versions list:
http://issues.apache.org/SpamAssassin/editversions.cgi?product=Spamassassin&action=add
- Add a milestone for the next version to the Bugzilla milestones list:
https://issues.apache.org/SpamAssassin/editmilestones.cgi?action=add&product=Spamassassin
(use a sortkey of "0")
- Change the sortkey for the previous release's milestone from "0" to "10":
https://issues.apache.org/SpamAssassin/editmilestones.cgi?product=Spamassassin
- When releasing 3.4.0, we also did the following tasks which are less
documented in this README:
- Reviewed the CREDITS to update inactive Committers
- Reviewed the CREDITS to update Copyright
- Reviewed the website and wiki for less obvious items like:
- updating things that refered to 3.3 to 3.4 such as SVN Checkout
instructions.
- The how to Upgrade Instructions
- Updated http://en.wikipedia.org/wiki/SpamAssassin
- Reviewed the Project Branding Report Checklist
- Issues not resolved with the release of 3.4.0 2014-02-11 and still not
resolved with the release of 3.4.1 2015-04-28
- We updated to 3.4.0 and 3.4.1 is the current trunk version. Need to
confirm what needs to change on the updatesd scripts to make this
happen.
- Do we stop 3.3.X update releases? Or do we just make sure we aren't
running masscheck for that version and release the updates with proper
ifplugin loops for any new features?
-------------
[1]
Hello PMCs --a quick reminder that messages sent to announce@apache.org
feature prominently in the weekly Apache News Round-ups [1].
In order for your announcement be happily moderated through, don't forget
to:
0) write "[ANNOUNCE]", "[ANN]", or "[SECURITY]" in the subject line as
appropriate
1) send the email in PLAINTEXT --all else will be rejected
2) include the appropriate URL for downloading the release or more
information regarding the announcement
3) include the DOAP! Whilst you may know what your project is, there's a
chance that others don't, particularly those with funky names/acronyms.
Hone your message and polish that one-liner!
3a) speaking of DOAP, relationships are nice --does your project
involve/play with other technologies? If so, state the relationship.
4) include a way for folks to learn more. Link to the project home page or
a mailing list or some other relevant resource.
5) personalize your closing --whether your choose to sign off using your
name or (on behalf of) the collective PMC, doing something friendly
reinforces the "community" aspect of the ASF.
Thanks so much,
Sally
[1] https://blogs.apache.org/foundation/date/20150612
// vim:tw=74: