The Apache SpamAssassin project uses a subversion repo for its development process. A read-only mirror of the repo is maintained on GitHub here.
The .github directory containing this README file is not part of the Apache SpamAssassin release package. The files in this directory are intended for use by developers to run tests using GitHub's Actions facility on GitHub hosted runners.
The Apache SpamAssassin Project Management Committe has not made any arrangements to use the resources allocated to the Apache Software Foundation by GitHub to run builds and tests. The actions defined in this directory are available for anyone, including active developers of SpamAssassin, to run in their personal GitHub fork of the repo. However, the inclusion of the files in this repository does not comprise a formal release of the software to the public.
The workflow run you submit will have one job for every valid combination of values from the first three input boxes.
The fourth input box allows you to enter the tests to be run, im the same format as used for TEST_FILES in a make test command line. If left empty, it means run all tests.
No matter what is entered in the tests box, the tests that use SQL will only be run in the jobs that have postgres or mysql specified for database. Also, the spamd stress tests and root tests are never run.
GitHub has limits on number of jobs you can run simultaneously on the various platforms. Jobs you submit that are over that limit will be queued to be started as other jobs finish.
Clicking on a job listed in the left sidebar will open a pane showing the log output of the job. A job that ends with errors will have a red X icon. You can check the log output for details. Some errors will result in the t/log directory contents being zipped up as an artifact you can download. When you are viewing the log pane, click on the Summary icon above the left sidebar, If there are any artifacts to download, there will be a number you can click on under the heading Artifacts.
The number of jobs run is the product of the options you specify in the three input boxes. Unless you want to test SpamAssassin on every possible version of perl, which you might if you are the release manager preparing a new release, you will likely want to select only one recent version of perl.
The options box for runners only shows the “-latest” names, but you can enter any GitHub hosted runner that they make available, e.g. ubuntu-20.04 or macos-11.
Windows is tested using Strawberry Perl, of which the latest release is 5.32. If you have 34 or 36 in the perl versions list, they will not generate jobs on the Windows platform.
Jobs run using the database option postgres or mysql will only run the various sql tests. Jobs run using the none option for database will run all the other tests.
Some tests, especially some that rely on network access such as t/dnsbl.t, seem to fail occassionally, especially when you are running many jobs simultaneously. After all the jobs of a workflow have completed, you can rerun just those that have failed by clicking on the Re-run jobs button in the overview page for the jobs, and then selecting Re-run failed jobs. Repeat until jobs that seem to be only intermittent failures have successfully passed.