The entry point for our build is the “CI” workflow which is defined in ci.yml. This is used for both PR and trunk builds. The jobs and steps of the workflow are defined in build.yml.
For Pull Requests, the “CI” workflow runs in an unprivileged context. This means it does not have access to repository secrets. After the “CI” workflow is complete, the “CI Complete” workflow is automatically run. This workflow consumes artifacts from the “CI” workflow and does run in a privileged context. This is how we are able to upload Gradle Build Scans to Develocity without exposing our access token to the Pull Requests.
By default, GitHub sends an email for each failed action run. To change this, visit https://github.com/settings/notifications and find System -> Actions. Here you can change your notification preferences.
Please read the following GitHub articles before authoring new workflows.
Any workflows that use the run directive should avoid using the ${{ ... }} syntax. Instead, declare all injectable variables as environment variables. For example:
- name: Copy RC Image to promoted image env: PROMOTED_DOCKER_IMAGE: ${{ github.event.inputs.promoted_docker_image }} RC_DOCKER_IMAGE: ${{ github.event.inputs.rc_docker_image }} run: | docker buildx imagetools create --tag $PROMOTED_DOCKER_IMAGE $RC_DOCKER_IMAGE
This prevents untrusted inputs from doing script injection in the run steps.
pull_request_target eventsIn addition to the above security articles, please review the official documentation on pull_request_target. This event type allows PRs to trigger actions that run with elevated permission and access to repository secrets. We should only be using this for very simple tasks such as applying labels or adding comments to PRs.
We must never run the untrusted PR code in the elevated pull_request_target context
The ci.yml is run when commits are pushed to trunk. This calls into build.yml to run our main build. In the trunk build, we do not read from the Gradle cache, but we do write to it. Also, the test catalog is only updated from trunk builds.
Similar to trunk, this workflow starts in ci.yml and calls into build.yml. Unlike trunk, the PR builds will utilize the Gradle cache.
In order to get the attention of committers, we have a triage workflow for Pull Requests opened by non-committers. This workflow consists of two files:
triage label if the PR was opened by a non-committer.needs-attention label to community PRs that have not been reviewed after 7 days. Also includes a cron job to remove the triage and needs-attention labels from PRs which have been reviewed.The pr-update.yml workflow includes pull_request_target!
For committers to avoid having this label added, their membership in the ASF GitHub organization must be public. Here are the steps to take:
Full documentation for this process can be found in GitHub's docs: https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-your-membership-in-organizations/publicizing-or-hiding-organization-membership
If you are a committer and do not want your membership in the ASF org listed as public, you will need to remove the triage label manually.
Due to a combination of GitHub security and ASF's policy, we required explicit approval of workflows on PRs submitted by non-committers (and non-contributors). To simply this process, we have a ci-approved label which automatically approves these workflows.
There are two files related to this workflow:
ci-approvedci-approved labelThe pr-labeled.yml workflow includes pull_request_target!
To help ensure good commit messages, we have added a “Pull Request Linter” job that checks the title and body of the PR.
There are two files related to this workflow:
Note that the pr-reviewed.yml workflow uses the ci-approved mechanism described above.
The following checks are performed on our PRs:
With the merge queue, our PR title and body will become the commit subject and message. This linting step will help to ensure that we have nice looking commits.
This one is straightforward. Using the “actions/stale” GitHub Action, we automatically label and eventually close PRs which have not had activity for some time. See the stale.yml workflow file for specifics.
Composite actions are a convenient way to reuse build logic, but they have some limitations.
workflow_call instead)If your PR is running for longer than you would expect due to cache misses, there are a few things to check.
First, find the cache that was loaded into your PR build. This is found in the Setup Gradle output. Look for a line starting with “Restored Gradle User Home from cache key”. For example,
Restored Gradle User Home from cache key: gradle-home-v1|Linux-X64|test[188616818c9a3165053ef8704c27b28e]-5c20aa187aa8f51af4270d7d1b0db4963b0cd10b
The last part of the cache key is the SHA of the commit on trunk where the cache was created. If that commit is not on your branch, it means your build loaded a cache that includes changes your PR does not yet have. This is a common way to have cache misses. To resolve this, update your PR with the latest cached trunk commit:
git fetch origin ./committer-tools/update-cache.sh git merge trunk-cached
then push your branch.
If your build seems to be using the correct cache, the next thing to check is for changes to task inputs. You can find this by locating the trunk Build Scan from the cache commit on trunk and comparing it with the build scan of your PR build. This is done in the Develocity UI using the two overlapping circles like (A()B). This will show you differences in the task inputs for the two builds.
Finally, you can run your PR with extra cache debugging. Add this to the gradle invocation in run-gradle/action.yml.
-Dorg.gradle.caching.debug=true
This will dump out a lot of output, so you may also reduce the test target to one module.