Most workflows will get kicked off automatically when you open a PR, push code, or on a schedule.
If you would like to manually trigger a job, you have 2 options:
Trigger Phrases: Some jobs have trigger phrases associated with them (e.g. Run XYZ PreCommit). These will appear in statuses of previous PR runs of that check. You can trigger the job on any PR by commenting that trigger phrase in the PR.
Note: this approach is found not scalable (#28909) and currently only enabled for PreCommit workflows. For PostCommit jobs, it is currently replaced by a temporary approach: test suites are configured to trigger whenever a particular trigger file is modified. Test workflows have pull_request_target paths, which include a trigger file. Whenever a trigger file is modified, the test suite will trigger on the pull request. Make any change to this file to trigger the job. The trigger file looks like the following: .github/trigger_files/<workflow_file_name.json>.
Committers only - Manual triggering: Any committer can start any job with a workflow_dispatch trigger defined (all jobs should have these). To do so, navigate to the Actions tab, click on your desired workflow in the left navigation bar, and then click Run Workflow.
On top of normal Actions workflow steps, all new CI workflows (excluding release workflows or other workflow automation) should have the following:
Each of these is described in more detail below.
Due to specifics on how the comment triggered rerun is handled it is required that all jobs have name and phrase set via matrix elements. See the following example:
jobs:
beam_job_name:
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
strategy:
matrix:
job_name: [beam_job_name]
job_phrase: [Run Job Phrase]
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
github.event_name == 'workflow_dispatch' ||
github.event.comment.body == 'Run Job Phrase'
steps:
- uses: actions/checkout@v3
- name: Setup repository
uses: ./.github/actions/setup-action
with:
comment_phrase: ${{ matrix.job_phrase }}
github_token: ${{ secrets.GITHUB_TOKEN }}
github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
And in case when the workflow already utilizes matrix do the following:
jobs:
beam_job_with_matrix:
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
runs-on: [self-hosted, ubuntu-20.04, main]
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
job_name: ["beam_job_with_matrix"]
job_phrase: ["Run Job With Matrix"]
python_version: ['3.8','3.9','3.10','3.11']
if: |
github.event_name == 'push' ||
github.event_name == 'pull_request_target' ||
(github.event_name == 'schedule' && github.repository == 'apache/beam') ||
github.event_name == 'workflow_dispatch' ||
startsWith(github.event.comment.body, 'Run Job With Matrix')
steps:
- uses: actions/checkout@v3
- name: Setup repository
uses: ./.github/actions/setup-action
with:
comment_phrase: ${{ matrix.job_phrase }}
github_token: ${{ secrets.GITHUB_TOKEN }}
github_job: ${{ matrix.job_name }} (${{ matrix.job_phrase }} ${{ matrix.python_version }})
Notice how the matrix element of python_version is appended to the name.
GitHub allows workflows to define a set of triggers that dictate when a job should be run. For more info, see documentation here.
For the purposes of Beam, each CI workflow should define the following triggers:
push triggerpull_request_target triggerThe push/pull_request_target triggers should only run when appropriate paths are modified. See https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Go.yml#L4 for an example (you can copy and paste this into your workflow, you just need to change the paths).
Because we use the pull_request_target trigger instead of pull_request, we need an explicit checkout of the correct commit. This can be done as a step that uses the setup-action action in your workflow. See https://github.com/apache/beam/blob/0ee2dc73ec6f555a5bf1a643dffd37f4927be67e/.github/workflows/beam_PreCommit_Go.yml#L65-L70 for an example (you can copy and paste this into your workflow). Please make sure that you checkout the code before using the composite action.
You should explicitly define the GitHub Actions token scopes needed by your job. For most jobs, this will be actions: write (needed for comment triggering support) and read permissions for all other options. See https://github.com/apache/beam/blob/907c5110163b0efe52e9e12127fd013c7fc455d7/.github/workflows/beam_PreCommit_Go.yml#L16 for an example (you can copy and paste this into your workflow).
Concurrency groups are a way of making sure that no more than one Actions run is running per job/group at any given time (previous ones can be cancelled). To reduce resource usage, you should define the following concurrency group:
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.issue.number || github.sha || github.head_ref || github.ref }}-${{ github.event.schedule || github.event.comment.id || github.event.sender.login }}'
cancel-in-progress: true
this defines the following groups:
In order to make it easier for non-committers to interact with workflows, workflows should implement comment triggering support. This requires 3 additional components beyond the ones mentioned above:
setup-action action in your workflow, which encapsulates the checkout and rerun logic in one place. This should be gated on the comment trigger (example: https://github.com/apache/beam/blob/0ee2dc73ec6f555a5bf1a643dffd37f4927be67e/.github/workflows/beam_PreCommit_Go.yml#L65-L70)Note: Comment triggering is found not scalable (#28909) and is currently limited to a subset of suites. For more information see the Running Workflows Manually section.
New workflows are not added to the UI on the Actions tab until they are merged into the repo's main branch (master for Beam). To test new workflows, we recommend the following pattern:
Run workflow button.Note: most workflows use self-hosted runners with the main and ubuntu labels to execute (example). If you are testing on a fork, you likely will not have self-hosted runners set up. To work around this, you can start using hosted runners and then switch over when you're ready to create a PR. You can do this by changing runs-on: [self-hosted, ubuntu-20.04, main] (self-hosted, use in your PR) to runs-on: ubuntu-20.04 (GitHub hosted, use for local testing).
Note when using ubuntu-20.04 as the host, you might need to choose the Java version since some gradle tasks only work with a certain Java version. One example is below to use Java 8 when testing your workflow:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '8'
If you need to make more changes to the workflow yaml file after it has been added to the repo, you can develop normally on a branch (using your fork or the main Beam repo if you are a committer). If you are a non-committer, this flow has several caveats mentioned below. To do this:
Note: If you run a workflow from your fork of Beam, it will not have access to secrets stored in the Beam repository. This will cause some things like authenticating to external services to fail, which may cause your workflow to fail. If you run into this issue, you can either:
Additionally, as mentioned above your fork likely will not have self-hosted runners set up. To work around this, you can start using hosted runners and then switch over when you're ready to create a PR. You can do this by changing runs-on: [self-hosted, ubuntu-20.04, main] (self-hosted, use in your PR) to runs-on: ubuntu-20.04 (GitHub hosted, use for local testing).
Please note that jobs with matrix need to have matrix element in the comment. Example: Run Python PreCommit (3.8)
PreCommit Jobs run in a schedule and also get triggered in a PR if relevant sources has changed. To manually trigger certain PreCommit job, comment with the Trigger Phrase (listed below) in the PR.
| Workflow name | Matrix | Trigger Phrase | Cron Status |
|---|---|---|---|
| PreCommit Community Metrics | N/A | Run CommunityMetrics PreCommit |  |
| PreCommit GHA | N/A | Run GHA PreCommit |  |
| PreCommit Go | N/A | Run Go PreCommit |  |
| PreCommit GoPortable | N/A | Run GoPortable PreCommit |  |
| PreCommit Java | N/A | Run Java PreCommit |  |
| PreCommit Java Amazon Web Services IO Direct | N/A | Run Java_Amazon-Web-Services_IO_Direct PreCommit |  |
| PreCommit Java Amazon Web Services2 IO Direct | N/A | Run Java_Amazon-Web-Services2_IO_Direct PreCommit |  |
| PreCommit Java Amqp IO Direct | N/A | Run Java_Amqp_IO_Direct PreCommit |  |
| PreCommit Java Azure IO Direct | N/A | Run Java_Azure_IO_Direct PreCommit |  |
| PreCommit Java Cassandra IO Direct | N/A | Run Java_Cassandra_IO_Direct PreCommit |  |
| PreCommit Java Cdap IO Direct | N/A | Run Java_Cdap_IO_Direct PreCommit |  |
| PreCommit Java Clickhouse IO Direct | N/A | Run Java_Clickhouse_IO_Direct PreCommit |  |
| PreCommit Java Csv IO Direct | N/A | Run Java_Csv_IO_Direct PreCommit |  |
| PreCommit Java Debezium IO Direct | N/A | Run Java_Debezium_IO_Direct PreCommit |  |
| PreCommit Java ElasticSearch IO Direct | N/A | Run Java_ElasticSearch_IO_Direct PreCommit |  |
| PreCommit Java Examples Dataflow | N/A | Run Java_Examples_Dataflow PreCommit |  |
| PreCommit Java Examples Dataflow Java21 | N/A | Run Java_Examples_Dataflow_Java21 PreCommit |  |
| PreCommit Java File-schema-transform IO Direct | N/A | Run Java_File-schema-transform_IO_Direct PreCommit |  |
| PreCommit Java Flink Versions | N/A | Run Java_Flink_Versions PreCommit |  |
| PreCommit Java GCP IO Direct | N/A | Run Java_GCP_IO_Direct PreCommit |  |
| PreCommit Java Google-ads IO Direct | N/A | Run Java_Google-ads_IO_Direct PreCommit |  |
| PreCommit Java Hadoop IO Direct | N/A | Run Java_Hadoop_IO_Direct PreCommit |  |
| PreCommit Java HBase IO Direct | N/A | Run Java_HBase_IO_Direct PreCommit |  |
| PreCommit Java HCatalog IO Direct | N/A | Run Java_HCatalog_IO_Direct PreCommit |  |
| PreCommit Java Kafka IO Direct | N/A | Run Java_Kafka_IO_Direct PreCommit |  |
| PreCommit Java InfluxDb IO Direct | N/A | Run Java_InfluxDb_IO_Direct PreCommit |  |
| PreCommit Java IOs Direct | N/A | Run Java_IOs_Direct PreCommit | N/A |
| PreCommit Java JDBC IO Direct | N/A | Run Java_JDBC_IO_Direct PreCommit |  |
| PreCommit Java Jms IO Direct | N/A | Run Java_Jms_IO_Direct PreCommit |  |
| PreCommit Java Kinesis IO Direct | N/A | Run Java_Kinesis_IO_Direct PreCommit |  |
| PreCommit Java Kudu IO Direct | N/A | Run Java_Kudu_IO_Direct PreCommit |  |
| PreCommit Java MongoDb IO Direct | N/A | Run Java_MongoDb_IO_Direct PreCommit |  |
| PreCommit Java Mqtt IO Direct | N/A | Run Java_Mqtt_IO_Direct PreCommit |  |
| PreCommit Java Neo4j IO Direct | N/A | Run Java_Neo4j_IO_Direct PreCommit |  |
| PreCommit Java Parquet IO Direct | N/A | Run Java_Parquet_IO_Direct PreCommit |  |
| PreCommit Java Pulsar IO Direct | N/A | Run Java_Pulsar_IO_Direct PreCommit |  |
| PreCommit Java PVR Flink Batch | N/A | Run Java_PVR_Flink_Batch PreCommit |  |
| PreCommit Java PVR Flink Docker | N/A | Run Java_PVR_Flink_Docker PreCommit |  |
| PreCommit Java RabbitMq IO Direct | N/A | Run Java_RabbitMq_IO_Direct PreCommit |  |
| PreCommit Java Redis IO Direct | N/A | Run Java_Redis_IO_Direct PreCommit |  |
| PreCommit Java RequestResponse IO Direct | N/A | Run Java_RequestResponse_IO_Direct PreCommit |  |
| PreCommit Java SingleStore IO Direct | N/A | Run Java_SingleStore_IO_Direct PreCommit |  |
| PreCommit Java Snowflake IO Direct | N/A | Run Java_Snowflake_IO_Direct PreCommit |  |
| PreCommit Java Solr IO Direct | N/A | Run Java_Solr_IO_Direct PreCommit |  |
| PreCommit Java Spark3 Versions | N/A | Run Java_Spark3_Versions PreCommit |  |
| PreCommit Java Splunk IO Direct | N/A | Run Java_Splunk_IO_Direct PreCommit |  |
| PreCommit Java Thrift IO Direct | N/A | Run Java_Thrift_IO_Direct PreCommit |  |
| PreCommit Java Tika IO Direct | N/A | Run Java_Tika_IO_Direct PreCommit |  |
| PreCommit Kotlin Examples | N/A | Run Kotlin_Examples PreCommit |  |
| PreCommit Portable Python | [‘3.8’,‘3.11’] | Run Portable_Python PreCommit |  |
| PreCommit Python | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | Run Python PreCommit (matrix_element) |  |
| PreCommit Python Coverage | N/A | Run Python_Coverage PreCommit |  |
| PreCommit Python Dataframes | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | Run Python_Dataframes PreCommit (matrix_element) |  |
| PreCommit Python Docker | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | Run PythonDocker PreCommit (matrix_element) |  |
| PreCommit Python Docs | N/A | Run PythonDocs PreCommit |  |
| PreCommit Python Examples | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | Run Python_Examples PreCommit (matrix_element) |  |
| PreCommit Python Formatter | N/A | Run PythonFormatter PreCommit |  |
| PreCommit Python Integration | [‘3.8’,‘3.11’] | Run Python_Integration PreCommit (matrix_element) |  |
| PreCommit Python Lint | N/A | Run PythonLint PreCommit |  |
| PreCommit Python PVR Flink | N/A | Run Python_PVR_Flink PreCommit |  |
| PreCommit Python Runners | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | Run Python_Runners PreCommit (matrix_element) |  |
| PreCommit Python Transforms | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | Run Python_Transforms PreCommit (matrix_element) |  |
| PreCommit RAT | N/A | Run RAT PreCommit |  |
| PreCommit Spotless | N/A | Run Spotless PreCommit |  |
| PreCommit SQL | N/A | Run SQL PreCommit |  |
| PreCommit SQL Java11 | N/A | Run SQL_Java11 PreCommit |  |
| PreCommit SQL Java17 | N/A | Run SQL_Java17 PreCommit |  |
| PreCommit Typescript | N/A | Run Typescript PreCommit |  |
| PreCommit Website | N/A | Run Website PreCommit |  |
| PreCommit Website Stage GCS | N/A | Run Website_Stage_GCS PreCommit |  |
| PreCommit Whitespace | N/A | Run Whitespace PreCommit |  |
| PreCommit Xlang Generated Transforms | N/A | Run Xlang_Generated_Transforms PreCommit |  |
| PreCommit YAML Xlang Direct | N/A | Run Yaml_Xlang_Direct PreCommit |  |
PostCommit Jobs run in a schedule against master branch and generally do not get triggered in a PR. To manually trigger certain PostCommit job, attach a file named “Trigger file” (listed below) under .github/trigger_files/ folder.
| Workflow name | Matrix | Trigger file | Cron Status |
|---|---|---|---|
| PostCommit Go | N/A | beam_PostCommit_Go.json |  |
| PostCommit Go Dataflow ARM | N/A | beam_PostCommit_Go_Dataflow_ARM.json |  |
| PostCommit Go VR Flink | N/A | beam_PostCommit_Go_VR_Flink.json |  |
| PostCommit Go VR Samza | N/A | beam_PostCommit_Go_VR_Samza.json |  |
| PostCommit Go VR Spark | N/A | beam_PostCommit_Go_VR_Spark.json |  |
| PostCommit Java Avro Versions | N/A | beam_PostCommit_Java_Avro_Versions.json |  |
| PostCommit Java BigQueryEarlyRollout | N/A | beam_PostCommit_Java_BigQueryEarlyRollout.json |  |
| PostCommit Java Dataflow V1 | N/A | beam_PostCommit_Java_DataflowV1.json |  |
| PostCommit Java Dataflow V2 | N/A | beam_PostCommit_Java_DataflowV2.json |  |
| PostCommit Java Examples Dataflow ARM | [‘8’,‘11’,‘17’,‘21’] | beam_PostCommit_Java_Examples_Dataflow_ARM.json |  |
| PostCommit Java Examples Dataflow | N/A | beam_PostCommit_Java_Examples_Dataflow.json |  |
| PostCommit Java Examples Dataflow Java | [‘11’,‘17’,‘21’] | beam_PostCommit_Java_Examples_Dataflow_Java.json |  |
| PostCommit Java Examples Dataflow V2 | N/A | beam_PostCommit_Java_Examples_Dataflow_V2.json |  |
| PostCommit Java Examples Dataflow V2 Java | [‘11’,‘17’,‘21’] | beam_PostCommit_Java_Examples_Dataflow_V2_Java.json |  |
| PostCommit Java Examples Direct | N/A | beam_PostCommit_Java_Examples_Direct.json |  |
| PostCommit Java Examples Flink | N/A | beam_PostCommit_Java_Examples_Flink.json |  |
| PostCommit Java Examples Spark | N/A | beam_PostCommit_Java_Examples_Spark.json |  |
| PostCommit Java Hadoop Versions | N/A | beam_PostCommit_Java_Hadoop_Versions.json |  |
| PostCommit Java IO Performance Tests | N/A | beam_PostCommit_Java_IO_Performance_Tests.json |  |
| PostCommit Java InfluxDbIO Integration Test | N/A | beam_PostCommit_Java_InfluxDbIO_IT.json |  |
| PostCommit Java Jpms Dataflow Java11 | N/A | beam_PostCommit_Java_Jpms_Dataflow_Java11.json |  |
| PostCommit Java Jpms Dataflow Java17 | N/A | beam_PostCommit_Java_Jpms_Dataflow_Java17.json |  |
| PostCommit Java Jpms Direct Java11 | N/A | beam_PostCommit_Java_Jpms_Direct_Java11.json |  |
| PostCommit Java Jpms Direct Java17 | N/A | beam_PostCommit_Java_Jpms_Direct_Java17.json |  |
| PostCommit Java Jpms Direct Java21 | N/A | beam_PostCommit_Java_Jpms_Direct_Java21.json |  |
| PostCommit Java Jpms Flink Java11 | N/A | beam_PostCommit_Java_Jpms_Flink_Java11.json |  |
| PostCommit Java Jpms Spark Java11 | N/A | beam_PostCommit_Java_Jpms_Spark_Java11.json |  |
| PostCommit Java Nexmark Dataflow | N/A | beam_PostCommit_Java_Nexmark_Dataflow.json |  |
| PostCommit Java Nexmark Dataflow V2 | N/A | beam_PostCommit_Java_Nexmark_Dataflow_V2.json |  |
| PostCommit Java Nexmark Dataflow V2 Java | [‘11’,‘17’] | beam_PostCommit_Java_Nexmark_Dataflow_V2_Java.json |  |
| PostCommit Java Nexmark Direct | N/A | beam_PostCommit_Java_Nexmark_Direct.json |  |
| PostCommit Java Nexmark Flink | N/A | beam_PostCommit_Java_Nexmark_Flink.json |  |
| PostCommit Java Nexmark Spark | N/A | beam_PostCommit_Java_Nexmark_Spark.json |  |
| PostCommit Java PVR Flink Streaming | N/A | beam_PostCommit_Java_PVR_Flink_Streaming.json |  |
| PostCommit Java PVR Samza | N/A | beam_PostCommit_Java_PVR_Samza.json |  |
| PostCommit Java SingleStoreIO IT | N/A | beam_PostCommit_Java_SingleStoreIO_IT.json |  |
| PostCommit Java PVR Spark3 Streaming | N/A | beam_PostCommit_Java_PVR_Spark3_Streaming.json |  |
| PostCommit Java PVR Spark Batch | N/A | beam_PostCommit_Java_PVR_Spark_Batch.json |  |
| PostCommit Java Sickbay | N/A | beam_PostCommit_Java_Sickbay.json |  |
| PostCommit Java Tpcds Dataflow | N/A | beam_PostCommit_Java_Tpcds_Dataflow.json |  |
| PostCommit Java Tpcds Flink | N/A | beam_PostCommit_Java_Tpcds_Flink.json |  |
| PostCommit Java Tpcds Spark | N/A | beam_PostCommit_Java_Tpcds_Spark.json |  |
| PostCommit Java ValidatesRunner Dataflow JavaVersions | [‘11’,‘17’] | beam_PostCommit_Java_ValidatesRunner_Dataflow_JavaVersions.json |  |
| PostCommit Java ValidatesRunner Dataflow Streaming | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming.json |  |
| PostCommit Java ValidatesRunner Dataflow V2 Streaming | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow_V2_Streaming.json |  |
| PostCommit Java ValidatesRunner Dataflow V2 | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow_V2.json |  |
| PostCommit Java ValidatesRunner Dataflow | N/A | beam_PostCommit_Java_ValidatesRunner_Dataflow.json |  |
| PostCommit Java ValidatesRunner Direct JavaVersions | [‘11’,‘17’] | beam_PostCommit_Java_ValidatesRunner_Direct_JavaVersions.json |  |
| PostCommit Java ValidatesRunner Direct | N/A | beam_PostCommit_Java_ValidatesRunner_Direct.json |  |
| PostCommit Java ValidatesRunner Flink Java11 | N/A | beam_PostCommit_Java_ValidatesRunner_Flink_Java11.json |  |
| PostCommit Java ValidatesRunner Flink | N/A | beam_PostCommit_Java_ValidatesRunner_Flink.json |  |
| PostCommit Java ValidatesRunner Samza | N/A | beam_PostCommit_Java_ValidatesRunner_Samza.json |  |
| PostCommit Java ValidatesRunner Spark Java11 | N/A | beam_PostCommit_Java_ValidatesRunner_Spark_Java11.json |  |
| PostCommit Java ValidatesRunner Spark | N/A | beam_PostCommit_Java_ValidatesRunner_Spark.json |  |
| PostCommit Java ValidatesRunner SparkStructuredStreaming | N/A | beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.json |  |
| PostCommit Java ValidatesRunner Twister2 | N/A | beam_PostCommit_Java_ValidatesRunner_Twister2.json |  |
| PostCommit Java ValidatesRunner ULR | N/A | beam_PostCommit_Java_ValidatesRunner_ULR.json |  |
| PostCommit Java | N/A | beam_PostCommit_Java.json |  |
| PostCommit Javadoc | N/A | beam_PostCommit_Javadoc.json |  |
| PostCommit PortableJar Flink | N/A | beam_PostCommit_PortableJar_Flink.json |  |
| PostCommit PortableJar Spark | N/A | beam_PostCommit_PortableJar_Spark.json |  |
| PostCommit Python | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_PostCommit_Python.json |  |
| PostCommit Python Arm | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_PostCommit_Python_Arm.json |  |
| PostCommit Python Dependency | N/A | beam_PostCommit_Python_Dependency.json |  |
| PostCommit Python Examples Dataflow | N/A | beam_PostCommit_Python_Examples_Dataflow.json |  |
| PostCommit Python Examples Direct | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_PostCommit_Python_Examples_Direct.json |  |
| PostCommit Python Examples Flink | [‘3.8’,‘3.11’] | beam_PostCommit_Python_Examples_Flink.json |  |
| PostCommit Python Examples Spark | [‘3.8’,‘3.11’] | beam_PostCommit_Python_Examples_Spark.json |  |
| PostCommit Python MongoDBIO IT | N/A | beam_PostCommit_Python_MongoDBIO_IT.json |  |
| PostCommit Python Nexmark Direct | N/A | beam_PostCommit_Python_Nexmark_Direct.json |  |
| PostCommit Python ValidatesContainer Dataflow | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_PostCommit_Python_ValidatesContainer_Dataflow.json |  |
| PostCommit Python ValidatesContainer Dataflow With RC | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_PostCommit_Python_ValidatesContainer_Dataflow_With_RC.json |  |
| PostCommit Python ValidatesRunner Dataflow | [‘3.8’,‘3.11’] | beam_PostCommit_Python_ValidatesRunner_Dataflow.json |  |
| PostCommit Python ValidatesRunner Flink | [‘3.8’,‘3.11’] | beam_PostCommit_Python_ValidatesRunner_Flink.json |  |
| PostCommit Python ValidatesRunner Samza | [‘3.8’,‘3.11’] | beam_PostCommit_Python_ValidatesRunner_Samza.json |  |
| PostCommit Python ValidatesRunner Spark | [‘3.8’,‘3.9’,‘3.11’] | beam_PostCommit_Python_ValidatesRunner_Spark.json |  |
| PostCommit Python Xlang Gcp Dataflow | N/A | beam_PostCommit_Python_Xlang_Gcp_Dataflow.json |  |
| PostCommit Python Xlang Gcp Direct | N/A | beam_PostCommit_Python_Xlang_Gcp_Direct.json |  |
| PostCommit Python Xlang IO Dataflow | N/A | beam_PostCommit_Python_Xlang_IO_Dataflow.json |  |
| PostCommit Sickbay Python | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_PostCommit_Sickbay_Python.json |  |
| PostCommit SQL | N/A | beam_PostCommit_SQL.json |  |
| PostCommit TransformService Direct | N/A | beam_PostCommit_TransformService_Direct.json |  |
| PostCommit Website Test | N/A | beam_PostCommit_Website_Test.json |  |
| PostCommit XVR GoUsingJava Dataflow | N/A | beam_PostCommit_XVR_GoUsingJava_Dataflow.json |  |
| PostCommit XVR Direct | N/A | beam_PostCommit_XVR_Direct.json |  |
| PostCommit XVR Flink | N/A | beam_PostCommit_XVR_Flink.json |  |
| PostCommit XVR JavaUsingPython Dataflow | N/A | beam_PostCommit_XVR_JavaUsingPython_Dataflow.json |  |
| PostCommit XVR PythonUsingJava Dataflow | N/A | beam_PostCommit_XVR_PythonUsingJava_Dataflow.json |  |
| PostCommit XVR PythonUsingJavaSQL Dataflow | N/A | beam_PostCommit_XVR_PythonUsingJavaSQL_Dataflow.json |  |
| PostCommit XVR Samza | N/A | beam_PostCommit_XVR_Samza.json |  |
| PostCommit XVR Spark3 | N/A | beam_PostCommit_XVR_Spark3.json |  |
| Python Validates Container Dataflow ARM | [‘3.8’,‘3.9’,‘3.10’,‘3.11’] | beam_Python_ValidatesContainer_Dataflow_ARM.json |  |
| Workflow name | Matrix | Cron Status |
|---|---|---|
| Cancel Stale Dataflow Jobs | N/A |  |
| Clean Up GCP Resources | N/A |  |
| Clean Up Prebuilt SDK Images | N/A |  |
| Cleanup Dataproc Resources | N/A |  |
| Community Metrics Prober | N/A |  |
| Playground CI Nightly | [“java”, “go”, “python”] |  |
| Publish Beam SDK Snapshots | N/A |  |
| Publish BeamMetrics | N/A |  |
| Publish Docker Snapshots | N/A |  |
| Publish Website | N/A |  |
| Release Nightly Snapshot | N/A |  |
| Release Nightly Snapshot Python | N/A |  |
| Rotate IO-Datastores Cluster Credentials | N/A |  |
| Rotate Metrics Cluster Credentials | N/A |  |
