This directory provisions Google Cloud project networking for Dataflow usage.
The following table lists all provisioned resources and their rationale.
| resource | reason |
|---|---|
| Network | Run workload in its isolated GCP VPC |
| Subnetwork | Worker needs at least one subnetwork |
| Firewall Rules | Limit traffic to Worker service account VMS |
Follow terraform workflow convention to apply this module. It assumes the working directory is at .test-infra/pipelines
This module uses a Google Cloud Storage bucket backend.
Initialize the terraform workspace for the apache-beam-testing project:
DIR=infrastructure/02.network terraform -chdir=$DIR init -backend-config=apache-beam-testing.tfbackend
or for your own Google Cloud project:
DIR=infrastructure/02.network terraform init -backend-config=path/to/your/backend-config-file.tfbackend
where your backend-config-file.tfbackend contains:
bucket = <Google Cloud Storage Bucket Name>
Notice the -var-file flag referencing common.tfvars that provides opinionated variable defaults.
For apache-beam-testing:
DIR=infrastructure/02.network terraform -chdir=$DIR apply -var-file=common.tfvars -var-file=apache-beam-testing.tfvars
or for your own Google Cloud project:
DIR=infrastructure/02.network terraform -chdir=$DIR apply -var-file=common.tfvars