| :py:mod:`airflow.providers.google.cloud.example_dags.example_bigtable` |
| ====================================================================== |
| |
| .. py:module:: airflow.providers.google.cloud.example_dags.example_bigtable |
| |
| .. autoapi-nested-parse:: |
| |
| Example Airflow DAG that creates and performs following operations on Cloud Bigtable: |
| - creates an Instance |
| - creates a Table |
| - updates Cluster |
| - waits for Table replication completeness |
| - deletes the Table |
| - deletes the Instance |
| |
| This DAG relies on the following environment variables: |
| |
| * GCP_PROJECT_ID - Google Cloud project |
| * CBT_INSTANCE_ID - desired ID of a Cloud Bigtable instance |
| * CBT_INSTANCE_DISPLAY_NAME - desired human-readable display name of the Instance |
| * CBT_INSTANCE_TYPE - type of the Instance, e.g. 1 for DEVELOPMENT |
| See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance # noqa E501 |
| * CBT_INSTANCE_LABELS - labels to add for the Instance |
| * CBT_CLUSTER_ID - desired ID of the main Cluster created for the Instance |
| * CBT_CLUSTER_ZONE - zone in which main Cluster will be created. e.g. europe-west1-b |
| See available zones: https://cloud.google.com/bigtable/docs/locations |
| * CBT_CLUSTER_NODES - initial amount of nodes of the Cluster |
| * CBT_CLUSTER_NODES_UPDATED - amount of nodes for BigtableClusterUpdateOperator |
| * CBT_CLUSTER_STORAGE_TYPE - storage for the Cluster, e.g. 1 for SSD |
| See https://googleapis.github.io/google-cloud-python/latest/bigtable/instance.html#google.cloud.bigtable.instance.Instance.cluster # noqa E501 |
| * CBT_TABLE_ID - desired ID of the Table |
| * CBT_POKE_INTERVAL - number of seconds between every attempt of Sensor check |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:data:: GCP_PROJECT_ID |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_ID |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_DISPLAY_NAME |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_DISPLAY_NAME_UPDATED |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_TYPE |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_TYPE_PROD |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_LABELS |
| |
| |
| |
| |
| .. py:data:: CBT_INSTANCE_LABELS_UPDATED |
| |
| |
| |
| |
| .. py:data:: CBT_CLUSTER_ID |
| |
| |
| |
| |
| .. py:data:: CBT_CLUSTER_ZONE |
| |
| |
| |
| |
| .. py:data:: CBT_CLUSTER_NODES |
| |
| |
| |
| |
| .. py:data:: CBT_CLUSTER_NODES_UPDATED |
| |
| |
| |
| |
| .. py:data:: CBT_CLUSTER_STORAGE_TYPE |
| |
| |
| |
| |
| .. py:data:: CBT_TABLE_ID |
| |
| |
| |
| |
| .. py:data:: CBT_POKE_INTERVAL |
| |
| |
| |
| |
| .. py:data:: create_instance_task |
| |
| |
| |
| |
