blob: 9eafcf35a22ee2e654531a88be7012aa19022b68 [file] [log] [blame]
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
Google Cloud Workflows Operators
================================
You can use Workflows to create serverless workflows that link series of serverless tasks together
in an order you define. Combine the power of Google Cloud's APIs, serverless products like Cloud
Functions and Cloud Run, and calls to external APIs to create flexible serverless applications.
For more information about the service visit
`Workflows production documentation <Product documentation <https://cloud.google.com/workflows/docs/overview>`__.
Prerequisite Tasks
------------------
.. include::/operators/_partials/prerequisite_tasks.rst
.. _howto/operator:WorkflowsCreateWorkflowOperator:
Create workflow
---------------
To create a workflow use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsCreateWorkflowOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_create_workflow]
:end-before: [END how_to_create_workflow]
The workflow should be define in similar why to this example:
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 0
:start-after: [START how_to_define_workflow]
:end-before: [END how_to_define_workflow]
For more information about authoring workflows check official
production documentation `<Product documentation <https://cloud.google.com/workflows/docs/overview>`__.
.. _howto/operator:WorkflowsUpdateWorkflowOperator:
Update workflow
---------------
To update a workflow use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsUpdateWorkflowOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_update_workflow]
:end-before: [END how_to_update_workflow]
.. _howto/operator:WorkflowsGetWorkflowOperator:
Get workflow
------------
To get a workflow use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsGetWorkflowOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_get_workflow]
:end-before: [END how_to_get_workflow]
.. _howto/operator:WorkflowsListWorkflowsOperator:
List workflows
--------------
To list workflows use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsListWorkflowsOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_list_workflows]
:end-before: [END how_to_list_workflows]
.. _howto/operator:WorkflowsDeleteWorkflowOperator:
Delete workflow
---------------
To delete a workflow use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsDeleteWorkflowOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_delete_workflow]
:end-before: [END how_to_delete_workflow]
.. _howto/operator:WorkflowsCreateExecutionOperator:
Create execution
----------------
To create an execution use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsCreateExecutionOperator`.
This operator is not idempotent due to API limitation.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_create_execution]
:end-before: [END how_to_create_execution]
The create operator does not wait for execution to complete. To wait for execution result use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowExecutionSensor`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_wait_for_execution]
:end-before: [END how_to_wait_for_execution]
.. _howto/operator:WorkflowsGetExecutionOperator:
Get execution
-------------
To get an execution use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsGetExecutionOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_get_execution]
:end-before: [END how_to_get_execution]
.. _howto/operator:WorkflowsListExecutionsOperator:
List executions
---------------
To list executions use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsListExecutionsOperator`.
By default this operator will return only executions for last 60 minutes.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_list_executions]
:end-before: [END how_to_list_executions]
.. _howto/operator:WorkflowsCancelExecutionOperator:
Cancel execution
----------------
To cancel an execution use
:class:`~airflow.providers.google.cloud.operators.dataproc.WorkflowsCancelExecutionOperator`.
.. exampleinclude:: /../../tests/system/providers/google/cloud/workflows/example_workflows.py
:language: python
:dedent: 4
:start-after: [START how_to_cancel_execution]
:end-before: [END how_to_cancel_execution]