| :py:mod:`airflow.providers.google.cloud.sensors.dataproc` |
| ========================================================= |
| |
| .. py:module:: airflow.providers.google.cloud.sensors.dataproc |
| |
| .. autoapi-nested-parse:: |
| |
| This module contains a Dataproc Job sensor. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.sensors.dataproc.DataprocJobSensor |
| |
| |
| |
| |
| .. py:class:: DataprocJobSensor(*, dataproc_job_id, region, project_id = None, gcp_conn_id = 'google_cloud_default', wait_timeout = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.sensors.base.BaseSensorOperator` |
| |
| Check for the state of a previously submitted Dataproc job. |
| |
| :param dataproc_job_id: The Dataproc job ID to poll. (templated) |
| :param region: Required. The Cloud Dataproc region in which to handle the request. (templated) |
| :param project_id: The ID of the google cloud project in which |
| to create the cluster. (templated) |
| :param gcp_conn_id: The connection ID to use connecting to Google Cloud Platform. |
| :param wait_timeout: How many seconds wait for job to be ready. |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['project_id', 'region', 'dataproc_job_id'] |
| |
| |
| |
| .. py:attribute:: ui_color |
| :annotation: = #f0eee4 |
| |
| |
| |
| .. py:method:: execute(context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| .. py:method:: poke(context) |
| |
| Function that the sensors defined while deriving this class should |
| override. |
| |
| |
| |
