| :py:mod:`airflow.providers.google.cloud.operators.dataprep` |
| =========================================================== |
| |
| .. py:module:: airflow.providers.google.cloud.operators.dataprep |
| |
| .. autoapi-nested-parse:: |
| |
| This module contains a Google Dataprep operator. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.operators.dataprep.DataprepGetJobsForJobGroupOperator |
| airflow.providers.google.cloud.operators.dataprep.DataprepGetJobGroupOperator |
| airflow.providers.google.cloud.operators.dataprep.DataprepRunJobGroupOperator |
| |
| |
| |
| |
| .. py:class:: DataprepGetJobsForJobGroupOperator(*, dataprep_conn_id = 'dataprep_default', job_id, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Get information about the batch jobs within a Cloud Dataprep job. |
| API documentation https://clouddataprep.com/documentation/api#section/Overview |
| |
| .. seealso:: |
| For more information on how to use this operator, take a look at the guide: |
| :ref:`howto/operator:DataprepGetJobsForJobGroupOperator` |
| |
| :param job_id The ID of the job that will be requests |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['job_id'] |
| |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |
| .. py:class:: DataprepGetJobGroupOperator(*, dataprep_conn_id = 'dataprep_default', job_group_id, embed, include_deleted, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Get the specified job group. |
| A job group is a job that is executed from a specific node in a flow. |
| API documentation https://clouddataprep.com/documentation/api#section/Overview |
| |
| .. seealso:: |
| For more information on how to use this operator, take a look at the guide: |
| :ref:`howto/operator:DataprepGetJobGroupOperator` |
| |
| :param job_group_id: The ID of the job that will be requests |
| :param embed: Comma-separated list of objects to pull in as part of the response |
| :param include_deleted: if set to "true", will include deleted objects |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['job_group_id', 'embed'] |
| |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |
| .. py:class:: DataprepRunJobGroupOperator(*, dataprep_conn_id = 'dataprep_default', body_request, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Create a ``jobGroup``, which launches the specified job as the authenticated user. |
| This performs the same action as clicking on the Run Job button in the application. |
| To get recipe_id please follow the Dataprep API documentation |
| https://clouddataprep.com/documentation/api#operation/runJobGroup |
| |
| .. seealso:: |
| For more information on how to use this operator, take a look at the guide: |
| :ref:`howto/operator:DataprepRunJobGroupOperator` |
| |
| :param dataprep_conn_id: The Dataprep connection ID |
| :param body_request: Passed as the body_request to GoogleDataprepHook's run_job_group, |
| where it's the identifier for the recipe to run |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['body_request'] |
| |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |