blob: 4464ba50b8594061648d2b4c82454ea8e399f95a [file] [log] [blame]
:py:mod:`airflow.providers.google.cloud.operators.dataprep`
===========================================================
.. py:module:: airflow.providers.google.cloud.operators.dataprep
.. autoapi-nested-parse::
This module contains a Google Dataprep operator.
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.google.cloud.operators.dataprep.DataprepGetJobsForJobGroupOperator
airflow.providers.google.cloud.operators.dataprep.DataprepGetJobGroupOperator
airflow.providers.google.cloud.operators.dataprep.DataprepRunJobGroupOperator
.. py:class:: DataprepGetJobsForJobGroupOperator(*, dataprep_conn_id = 'dataprep_default', job_id, **kwargs)
Bases: :py:obj:`airflow.models.BaseOperator`
Get information about the batch jobs within a Cloud Dataprep job.
API documentation https://clouddataprep.com/documentation/api#section/Overview
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:DataprepGetJobsForJobGroupOperator`
:param job_id The ID of the job that will be requests
.. py:attribute:: template_fields
:annotation: :Sequence[str] = ['job_id']
.. py:method:: execute(context)
This is the main method to derive when creating an operator.
Context is the same dictionary used as when rendering jinja templates.
Refer to get_template_context for more context.
.. py:class:: DataprepGetJobGroupOperator(*, dataprep_conn_id = 'dataprep_default', job_group_id, embed, include_deleted, **kwargs)
Bases: :py:obj:`airflow.models.BaseOperator`
Get the specified job group.
A job group is a job that is executed from a specific node in a flow.
API documentation https://clouddataprep.com/documentation/api#section/Overview
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:DataprepGetJobGroupOperator`
:param job_group_id: The ID of the job that will be requests
:param embed: Comma-separated list of objects to pull in as part of the response
:param include_deleted: if set to "true", will include deleted objects
.. py:attribute:: template_fields
:annotation: :Sequence[str] = ['job_group_id', 'embed']
.. py:method:: execute(context)
This is the main method to derive when creating an operator.
Context is the same dictionary used as when rendering jinja templates.
Refer to get_template_context for more context.
.. py:class:: DataprepRunJobGroupOperator(*, dataprep_conn_id = 'dataprep_default', body_request, **kwargs)
Bases: :py:obj:`airflow.models.BaseOperator`
Create a ``jobGroup``, which launches the specified job as the authenticated user.
This performs the same action as clicking on the Run Job button in the application.
To get recipe_id please follow the Dataprep API documentation
https://clouddataprep.com/documentation/api#operation/runJobGroup
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:DataprepRunJobGroupOperator`
:param dataprep_conn_id: The Dataprep connection ID
:param body_request: Passed as the body_request to GoogleDataprepHook's run_job_group,
where it's the identifier for the recipe to run
.. py:attribute:: template_fields
:annotation: :Sequence[str] = ['body_request']
.. py:method:: execute(context)
This is the main method to derive when creating an operator.
Context is the same dictionary used as when rendering jinja templates.
Refer to get_template_context for more context.