| :py:mod:`airflow.providers.google.cloud.operators.dataplex` |
| =========================================================== |
| |
| .. py:module:: airflow.providers.google.cloud.operators.dataplex |
| |
| .. autoapi-nested-parse:: |
| |
| This module contains Google Dataplex operators. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.operators.dataplex.DataplexCreateTaskOperator |
| airflow.providers.google.cloud.operators.dataplex.DataplexDeleteTaskOperator |
| airflow.providers.google.cloud.operators.dataplex.DataplexListTasksOperator |
| airflow.providers.google.cloud.operators.dataplex.DataplexGetTaskOperator |
| |
| |
| |
| |
| .. py:class:: DataplexCreateTaskOperator(project_id, region, lake_id, body, dataplex_task_id, validate_only = None, api_version = 'v1', retry = DEFAULT, timeout = None, metadata = (), gcp_conn_id = 'google_cloud_default', delegate_to = None, impersonation_chain = None, asynchronous = False, *args, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Creates a task resource within a lake. |
| |
| :param project_id: Required. The ID of the Google Cloud project that the task belongs to. |
| :param region: Required. The ID of the Google Cloud region that the task belongs to. |
| :param lake_id: Required. The ID of the Google Cloud lake that the task belongs to. |
| :param body: Required. The Request body contains an instance of Task. |
| :param dataplex_task_id: Required. Task identifier. |
| :param validate_only: Optional. Only validate the request, but do not perform mutations. The default is |
| false. |
| :param api_version: The version of the api that will be requested for example 'v3'. |
| :param retry: A retry object used to retry requests. If `None` is specified, requests |
| will not be retried. |
| :param timeout: The amount of time, in seconds, to wait for the request to complete. |
| Note that if `retry` is specified, the timeout applies to each individual attempt. |
| :param metadata: Additional metadata that is provided to the method. |
| :param gcp_conn_id: The connection ID to use when fetching connection info. |
| :param delegate_to: The account to impersonate, if any. For this to work, the service accountmaking the |
| request must have domain-wide delegation enabled. |
| :param impersonation_chain: Optional service account to impersonate using short-term |
| credentials, or chained list of accounts required to get the access_token |
| of the last account in the list, which will be impersonated in the request. |
| If set as a string, the account must grant the originating account |
| the Service Account Token Creator IAM role. |
| If set as a sequence, the identities from the list must grant |
| Service Account Token Creator IAM role to the directly preceding identity, with first |
| account from the list granting this role to the originating account (templated). |
| :param asynchronous: Flag informing should the Dataplex task be created asynchronously. |
| This is useful for long running creating tasks and |
| waiting on them asynchronously using the DataplexTaskSensor |
| |
| .. py:attribute:: template_fields |
| :annotation: = ['project_id', 'dataplex_task_id', 'body', 'validate_only', 'delegate_to', 'impersonation_chain'] |
| |
| |
| |
| .. py:attribute:: template_fields_renderers |
| |
| |
| |
| |
| .. py:attribute:: operator_extra_links |
| |
| |
| |
| |
| .. py:method:: execute(context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |
| .. py:class:: DataplexDeleteTaskOperator(project_id, region, lake_id, dataplex_task_id, api_version = 'v1', retry = DEFAULT, timeout = None, metadata = (), gcp_conn_id = 'google_cloud_default', delegate_to = None, impersonation_chain = None, *args, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Delete the task resource. |
| |
| :param project_id: Required. The ID of the Google Cloud project that the task belongs to. |
| :param region: Required. The ID of the Google Cloud region that the task belongs to. |
| :param lake_id: Required. The ID of the Google Cloud lake that the task belongs to. |
| :param dataplex_task_id: Required. Task identifier. |
| :param api_version: The version of the api that will be requested for example 'v3'. |
| :param retry: A retry object used to retry requests. If `None` is specified, requests |
| will not be retried. |
| :param timeout: The amount of time, in seconds, to wait for the request to complete. |
| Note that if `retry` is specified, the timeout applies to each individual attempt. |
| :param metadata: Additional metadata that is provided to the method. |
| :param gcp_conn_id: The connection ID to use when fetching connection info. |
| :param delegate_to: The account to impersonate, if any. For this to work, the service accountmaking the |
| request must have domain-wide delegation enabled. |
| :param impersonation_chain: Optional service account to impersonate using short-term |
| credentials, or chained list of accounts required to get the access_token |
| of the last account in the list, which will be impersonated in the request. |
| If set as a string, the account must grant the originating account |
| the Service Account Token Creator IAM role. |
| If set as a sequence, the identities from the list must grant |
| Service Account Token Creator IAM role to the directly preceding identity, with first |
| account from the list granting this role to the originating account (templated). |
| |
| .. py:attribute:: template_fields |
| :annotation: = ['project_id', 'dataplex_task_id', 'delegate_to', 'impersonation_chain'] |
| |
| |
| |
| .. py:method:: execute(context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |
| .. py:class:: DataplexListTasksOperator(project_id, region, lake_id, page_size = None, page_token = None, filter = None, order_by = None, api_version = 'v1', retry = DEFAULT, timeout = None, metadata = (), gcp_conn_id = 'google_cloud_default', delegate_to = None, impersonation_chain = None, *args, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Lists tasks under the given lake. |
| |
| :param project_id: Required. The ID of the Google Cloud project that the task belongs to. |
| :param region: Required. The ID of the Google Cloud region that the task belongs to. |
| :param lake_id: Required. The ID of the Google Cloud lake that the task belongs to. |
| :param page_size: Optional. Maximum number of tasks to return. The service may return fewer than this |
| value. If unspecified, at most 10 tasks will be returned. The maximum value is 1000; values above 1000 |
| will be coerced to 1000. |
| :param page_token: Optional. Page token received from a previous ListZones call. Provide this to retrieve |
| the subsequent page. When paginating, all other parameters provided to ListZones must match the call |
| that provided the page token. |
| :param filter: Optional. Filter request. |
| :param order_by: Optional. Order by fields for the result. |
| :param api_version: The version of the api that will be requested for example 'v3'. |
| :param retry: A retry object used to retry requests. If `None` is specified, requests |
| will not be retried. |
| :param timeout: The amount of time, in seconds, to wait for the request to complete. |
| Note that if `retry` is specified, the timeout applies to each individual attempt. |
| :param metadata: Additional metadata that is provided to the method. |
| :param gcp_conn_id: The connection ID to use when fetching connection info. |
| :param delegate_to: The account to impersonate, if any. For this to work, the service accountmaking the |
| request must have domain-wide delegation enabled. |
| :param impersonation_chain: Optional service account to impersonate using short-term |
| credentials, or chained list of accounts required to get the access_token |
| of the last account in the list, which will be impersonated in the request. |
| If set as a string, the account must grant the originating account |
| the Service Account Token Creator IAM role. |
| If set as a sequence, the identities from the list must grant |
| Service Account Token Creator IAM role to the directly preceding identity, with first |
| account from the list granting this role to the originating account (templated). |
| |
| .. py:attribute:: template_fields |
| :annotation: = ['project_id', 'page_size', 'page_token', 'filter', 'order_by', 'delegate_to', 'impersonation_chain'] |
| |
| |
| |
| .. py:attribute:: operator_extra_links |
| |
| |
| |
| |
| .. py:method:: execute(context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |
| .. py:class:: DataplexGetTaskOperator(project_id, region, lake_id, dataplex_task_id, api_version = 'v1', retry = DEFAULT, timeout = None, metadata = (), gcp_conn_id = 'google_cloud_default', delegate_to = None, impersonation_chain = None, *args, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Get task resource. |
| |
| :param project_id: Required. The ID of the Google Cloud project that the task belongs to. |
| :param region: Required. The ID of the Google Cloud region that the task belongs to. |
| :param lake_id: Required. The ID of the Google Cloud lake that the task belongs to. |
| :param dataplex_task_id: Required. Task identifier. |
| :param api_version: The version of the api that will be requested for example 'v3'. |
| :param retry: A retry object used to retry requests. If `None` is specified, requests |
| will not be retried. |
| :param timeout: The amount of time, in seconds, to wait for the request to complete. |
| Note that if `retry` is specified, the timeout applies to each individual attempt. |
| :param metadata: Additional metadata that is provided to the method. |
| :param gcp_conn_id: The connection ID to use when fetching connection info. |
| :param delegate_to: The account to impersonate, if any. For this to work, the service accountmaking the |
| request must have domain-wide delegation enabled. |
| :param impersonation_chain: Optional service account to impersonate using short-term |
| credentials, or chained list of accounts required to get the access_token |
| of the last account in the list, which will be impersonated in the request. |
| If set as a string, the account must grant the originating account |
| the Service Account Token Creator IAM role. |
| If set as a sequence, the identities from the list must grant |
| Service Account Token Creator IAM role to the directly preceding identity, with first |
| account from the list granting this role to the originating account (templated). |
| |
| .. py:attribute:: template_fields |
| :annotation: = ['project_id', 'dataplex_task_id', 'delegate_to', 'impersonation_chain'] |
| |
| |
| |
| .. py:attribute:: operator_extra_links |
| |
| |
| |
| |
| .. py:method:: execute(context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |
