docs-archive/apache-airflow-providers-amazon/1.0.0/_sources/_api/airflow/providers/amazon/aws/transfers/google_api_to_s3/index.rst.txt

:mod:`airflow.providers.amazon.aws.transfers.google_api_to_s3`
==============================================================

.. py:module:: airflow.providers.amazon.aws.transfers.google_api_to_s3

.. autoapi-nested-parse::

   This module allows you to transfer data from any Google API endpoint into a S3 Bucket.



Module Contents
---------------

.. py:class:: GoogleApiToS3Operator(*, google_api_service_name: str, google_api_service_version: str, google_api_endpoint_path: str, google_api_endpoint_params: dict, s3_destination_key: str, google_api_response_via_xcom: Optional[str] = None, google_api_endpoint_params_via_xcom: Optional[str] = None, google_api_endpoint_params_via_xcom_task_ids: Optional[str] = None, google_api_pagination: bool = False, google_api_num_retries: int = 0, s3_overwrite: bool = False, gcp_conn_id: str = 'google_cloud_default', delegate_to: Optional[str] = None, aws_conn_id: str = 'aws_default', google_impersonation_chain: Optional[Union[str, Sequence[str]]] = None, **kwargs)

   Bases: :class:`airflow.models.BaseOperator`

   Basic class for transferring data from a Google API endpoint into a S3 Bucket.

   This discovery-based operator use
   :class:`~airflow.providers.google.common.hooks.discovery_api.GoogleDiscoveryApiHook` to communicate
   with Google Services via the
   `Google API Python Client <https://github.com/googleapis/google-api-python-client>`__.
   Please note that this library is in maintenance mode hence it won't fully support Google Cloud in
   the future.
   Therefore it is recommended that you use the custom Google Cloud Service Operators for working
   with the Google Cloud Platform.

   :param google_api_service_name: The specific API service that is being requested.
   :type google_api_service_name: str
   :param google_api_service_version: The version of the API that is being requested.
   :type google_api_service_version: str
   :param google_api_endpoint_path: The client libraries path to the api call's executing method.
       For example: 'analyticsreporting.reports.batchGet'

       .. note:: See https://developers.google.com/apis-explorer
           for more information on which methods are available.

   :type google_api_endpoint_path: str
   :param google_api_endpoint_params: The params to control the corresponding endpoint result.
   :type google_api_endpoint_params: dict
   :param s3_destination_key: The url where to put the data retrieved from the endpoint in S3.
   :type s3_destination_key: str
   :param google_api_response_via_xcom: Can be set to expose the google api response to xcom.
   :type google_api_response_via_xcom: str
   :param google_api_endpoint_params_via_xcom: If set to a value this value will be used as a key
       for pulling from xcom and updating the google api endpoint params.
   :type google_api_endpoint_params_via_xcom: str
   :param google_api_endpoint_params_via_xcom_task_ids: Task ids to filter xcom by.
   :type google_api_endpoint_params_via_xcom_task_ids: str or list of str
   :param google_api_pagination: If set to True Pagination will be enabled for this request
       to retrieve all data.

       .. note:: This means the response will be a list of responses.

   :type google_api_pagination: bool
   :param google_api_num_retries: Define the number of retries for the google api requests being made
       if it fails.
   :type google_api_num_retries: int
   :param s3_overwrite: Specifies whether the s3 file will be overwritten if exists.
   :type s3_overwrite: bool
   :param gcp_conn_id: The connection ID to use when fetching connection info.
   :type gcp_conn_id: str
   :param delegate_to: Google account to impersonate using domain-wide delegation of authority,
       if any. For this to work, the service account making the request must have
       domain-wide delegation enabled.
   :type delegate_to: str
   :param aws_conn_id: The connection id specifying the authentication information for the S3 Bucket.
   :type aws_conn_id: str
   :param google_impersonation_chain: Optional Google service account to impersonate using
       short-term credentials, or chained list of accounts required to get the access_token
       of the last account in the list, which will be impersonated in the request.
       If set as a string, the account must grant the originating account
       the Service Account Token Creator IAM role.
       If set as a sequence, the identities from the list must grant
       Service Account Token Creator IAM role to the directly preceding identity, with first
       account from the list granting this role to the originating account (templated).
   :type google_impersonation_chain: Union[str, Sequence[str]]

   .. attribute:: template_fields
      :annotation: = ['google_api_endpoint_params', 's3_destination_key', 'google_impersonation_chain']

      

   .. attribute:: template_ext
      :annotation: = []

      

   .. attribute:: ui_color
      :annotation: = #cc181e

      

   
   .. method:: execute(self, context)

      Transfers Google APIs json data to S3.

      :param context: The context that is being provided when executing.
      :type context: dict



   
   .. method:: _retrieve_data_from_google_api(self)



   
   .. method:: _load_data_to_s3(self, data: dict)



   
   .. method:: _update_google_api_endpoint_params_via_xcom(self, task_instance: TaskInstance)



   
   .. method:: _expose_google_api_response_via_xcom(self, task_instance: TaskInstance, data: dict)