| :py:mod:`airflow.providers.google.cloud.transfers.bigquery_to_mysql` |
| ==================================================================== |
| |
| .. py:module:: airflow.providers.google.cloud.transfers.bigquery_to_mysql |
| |
| .. autoapi-nested-parse:: |
| |
| This module contains Google BigQuery to MySQL operator. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.google.cloud.transfers.bigquery_to_mysql.BigQueryToMySqlOperator |
| |
| |
| |
| |
| .. py:class:: BigQueryToMySqlOperator(*, dataset_table, mysql_table, selected_fields = None, gcp_conn_id = 'google_cloud_default', mysql_conn_id = 'mysql_default', database = None, delegate_to = None, replace = False, batch_size = 1000, location = None, impersonation_chain = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Fetches the data from a BigQuery table (alternatively fetch data for selected columns) |
| and insert that data into a MySQL table. |
| |
| |
| .. note:: |
| If you pass fields to ``selected_fields`` which are in different order than the |
| order of columns already in |
| BQ table, the data will still be in the order of BQ table. |
| For example if the BQ table has 3 columns as |
| ``[A,B,C]`` and you pass 'B,A' in the ``selected_fields`` |
| the data would still be of the form ``'A,B'`` and passed through this form |
| to MySQL |
| |
| **Example**: :: |
| |
| transfer_data = BigQueryToMySqlOperator( |
| task_id='task_id', |
| dataset_table='origin_bq_table', |
| mysql_table='dest_table_name', |
| replace=True, |
| ) |
| |
| :param dataset_table: A dotted ``<dataset>.<table>``: the big query table of origin |
| :param selected_fields: List of fields to return (comma-separated). If |
| unspecified, all fields are returned. |
| :param gcp_conn_id: reference to a specific Google Cloud hook. |
| :param delegate_to: The account to impersonate using domain-wide delegation of authority, |
| if any. For this to work, the service account making the request must have |
| domain-wide delegation enabled. |
| :param mysql_conn_id: Reference to :ref:`mysql connection id <howto/connection:mysql>`. |
| :param database: name of database which overwrite defined one in connection |
| :param replace: Whether to replace instead of insert |
| :param batch_size: The number of rows to take in each batch |
| :param location: The location used for the operation. |
| :param impersonation_chain: Optional service account to impersonate using short-term |
| credentials, or chained list of accounts required to get the access_token |
| of the last account in the list, which will be impersonated in the request. |
| If set as a string, the account must grant the originating account |
| the Service Account Token Creator IAM role. |
| If set as a sequence, the identities from the list must grant |
| Service Account Token Creator IAM role to the directly preceding identity, with first |
| account from the list granting this role to the originating account (templated). |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['dataset_id', 'table_id', 'mysql_table', 'impersonation_chain'] |
| |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| |