blob: 0dbb4b22137a19c28973fe83be8a999cbf64c48e [file] [log] [blame]
:py:mod:`airflow.providers.google.cloud.transfers.mysql_to_gcs`
===============================================================
.. py:module:: airflow.providers.google.cloud.transfers.mysql_to_gcs
.. autoapi-nested-parse::
MySQL to GCS operator.
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.google.cloud.transfers.mysql_to_gcs.MySQLToGCSOperator
.. py:class:: MySQLToGCSOperator(*, mysql_conn_id='mysql_default', ensure_utc=False, **kwargs)
Bases: :py:obj:`airflow.providers.google.cloud.transfers.sql_to_gcs.BaseSQLToGCSOperator`
Copy data from MySQL to Google Cloud Storage in JSON, CSV or Parquet format.
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:MySQLToGCSOperator`
:param mysql_conn_id: Reference to :ref:`mysql connection id <howto/connection:mysql>`.
:param ensure_utc: Ensure TIMESTAMP columns exported as UTC. If set to
`False`, TIMESTAMP columns will be exported using the MySQL server's
default timezone.
.. py:attribute:: ui_color
:annotation: = #a0e08c
.. py:attribute:: type_map
.. py:method:: query()
Queries mysql and returns a cursor to the results.
.. py:method:: field_to_bigquery(field)
Convert a DBAPI field to BigQuery schema format.
.. py:method:: convert_type(value, schema_type, **kwargs)
Takes a value from MySQLdb, and converts it to a value that's safe for
JSON/Google Cloud Storage/BigQuery.
* Datetimes are converted to `str(value)` (`datetime.isoformat(' ')`)
strings.
* Times are converted to `str((datetime.min + value).time())` strings.
* Decimals are converted to floats.
* Dates are converted to ISO formatted strings if given schema_type is
DATE, or `datetime.isoformat(' ')` strings otherwise.
* Binary type fields are converted to integer if given schema_type is
INTEGER, or encoded with base64 otherwise. Imported BYTES data must
be base64-encoded according to BigQuery documentation:
https://cloud.google.com/bigquery/data-types
:param value: MySQLdb column value
:param schema_type: BigQuery data type