blob: 85e1f816ca6a1d9ef8802d1eaf9140b918009de7 [file] [log] [blame]
:mod:`airflow.providers.google.cloud.transfers.postgres_to_gcs`
===============================================================
.. py:module:: airflow.providers.google.cloud.transfers.postgres_to_gcs
.. autoapi-nested-parse::
PostgreSQL to GCS operator.
Module Contents
---------------
.. py:class:: _PostgresServerSideCursorDecorator(cursor)
Inspired by `_PrestoToGCSPrestoCursorAdapter` to keep this consistent.
Decorator for allowing description to be available for postgres cursor in case server side
cursor is used. It doesn't provide other methods except those needed in BaseSQLToGCSOperator,
which is more of a safety feature.
.. attribute:: description
Fetch first row to initialize cursor description when using server side cursor.
.. method:: __iter__(self)
.. method:: __next__(self)
.. py:class:: PostgresToGCSOperator(*, postgres_conn_id='postgres_default', use_server_side_cursor=False, cursor_itersize=2000, **kwargs)
Bases: :class:`airflow.providers.google.cloud.transfers.sql_to_gcs.BaseSQLToGCSOperator`
Copy data from Postgres to Google Cloud Storage in JSON or CSV format.
:param postgres_conn_id: Reference to a specific Postgres hook.
:type postgres_conn_id: str
:param use_server_side_cursor: If server-side cursor should be used for querying postgres.
For detailed info, check https://www.psycopg.org/docs/usage.html#server-side-cursors
:type use_server_side_cursor: bool
:param cursor_itersize: How many records are fetched at a time in case of server-side cursor.
:type cursor_itersize: int
.. attribute:: ui_color
:annotation: = #a0e08c
.. attribute:: type_map
.. method:: _unique_name(self)
.. method:: query(self)
Queries Postgres and returns a cursor to the results.
.. method:: field_to_bigquery(self, field)
.. method:: convert_type(self, value, schema_type)
Takes a value from Postgres, and converts it to a value that's safe for
JSON/Google Cloud Storage/BigQuery. Dates are converted to UTC seconds.
Decimals are converted to floats. Times are converted to seconds.