blob: ea608525755f8959e95ab05399926d246baab5b8 [file] [log] [blame]
:py:mod:`airflow.providers.amazon.aws.transfers.s3_to_redshift`
===============================================================
.. py:module:: airflow.providers.amazon.aws.transfers.s3_to_redshift
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.amazon.aws.transfers.s3_to_redshift.S3ToRedshiftOperator
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.providers.amazon.aws.transfers.s3_to_redshift.AVAILABLE_METHODS
.. py:data:: AVAILABLE_METHODS
:annotation: = ['APPEND', 'REPLACE', 'UPSERT']
.. py:class:: S3ToRedshiftOperator(*, schema, table, s3_bucket, s3_key, redshift_conn_id = 'redshift_default', aws_conn_id = 'aws_default', verify = None, column_list = None, copy_options = None, autocommit = False, method = 'APPEND', upsert_keys = None, **kwargs)
Bases: :py:obj:`airflow.models.BaseOperator`
Executes an COPY command to load files from s3 to Redshift
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:S3ToRedshiftOperator`
:param schema: reference to a specific schema in redshift database
:param table: reference to a specific table in redshift database
:param s3_bucket: reference to a specific S3 bucket
:param s3_key: reference to a specific S3 key
:param redshift_conn_id: reference to a specific redshift database
:param aws_conn_id: reference to a specific S3 connection
If the AWS connection contains 'aws_iam_role' in ``extras``
the operator will use AWS STS credentials with a token
https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-authorization.html#copy-credentials
:param verify: Whether or not to verify SSL certificates for S3 connection.
By default SSL certificates are verified.
You can provide the following values:
- ``False``: do not validate SSL certificates. SSL will still be used
(unless use_ssl is False), but SSL certificates will not be
verified.
- ``path/to/cert/bundle.pem``: A filename of the CA cert bundle to uses.
You can specify this argument if you want to use a different
CA cert bundle than the one used by botocore.
:param column_list: list of column names to load
:param copy_options: reference to a list of COPY options
:param method: Action to be performed on execution. Available ``APPEND``, ``UPSERT`` and ``REPLACE``.
:param upsert_keys: List of fields to use as key on upsert action
.. py:attribute:: template_fields
:annotation: :Sequence[str] = ['s3_bucket', 's3_key', 'schema', 'table', 'column_list', 'copy_options']
.. py:attribute:: template_ext
:annotation: :Sequence[str] = []
.. py:attribute:: ui_color
:annotation: = #99e699
.. py:method:: execute(self, context)
This is the main method to derive when creating an operator.
Context is the same dictionary used as when rendering jinja templates.
Refer to get_template_context for more context.