blob: af67408cb66277aef817c33c8b633f1f97f41bdf [file] [log] [blame]
:py:mod:`airflow.providers.common.sql.hooks.sql`
================================================
.. py:module:: airflow.providers.common.sql.hooks.sql
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.common.sql.hooks.sql.ConnectorProtocol
airflow.providers.common.sql.hooks.sql.DbApiHook
Functions
~~~~~~~~~
.. autoapisummary::
airflow.providers.common.sql.hooks.sql.fetch_all_handler
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.providers.common.sql.hooks.sql.BaseForDbApiHook
.. py:function:: fetch_all_handler(cursor)
Handler for DbApiHook.run() to return results
.. py:class:: ConnectorProtocol
Bases: :py:obj:`typing_extensions.Protocol`
A protocol where you can connect to a database.
.. py:method:: connect(host, port, username, schema)
Connect to a database.
:param host: The database host to connect to.
:param port: The database port to connect to.
:param username: The database username used for the authentication.
:param schema: The database schema to connect to.
:return: the authorized connection object.
.. py:data:: BaseForDbApiHook
:annotation: :Type[airflow.hooks.base.BaseHook]
.. py:class:: DbApiHook(*args, schema = None, log_sql = True, **kwargs)
Bases: :py:obj:`airflow.hooks.dbapi.DbApiHook`
Abstract base class for sql hooks.
:param schema: Optional DB schema that overrides the schema specified in the connection. Make sure that
if you change the schema parameter value in the constructor of the derived Hook, such change
should be done before calling the ``DBApiHook.__init__()``.
:param log_sql: Whether to log SQL query when it's executed. Defaults to *True*.
.. py:attribute:: conn_name_attr
:annotation: :str
.. py:attribute:: default_conn_name
:annotation: = default_conn_id
.. py:attribute:: supports_autocommit
:annotation: = False
.. py:attribute:: connector
:annotation: :Optional[ConnectorProtocol]
.. py:method:: get_conn()
Returns a connection object
.. py:method:: get_uri()
Extract the URI from the connection.
:return: the extracted uri.
.. py:method:: get_sqlalchemy_engine(engine_kwargs=None)
Get an sqlalchemy_engine object.
:param engine_kwargs: Kwargs used in :func:`~sqlalchemy.create_engine`.
:return: the created engine.
.. py:method:: get_pandas_df(sql, parameters=None, **kwargs)
Executes the sql and returns a pandas dataframe
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param parameters: The parameters to render the SQL query with.
:param kwargs: (optional) passed into pandas.io.sql.read_sql method
.. py:method:: get_pandas_df_by_chunks(sql, parameters=None, *, chunksize, **kwargs)
Executes the sql and returns a generator
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param parameters: The parameters to render the SQL query with
:param chunksize: number of rows to include in each chunk
:param kwargs: (optional) passed into pandas.io.sql.read_sql method
.. py:method:: get_records(sql, parameters = None, **kwargs)
Executes the sql and returns a set of records.
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param parameters: The parameters to render the SQL query with.
.. py:method:: get_first(sql, parameters=None)
Executes the sql and returns the first resulting row.
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param parameters: The parameters to render the SQL query with.
.. py:method:: strip_sql_string(sql)
:staticmethod:
.. py:method:: split_sql_string(sql)
:staticmethod:
Splits string into multiple SQL expressions
:param sql: SQL string potentially consisting of multiple expressions
:return: list of individual expressions
.. py:method:: run(sql, autocommit = False, parameters = None, handler = None, split_statements = False, return_last = True)
Runs a command or a list of commands. Pass a list of sql
statements to the sql parameter to get them to execute
sequentially
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param autocommit: What to set the connection's autocommit setting to
before executing the query.
:param parameters: The parameters to render the SQL query with.
:param handler: The result handler which is called with the result of each statement.
:param split_statements: Whether to split a single SQL string into statements and run separately
:param return_last: Whether to return result for only last statement or for all after split
:return: return only result of the ALL SQL expressions if handler was provided.
.. py:method:: set_autocommit(conn, autocommit)
Sets the autocommit flag on the connection
.. py:method:: get_autocommit(conn)
Get autocommit setting for the provided connection.
Return True if conn.autocommit is set to True.
Return False if conn.autocommit is not set or set to False or conn
does not support autocommit.
:param conn: Connection to get autocommit setting from.
:return: connection autocommit setting.
:rtype: bool
.. py:method:: get_cursor()
Returns a cursor
.. py:method:: insert_rows(table, rows, target_fields=None, commit_every=1000, replace=False, **kwargs)
A generic way to insert a set of tuples into a table,
a new transaction is created every commit_every rows
:param table: Name of the target table
:param rows: The rows to insert into the table
:param target_fields: The names of the columns to fill in the table
:param commit_every: The maximum number of rows to insert in one
transaction. Set to 0 to insert all rows in one transaction.
:param replace: Whether to replace instead of insert
.. py:method:: bulk_dump(table, tmp_file)
:abstractmethod:
Dumps a database table into a tab-delimited file
:param table: The name of the source table
:param tmp_file: The path of the target file
.. py:method:: bulk_load(table, tmp_file)
:abstractmethod:
Loads a tab-delimited file into a database table
:param table: The name of the target table
:param tmp_file: The path of the file to load into the table
.. py:method:: test_connection()
Tests the connection using db-specific query