blob: ff393b9a211f6f0c094b0540b82a7ae3bb9db64f [file] [log] [blame]
:py:mod:`airflow.providers.databricks.hooks.databricks_sql`
===========================================================
.. py:module:: airflow.providers.databricks.hooks.databricks_sql
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.databricks.hooks.databricks_sql.DatabricksSqlHook
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.providers.databricks.hooks.databricks_sql.LIST_SQL_ENDPOINTS_ENDPOINT
airflow.providers.databricks.hooks.databricks_sql.USER_AGENT_STRING
.. py:data:: LIST_SQL_ENDPOINTS_ENDPOINT
:annotation: = ['GET', 'api/2.0/sql/endpoints']
.. py:data:: USER_AGENT_STRING
.. py:class:: DatabricksSqlHook(databricks_conn_id = BaseDatabricksHook.default_conn_name, http_path = None, sql_endpoint_name = None, session_configuration = None, http_headers = None, catalog = None, schema = None, **kwargs)
Bases: :py:obj:`airflow.providers.databricks.hooks.databricks_base.BaseDatabricksHook`, :py:obj:`airflow.hooks.dbapi.DbApiHook`
Hook to interact with Databricks SQL.
:param databricks_conn_id: Reference to the
:ref:`Databricks connection <howto/connection:databricks>`.
:param http_path: Optional string specifying HTTP path of Databricks SQL Endpoint or cluster.
If not specified, it should be either specified in the Databricks connection's extra parameters,
or ``sql_endpoint_name`` must be specified.
:param sql_endpoint_name: Optional name of Databricks SQL Endpoint. If not specified, ``http_path``
must be provided as described above.
:param session_configuration: An optional dictionary of Spark session parameters. Defaults to None.
If not specified, it could be specified in the Databricks connection's extra parameters.
:param http_headers: An optional list of (k, v) pairs that will be set as HTTP headers
on every request
:param catalog: An optional initial catalog to use. Requires DBR version 9.0+
:param schema: An optional initial schema to use. Requires DBR version 9.0+
:param kwargs: Additional parameters internal to Databricks SQL Connector parameters
.. py:attribute:: hook_name
:annotation: = Databricks SQL
.. py:method:: get_conn(self)
Returns a Databricks SQL connection object
.. py:method:: maybe_split_sql_string(sql)
:staticmethod:
Splits strings consisting of multiple SQL expressions into an
TODO: do we need something more sophisticated?
:param sql: SQL string potentially consisting of multiple expressions
:return: list of individual expressions
.. py:method:: run(self, sql, autocommit=True, parameters=None, handler=None)
Runs a command or a list of commands. Pass a list of sql
statements to the sql parameter to get them to execute
sequentially
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param autocommit: What to set the connection's autocommit setting to
before executing the query.
:param parameters: The parameters to render the SQL query with.
:param handler: The result handler which is called with the result of each statement.
:return: query results.
.. py:method:: test_connection(self)
Test the Databricks SQL connection by running a simple query.
.. py:method:: bulk_dump(self, table, tmp_file)
:abstractmethod:
Dumps a database table into a tab-delimited file
:param table: The name of the source table
:param tmp_file: The path of the target file
.. py:method:: bulk_load(self, table, tmp_file)
:abstractmethod:
Loads a tab-delimited file into a database table
:param table: The name of the target table
:param tmp_file: The path of the file to load into the table