blob: a643bb688506c129a98393501c7c8e9eb5d1b3a0 [file] [log] [blame]
:py:mod:`airflow.providers.databricks.hooks.databricks_sql`
===========================================================
.. py:module:: airflow.providers.databricks.hooks.databricks_sql
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.databricks.hooks.databricks_sql.DatabricksSqlHook
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.providers.databricks.hooks.databricks_sql.LIST_SQL_ENDPOINTS_ENDPOINT
.. py:data:: LIST_SQL_ENDPOINTS_ENDPOINT
:annotation: = ['GET', 'api/2.0/sql/endpoints']
.. py:class:: DatabricksSqlHook(databricks_conn_id = BaseDatabricksHook.default_conn_name, http_path = None, sql_endpoint_name = None, session_configuration = None, http_headers = None, catalog = None, schema = None, caller = 'DatabricksSqlHook', **kwargs)
Bases: :py:obj:`airflow.providers.databricks.hooks.databricks_base.BaseDatabricksHook`, :py:obj:`airflow.providers.common.sql.hooks.sql.DbApiHook`
Hook to interact with Databricks SQL.
:param databricks_conn_id: Reference to the
:ref:`Databricks connection <howto/connection:databricks>`.
:param http_path: Optional string specifying HTTP path of Databricks SQL Endpoint or cluster.
If not specified, it should be either specified in the Databricks connection's extra parameters,
or ``sql_endpoint_name`` must be specified.
:param sql_endpoint_name: Optional name of Databricks SQL Endpoint. If not specified, ``http_path``
must be provided as described above.
:param session_configuration: An optional dictionary of Spark session parameters. Defaults to None.
If not specified, it could be specified in the Databricks connection's extra parameters.
:param http_headers: An optional list of (k, v) pairs that will be set as HTTP headers
on every request
:param catalog: An optional initial catalog to use. Requires DBR version 9.0+
:param schema: An optional initial schema to use. Requires DBR version 9.0+
:param kwargs: Additional parameters internal to Databricks SQL Connector parameters
.. py:attribute:: hook_name
:annotation: = Databricks SQL
.. py:method:: get_conn()
Returns a Databricks SQL connection object
.. py:method:: run(sql, autocommit = False, parameters = None, handler = None, split_statements = True, return_last = True)
Runs a command or a list of commands. Pass a list of sql
statements to the sql parameter to get them to execute
sequentially
:param sql: the sql statement to be executed (str) or a list of
sql statements to execute
:param autocommit: What to set the connection's autocommit setting to
before executing the query.
:param parameters: The parameters to render the SQL query with.
:param handler: The result handler which is called with the result of each statement.
:param split_statements: Whether to split a single SQL string into statements and run separately
:param return_last: Whether to return result for only last statement or for all after split
:return: return only result of the LAST SQL expression if handler was provided.
.. py:method:: test_connection()
Test the Databricks SQL connection by running a simple query.
.. py:method:: bulk_dump(table, tmp_file)
:abstractmethod:
Dumps a database table into a tab-delimited file
:param table: The name of the source table
:param tmp_file: The path of the target file
.. py:method:: bulk_load(table, tmp_file)
:abstractmethod:
Loads a tab-delimited file into a database table
:param table: The name of the target table
:param tmp_file: The path of the file to load into the table