| :py:mod:`airflow.providers.databricks.hooks.databricks_sql` |
| =========================================================== |
| |
| .. py:module:: airflow.providers.databricks.hooks.databricks_sql |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.databricks.hooks.databricks_sql.DatabricksSqlHook |
| |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.databricks.hooks.databricks_sql.LIST_SQL_ENDPOINTS_ENDPOINT |
| airflow.providers.databricks.hooks.databricks_sql.USER_AGENT_STRING |
| |
| |
| .. py:data:: LIST_SQL_ENDPOINTS_ENDPOINT |
| :annotation: = ['GET', 'api/2.0/sql/endpoints'] |
| |
| |
| |
| .. py:data:: USER_AGENT_STRING |
| |
| |
| |
| |
| .. py:class:: DatabricksSqlHook(databricks_conn_id = BaseDatabricksHook.default_conn_name, http_path = None, sql_endpoint_name = None, session_configuration = None, http_headers = None, catalog = None, schema = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.providers.databricks.hooks.databricks_base.BaseDatabricksHook`, :py:obj:`airflow.hooks.dbapi.DbApiHook` |
| |
| Hook to interact with Databricks SQL. |
| |
| :param databricks_conn_id: Reference to the |
| :ref:`Databricks connection <howto/connection:databricks>`. |
| :param http_path: Optional string specifying HTTP path of Databricks SQL Endpoint or cluster. |
| If not specified, it should be either specified in the Databricks connection's extra parameters, |
| or ``sql_endpoint_name`` must be specified. |
| :param sql_endpoint_name: Optional name of Databricks SQL Endpoint. If not specified, ``http_path`` |
| must be provided as described above. |
| :param session_configuration: An optional dictionary of Spark session parameters. Defaults to None. |
| If not specified, it could be specified in the Databricks connection's extra parameters. |
| :param http_headers: An optional list of (k, v) pairs that will be set as HTTP headers |
| on every request |
| :param catalog: An optional initial catalog to use. Requires DBR version 9.0+ |
| :param schema: An optional initial schema to use. Requires DBR version 9.0+ |
| :param kwargs: Additional parameters internal to Databricks SQL Connector parameters |
| |
| .. py:attribute:: hook_name |
| :annotation: = Databricks SQL |
| |
| |
| |
| .. py:method:: get_conn(self) |
| |
| Returns a Databricks SQL connection object |
| |
| |
| .. py:method:: maybe_split_sql_string(sql) |
| :staticmethod: |
| |
| Splits strings consisting of multiple SQL expressions into an |
| TODO: do we need something more sophisticated? |
| |
| :param sql: SQL string potentially consisting of multiple expressions |
| :return: list of individual expressions |
| |
| |
| .. py:method:: run(self, sql, autocommit=True, parameters=None, handler=None) |
| |
| Runs a command or a list of commands. Pass a list of sql |
| statements to the sql parameter to get them to execute |
| sequentially |
| |
| :param sql: the sql statement to be executed (str) or a list of |
| sql statements to execute |
| :param autocommit: What to set the connection's autocommit setting to |
| before executing the query. |
| :param parameters: The parameters to render the SQL query with. |
| :param handler: The result handler which is called with the result of each statement. |
| :return: query results. |
| |
| |
| .. py:method:: test_connection(self) |
| |
| Test the Databricks SQL connection by running a simple query. |
| |
| |
| .. py:method:: bulk_dump(self, table, tmp_file) |
| :abstractmethod: |
| |
| Dumps a database table into a tab-delimited file |
| |
| :param table: The name of the source table |
| :param tmp_file: The path of the target file |
| |
| |
| .. py:method:: bulk_load(self, table, tmp_file) |
| :abstractmethod: |
| |
| Loads a tab-delimited file into a database table |
| |
| :param table: The name of the target table |
| :param tmp_file: The path of the file to load into the table |
| |
| |
| |