blob: 9829ae39981c40f9ce41034fe77e2049208864ae [file] [log] [blame]
:py:mod:`airflow.providers.databricks.hooks.databricks_base`
============================================================
.. py:module:: airflow.providers.databricks.hooks.databricks_base
.. autoapi-nested-parse::
Databricks hook.
This hook enable the submitting and running of jobs to the Databricks platform. Internally the
operators talk to the ``api/2.0/jobs/runs/submit``
`endpoint <https://docs.databricks.com/api/latest/jobs.html#runs-submit>`_.
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.databricks.hooks.databricks_base.BaseDatabricksHook
airflow.providers.databricks.hooks.databricks_base.BearerAuth
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.providers.databricks.hooks.databricks_base.AZURE_DEFAULT_AD_ENDPOINT
airflow.providers.databricks.hooks.databricks_base.AZURE_TOKEN_SERVICE_URL
airflow.providers.databricks.hooks.databricks_base.AZURE_METADATA_SERVICE_TOKEN_URL
airflow.providers.databricks.hooks.databricks_base.AZURE_METADATA_SERVICE_INSTANCE_URL
airflow.providers.databricks.hooks.databricks_base.TOKEN_REFRESH_LEAD_TIME
airflow.providers.databricks.hooks.databricks_base.AZURE_MANAGEMENT_ENDPOINT
airflow.providers.databricks.hooks.databricks_base.DEFAULT_DATABRICKS_SCOPE
.. py:data:: AZURE_DEFAULT_AD_ENDPOINT
:annotation: = https://login.microsoftonline.com
.. py:data:: AZURE_TOKEN_SERVICE_URL
:annotation: = {}/{}/oauth2/token
.. py:data:: AZURE_METADATA_SERVICE_TOKEN_URL
:annotation: = http://169.254.169.254/metadata/identity/oauth2/token
.. py:data:: AZURE_METADATA_SERVICE_INSTANCE_URL
:annotation: = http://169.254.169.254/metadata/instance
.. py:data:: TOKEN_REFRESH_LEAD_TIME
:annotation: = 120
.. py:data:: AZURE_MANAGEMENT_ENDPOINT
:annotation: = https://management.core.windows.net/
.. py:data:: DEFAULT_DATABRICKS_SCOPE
:annotation: = 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d
.. py:class:: BaseDatabricksHook(databricks_conn_id = default_conn_name, timeout_seconds = 180, retry_limit = 3, retry_delay = 1.0, retry_args = None, caller = 'Unknown')
Bases: :py:obj:`airflow.hooks.base.BaseHook`
Base for interaction with Databricks.
:param databricks_conn_id: Reference to the :ref:`Databricks connection <howto/connection:databricks>`.
:param timeout_seconds: The amount of time in seconds the requests library
will wait before timing-out.
:param retry_limit: The number of times to retry the connection in case of
service outages.
:param retry_delay: The number of seconds to wait between retries (it
might be a floating point number).
:param retry_args: An optional dictionary with arguments passed to ``tenacity.Retrying`` class.
.. py:attribute:: conn_name_attr
:annotation: = databricks_conn_id
.. py:attribute:: default_conn_name
:annotation: = databricks_default
.. py:attribute:: conn_type
:annotation: = databricks
.. py:attribute:: extra_parameters
:annotation: = ['token', 'host', 'use_azure_managed_identity', 'azure_ad_endpoint', 'azure_resource_id',...
.. py:method:: databricks_conn()
.. py:method:: get_conn()
Returns connection for the hook.
.. py:method:: user_agent_header()
.. py:method:: user_agent_value()
.. py:method:: host()
.. py:method:: __aenter__()
:async:
.. py:method:: __aexit__(*err)
:async:
.. py:class:: BearerAuth(token)
Bases: :py:obj:`aiohttp.BasicAuth`
aiohttp only ships BasicAuth, for Bearer auth we need a subclass of BasicAuth.
.. py:method:: encode()
Encode credentials.