| :py:mod:`airflow.providers.databricks.hooks.databricks_base` |
| ============================================================ |
| |
| .. py:module:: airflow.providers.databricks.hooks.databricks_base |
| |
| .. autoapi-nested-parse:: |
| |
| Databricks hook. |
| |
| This hook enable the submitting and running of jobs to the Databricks platform. Internally the |
| operators talk to the ``api/2.0/jobs/runs/submit`` |
| `endpoint <https://docs.databricks.com/api/latest/jobs.html#runs-submit>`_. |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.databricks.hooks.databricks_base.BaseDatabricksHook |
| airflow.providers.databricks.hooks.databricks_base.BearerAuth |
| |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.databricks.hooks.databricks_base.USER_AGENT_HEADER |
| airflow.providers.databricks.hooks.databricks_base.AZURE_DEFAULT_AD_ENDPOINT |
| airflow.providers.databricks.hooks.databricks_base.AZURE_TOKEN_SERVICE_URL |
| airflow.providers.databricks.hooks.databricks_base.AZURE_METADATA_SERVICE_TOKEN_URL |
| airflow.providers.databricks.hooks.databricks_base.AZURE_METADATA_SERVICE_INSTANCE_URL |
| airflow.providers.databricks.hooks.databricks_base.TOKEN_REFRESH_LEAD_TIME |
| airflow.providers.databricks.hooks.databricks_base.AZURE_MANAGEMENT_ENDPOINT |
| airflow.providers.databricks.hooks.databricks_base.DEFAULT_DATABRICKS_SCOPE |
| |
| |
| .. py:data:: USER_AGENT_HEADER |
| |
| |
| |
| |
| .. py:data:: AZURE_DEFAULT_AD_ENDPOINT |
| :annotation: = https://login.microsoftonline.com |
| |
| |
| |
| .. py:data:: AZURE_TOKEN_SERVICE_URL |
| :annotation: = {}/{}/oauth2/token |
| |
| |
| |
| .. py:data:: AZURE_METADATA_SERVICE_TOKEN_URL |
| :annotation: = http://169.254.169.254/metadata/identity/oauth2/token |
| |
| |
| |
| .. py:data:: AZURE_METADATA_SERVICE_INSTANCE_URL |
| :annotation: = http://169.254.169.254/metadata/instance |
| |
| |
| |
| .. py:data:: TOKEN_REFRESH_LEAD_TIME |
| :annotation: = 120 |
| |
| |
| |
| .. py:data:: AZURE_MANAGEMENT_ENDPOINT |
| :annotation: = https://management.core.windows.net/ |
| |
| |
| |
| .. py:data:: DEFAULT_DATABRICKS_SCOPE |
| :annotation: = 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d |
| |
| |
| |
| .. py:class:: BaseDatabricksHook(databricks_conn_id = default_conn_name, timeout_seconds = 180, retry_limit = 3, retry_delay = 1.0, retry_args = None) |
| |
| Bases: :py:obj:`airflow.hooks.base.BaseHook` |
| |
| Base for interaction with Databricks. |
| |
| :param databricks_conn_id: Reference to the :ref:`Databricks connection <howto/connection:databricks>`. |
| :param timeout_seconds: The amount of time in seconds the requests library |
| will wait before timing-out. |
| :param retry_limit: The number of times to retry the connection in case of |
| service outages. |
| :param retry_delay: The number of seconds to wait between retries (it |
| might be a floating point number). |
| :param retry_args: An optional dictionary with arguments passed to ``tenacity.Retrying`` class. |
| |
| .. py:attribute:: conn_name_attr |
| :annotation: = databricks_conn_id |
| |
| |
| |
| .. py:attribute:: default_conn_name |
| :annotation: = databricks_default |
| |
| |
| |
| .. py:attribute:: conn_type |
| :annotation: = databricks |
| |
| |
| |
| .. py:attribute:: extra_parameters |
| :annotation: = ['token', 'host', 'use_azure_managed_identity', 'azure_ad_endpoint', 'azure_resource_id',... |
| |
| |
| |
| .. py:method:: databricks_conn(self) |
| |
| |
| .. py:method:: get_conn(self) |
| |
| Returns connection for the hook. |
| |
| |
| .. py:method:: host(self) |
| |
| |
| .. py:method:: __aenter__(self) |
| :async: |
| |
| |
| .. py:method:: __aexit__(self, *err) |
| :async: |
| |
| |
| |
| .. py:class:: BearerAuth(token) |
| |
| Bases: :py:obj:`aiohttp.BasicAuth` |
| |
| aiohttp only ships BasicAuth, for Bearer auth we need a subclass of BasicAuth. |
| |
| .. py:method:: encode(self) |
| |
| Encode credentials. |
| |
| |
| |