blob: c87ab0aac49a96a2be6c49d704063ed914bfc8d4 [file] [log] [blame]
:py:mod:`airflow.providers.microsoft.azure.hooks.synapse`
=========================================================
.. py:module:: airflow.providers.microsoft.azure.hooks.synapse
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.microsoft.azure.hooks.synapse.AzureSynapseSparkBatchRunStatus
airflow.providers.microsoft.azure.hooks.synapse.AzureSynapseHook
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.providers.microsoft.azure.hooks.synapse.Credentials
.. py:data:: Credentials
.. py:class:: AzureSynapseSparkBatchRunStatus
Azure Synapse Spark Job operation statuses.
.. py:attribute:: NOT_STARTED
:annotation: = not_started
.. py:attribute:: STARTING
:annotation: = starting
.. py:attribute:: RUNNING
:annotation: = running
.. py:attribute:: IDLE
:annotation: = idle
.. py:attribute:: BUSY
:annotation: = busy
.. py:attribute:: SHUTTING_DOWN
:annotation: = shutting_down
.. py:attribute:: ERROR
:annotation: = error
.. py:attribute:: DEAD
:annotation: = dead
.. py:attribute:: KILLED
:annotation: = killed
.. py:attribute:: SUCCESS
:annotation: = success
.. py:attribute:: TERMINAL_STATUSES
.. py:class:: AzureSynapseHook(azure_synapse_conn_id = default_conn_name, spark_pool = '')
Bases: :py:obj:`airflow.hooks.base.BaseHook`
A hook to interact with Azure Synapse.
:param azure_synapse_conn_id: The :ref:`Azure Synapse connection id<howto/connection:synapse>`.
:param spark_pool: The Apache Spark pool used to submit the job
.. py:attribute:: conn_type
:annotation: :str = azure_synapse
.. py:attribute:: conn_name_attr
:annotation: :str = azure_synapse_conn_id
.. py:attribute:: default_conn_name
:annotation: :str = azure_synapse_default
.. py:attribute:: hook_name
:annotation: :str = Azure Synapse
.. py:method:: get_connection_form_widgets()
:staticmethod:
Returns connection widgets to add to connection form
.. py:method:: get_ui_field_behaviour()
:staticmethod:
Returns custom field behaviour
.. py:method:: get_conn()
Returns connection for the hook.
.. py:method:: run_spark_job(payload)
Run a job in an Apache Spark pool.
:param payload: Livy compatible payload which represents the spark job that a user wants to submit.
.. py:method:: get_job_run_status()
Get the job run status.
.. py:method:: wait_for_job_run_status(job_id, expected_statuses, check_interval = 60, timeout = 60 * 60 * 24 * 7)
Waits for a job run to match an expected status.
:param job_id: The job run identifier.
:param expected_statuses: The desired status(es) to check against a job run's current status.
:param check_interval: Time in seconds to check on a job run's status.
:param timeout: Time in seconds to wait for a job to reach a terminal status or the expected
status.
.. py:method:: cancel_job_run(job_id)
Cancel the spark job run.
:param job_id: The synapse spark job identifier.