blob: 268a10dec19f6fd0d5fdb71afe2f0bc1201bd1ea [file] [log] [blame]
:mod:`airflow.providers.microsoft.azure.hooks.azure_data_factory`
=================================================================
.. py:module:: airflow.providers.microsoft.azure.hooks.azure_data_factory
Module Contents
---------------
.. function:: provide_targeted_factory(func: Callable) -> Callable
Provide the targeted factory to the decorated function in case it isn't specified.
If ``resource_group_name`` or ``factory_name`` is not provided it defaults to the value specified in
the connection extras.
.. py:class:: AzureDataFactoryHook(conn_id: Optional[str] = default_conn_name)
Bases: :class:`airflow.hooks.base.BaseHook`
A hook to interact with Azure Data Factory.
:param conn_id: The Azure Data Factory connection id.
.. attribute:: conn_type
:annotation: :str = azure_data_factory
.. attribute:: conn_name_attr
:annotation: :str = azure_data_factory_conn_id
.. attribute:: default_conn_name
:annotation: :str = azure_data_factory_default
.. attribute:: hook_name
:annotation: :str = Azure Data Factory
.. method:: get_conn(self)
.. method:: get_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Get the factory.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The factory.
.. method:: _factory_exists(self, resource_group_name, factory_name)
Return whether or not the factory already exists.
.. method:: update_factory(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Update the factory.
:param factory: The factory resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the factory does not exist.
:return: The factory.
.. method:: create_factory(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Create the factory.
:param factory: The factory resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the factory already exists.
:return: The factory.
.. method:: delete_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Delete the factory.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: get_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Get the linked service.
:param linked_service_name: The linked service name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The linked service.
.. method:: _linked_service_exists(self, resource_group_name, factory_name, linked_service_name)
Return whether or not the linked service already exists.
.. method:: update_linked_service(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Update the linked service.
:param linked_service_name: The linked service name.
:param linked_service: The linked service resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the linked service does not exist.
:return: The linked service.
.. method:: create_linked_service(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Create the linked service.
:param linked_service_name: The linked service name.
:param linked_service: The linked service resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the linked service already exists.
:return: The linked service.
.. method:: delete_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Delete the linked service.
:param linked_service_name: The linked service name.
:param resource_group_name: The linked service name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: get_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Get the dataset.
:param dataset_name: The dataset name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The dataset.
.. method:: _dataset_exists(self, resource_group_name, factory_name, dataset_name)
Return whether or not the dataset already exists.
.. method:: update_dataset(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Update the dataset.
:param dataset_name: The dataset name.
:param dataset: The dataset resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the dataset does not exist.
:return: The dataset.
.. method:: create_dataset(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Create the dataset.
:param dataset_name: The dataset name.
:param dataset: The dataset resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the dataset already exists.
:return: The dataset.
.. method:: delete_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Delete the dataset.
:param dataset_name: The dataset name.
:param resource_group_name: The dataset name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: get_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Get the pipeline.
:param pipeline_name: The pipeline name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The pipeline.
.. method:: _pipeline_exists(self, resource_group_name, factory_name, pipeline_name)
Return whether or not the pipeline already exists.
.. method:: update_pipeline(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Update the pipeline.
:param pipeline_name: The pipeline name.
:param pipeline: The pipeline resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the pipeline does not exist.
:return: The pipeline.
.. method:: create_pipeline(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Create the pipeline.
:param pipeline_name: The pipeline name.
:param pipeline: The pipeline resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the pipeline already exists.
:return: The pipeline.
.. method:: delete_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Delete the pipeline.
:param pipeline_name: The pipeline name.
:param resource_group_name: The pipeline name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: run_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Run a pipeline.
:param pipeline_name: The pipeline name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The pipeline run.
.. method:: get_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Get the pipeline run.
:param run_id: The pipeline run identifier.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The pipeline run.
.. method:: cancel_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Cancel the pipeline run.
:param run_id: The pipeline run identifier.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: get_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Get the trigger.
:param trigger_name: The trigger name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: The trigger.
.. method:: _trigger_exists(self, resource_group_name, factory_name, trigger_name)
Return whether or not the trigger already exists.
.. method:: update_trigger(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Update the trigger.
:param trigger_name: The trigger name.
:param trigger: The trigger resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the trigger does not exist.
:return: The trigger.
.. method:: create_trigger(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Create the trigger.
:param trigger_name: The trigger name.
:param trigger: The trigger resource definition.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:raise AirflowException: If the trigger already exists.
:return: The trigger.
.. method:: delete_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Delete the trigger.
:param trigger_name: The trigger name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: start_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Start the trigger.
:param trigger_name: The trigger name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: An Azure operation poller.
.. method:: stop_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Stop the trigger.
:param trigger_name: The trigger name.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
:return: An Azure operation poller.
.. method:: rerun_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Rerun the trigger.
:param trigger_name: The trigger name.
:param run_id: The trigger run identifier.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.
.. method:: cancel_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config)
Cancel the trigger.
:param trigger_name: The trigger name.
:param run_id: The trigger run identifier.
:param resource_group_name: The resource group name.
:param factory_name: The factory name.
:param config: Extra parameters for the ADF client.