| :mod:`airflow.providers.microsoft.azure.hooks.azure_data_factory` |
| ================================================================= |
| |
| .. py:module:: airflow.providers.microsoft.azure.hooks.azure_data_factory |
| |
| |
| Module Contents |
| --------------- |
| |
| .. function:: provide_targeted_factory(func: Callable) -> Callable |
| Provide the targeted factory to the decorated function in case it isn't specified. |
| |
| If ``resource_group_name`` or ``factory_name`` is not provided it defaults to the value specified in |
| the connection extras. |
| |
| |
| .. py:class:: AzureDataFactoryHook(conn_id: Optional[str] = default_conn_name) |
| |
| Bases: :class:`airflow.hooks.base.BaseHook` |
| |
| A hook to interact with Azure Data Factory. |
| |
| :param conn_id: The Azure Data Factory connection id. |
| |
| .. attribute:: conn_type |
| :annotation: :str = azure_data_factory |
| |
| |
| |
| .. attribute:: conn_name_attr |
| :annotation: :str = azure_data_factory_conn_id |
| |
| |
| |
| .. attribute:: default_conn_name |
| :annotation: :str = azure_data_factory_default |
| |
| |
| |
| .. attribute:: hook_name |
| :annotation: :str = Azure Data Factory |
| |
| |
| |
| |
| .. method:: get_conn(self) |
| |
| |
| |
| |
| .. method:: get_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Get the factory. |
| |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The factory. |
| |
| |
| |
| |
| .. method:: _factory_exists(self, resource_group_name, factory_name) |
| |
| Return whether or not the factory already exists. |
| |
| |
| |
| |
| .. method:: update_factory(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Update the factory. |
| |
| :param factory: The factory resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the factory does not exist. |
| :return: The factory. |
| |
| |
| |
| |
| .. method:: create_factory(self, factory: Factory, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Create the factory. |
| |
| :param factory: The factory resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the factory already exists. |
| :return: The factory. |
| |
| |
| |
| |
| .. method:: delete_factory(self, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Delete the factory. |
| |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: get_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Get the linked service. |
| |
| :param linked_service_name: The linked service name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The linked service. |
| |
| |
| |
| |
| .. method:: _linked_service_exists(self, resource_group_name, factory_name, linked_service_name) |
| |
| Return whether or not the linked service already exists. |
| |
| |
| |
| |
| .. method:: update_linked_service(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Update the linked service. |
| |
| :param linked_service_name: The linked service name. |
| :param linked_service: The linked service resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the linked service does not exist. |
| :return: The linked service. |
| |
| |
| |
| |
| .. method:: create_linked_service(self, linked_service_name: str, linked_service: LinkedService, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Create the linked service. |
| |
| :param linked_service_name: The linked service name. |
| :param linked_service: The linked service resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the linked service already exists. |
| :return: The linked service. |
| |
| |
| |
| |
| .. method:: delete_linked_service(self, linked_service_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Delete the linked service. |
| |
| :param linked_service_name: The linked service name. |
| :param resource_group_name: The linked service name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: get_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Get the dataset. |
| |
| :param dataset_name: The dataset name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The dataset. |
| |
| |
| |
| |
| .. method:: _dataset_exists(self, resource_group_name, factory_name, dataset_name) |
| |
| Return whether or not the dataset already exists. |
| |
| |
| |
| |
| .. method:: update_dataset(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Update the dataset. |
| |
| :param dataset_name: The dataset name. |
| :param dataset: The dataset resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the dataset does not exist. |
| :return: The dataset. |
| |
| |
| |
| |
| .. method:: create_dataset(self, dataset_name: str, dataset: Dataset, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Create the dataset. |
| |
| :param dataset_name: The dataset name. |
| :param dataset: The dataset resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the dataset already exists. |
| :return: The dataset. |
| |
| |
| |
| |
| .. method:: delete_dataset(self, dataset_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Delete the dataset. |
| |
| :param dataset_name: The dataset name. |
| :param resource_group_name: The dataset name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: get_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Get the pipeline. |
| |
| :param pipeline_name: The pipeline name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The pipeline. |
| |
| |
| |
| |
| .. method:: _pipeline_exists(self, resource_group_name, factory_name, pipeline_name) |
| |
| Return whether or not the pipeline already exists. |
| |
| |
| |
| |
| .. method:: update_pipeline(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Update the pipeline. |
| |
| :param pipeline_name: The pipeline name. |
| :param pipeline: The pipeline resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the pipeline does not exist. |
| :return: The pipeline. |
| |
| |
| |
| |
| .. method:: create_pipeline(self, pipeline_name: str, pipeline: PipelineResource, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Create the pipeline. |
| |
| :param pipeline_name: The pipeline name. |
| :param pipeline: The pipeline resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the pipeline already exists. |
| :return: The pipeline. |
| |
| |
| |
| |
| .. method:: delete_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Delete the pipeline. |
| |
| :param pipeline_name: The pipeline name. |
| :param resource_group_name: The pipeline name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: run_pipeline(self, pipeline_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Run a pipeline. |
| |
| :param pipeline_name: The pipeline name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The pipeline run. |
| |
| |
| |
| |
| .. method:: get_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Get the pipeline run. |
| |
| :param run_id: The pipeline run identifier. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The pipeline run. |
| |
| |
| |
| |
| .. method:: cancel_pipeline_run(self, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Cancel the pipeline run. |
| |
| :param run_id: The pipeline run identifier. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: get_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Get the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: The trigger. |
| |
| |
| |
| |
| .. method:: _trigger_exists(self, resource_group_name, factory_name, trigger_name) |
| |
| Return whether or not the trigger already exists. |
| |
| |
| |
| |
| .. method:: update_trigger(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Update the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param trigger: The trigger resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the trigger does not exist. |
| :return: The trigger. |
| |
| |
| |
| |
| .. method:: create_trigger(self, trigger_name: str, trigger: Trigger, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Create the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param trigger: The trigger resource definition. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :raise AirflowException: If the trigger already exists. |
| :return: The trigger. |
| |
| |
| |
| |
| .. method:: delete_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Delete the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: start_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Start the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: An Azure operation poller. |
| |
| |
| |
| |
| .. method:: stop_trigger(self, trigger_name: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Stop the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| :return: An Azure operation poller. |
| |
| |
| |
| |
| .. method:: rerun_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Rerun the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param run_id: The trigger run identifier. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |
| .. method:: cancel_trigger(self, trigger_name: str, run_id: str, resource_group_name: Optional[str] = None, factory_name: Optional[str] = None, **config) |
| |
| Cancel the trigger. |
| |
| :param trigger_name: The trigger name. |
| :param run_id: The trigger run identifier. |
| :param resource_group_name: The resource group name. |
| :param factory_name: The factory name. |
| :param config: Extra parameters for the ADF client. |
| |
| |
| |
| |