blob: 8083e55c792f43b52021efaacd3295e32d49e022 [file] [log] [blame]
:py:mod:`airflow.providers.microsoft.azure.operators.adls`
==========================================================
.. py:module:: airflow.providers.microsoft.azure.operators.adls
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.providers.microsoft.azure.operators.adls.ADLSDeleteOperator
airflow.providers.microsoft.azure.operators.adls.ADLSListOperator
.. py:class:: ADLSDeleteOperator(*, path, recursive = False, ignore_not_found = True, azure_data_lake_conn_id = 'azure_data_lake_default', **kwargs)
Bases: :py:obj:`airflow.models.BaseOperator`
Delete files in the specified path.
.. seealso::
For more information on how to use this operator, take a look at the guide:
:ref:`howto/operator:ADLSDeleteOperator`
:param path: A directory or file to remove
:param recursive: Whether to loop into directories in the location and remove the files
:param ignore_not_found: Whether to raise error if file to delete is not found
:param azure_data_lake_conn_id: Reference to the :ref:`Azure Data Lake connection<howto/connection:adl>`.
.. py:attribute:: template_fields
:annotation: :Sequence[str] = ['path']
.. py:attribute:: ui_color
:annotation: = #901dd2
.. py:method:: execute(context)
This is the main method to derive when creating an operator.
Context is the same dictionary used as when rendering jinja templates.
Refer to get_template_context for more context.
.. py:class:: ADLSListOperator(*, path, azure_data_lake_conn_id = 'azure_data_lake_default', **kwargs)
Bases: :py:obj:`airflow.models.BaseOperator`
List all files from the specified path
This operator returns a python list with the names of files which can be used by
`xcom` in the downstream tasks.
:param path: The Azure Data Lake path to find the objects. Supports glob
strings (templated)
:param azure_data_lake_conn_id: Reference to the :ref:`Azure Data Lake connection<howto/connection:adl>`.
**Example**:
The following Operator would list all the Parquet files from ``folder/output/``
folder in the specified ADLS account ::
adls_files = ADLSListOperator(
task_id='adls_files',
path='folder/output/*.parquet',
azure_data_lake_conn_id='azure_data_lake_default'
)
.. py:attribute:: template_fields
:annotation: :Sequence[str] = ['path']
.. py:attribute:: ui_color
:annotation: = #901dd2
.. py:method:: execute(context)
This is the main method to derive when creating an operator.
Context is the same dictionary used as when rendering jinja templates.
Refer to get_template_context for more context.