| :mod:`airflow.contrib.operators.gcs_list_operator` |
| ================================================== |
| |
| .. py:module:: airflow.contrib.operators.gcs_list_operator |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: GoogleCloudStorageListOperator(bucket, prefix=None, delimiter=None, google_cloud_storage_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs) |
| |
| Bases: :class:`airflow.models.BaseOperator` |
| |
| List all objects from the bucket with the give string prefix and delimiter in name. |
| |
| This operator returns a python list with the name of objects which can be used by |
| `xcom` in the downstream task. |
| |
| :param bucket: The Google cloud storage bucket to find the objects. (templated) |
| :type bucket: str |
| :param prefix: Prefix string which filters objects whose name begin with |
| this prefix. (templated) |
| :type prefix: str |
| :param delimiter: The delimiter by which you want to filter the objects. (templated) |
| For e.g to lists the CSV files from in a directory in GCS you would use |
| delimiter='.csv'. |
| :type delimiter: str |
| :param google_cloud_storage_conn_id: The connection ID to use when |
| connecting to Google cloud storage. |
| :type google_cloud_storage_conn_id: str |
| :param delegate_to: The account to impersonate, if any. |
| For this to work, the service account making the request must have |
| domain-wide delegation enabled. |
| :type delegate_to: str |
| |
| **Example**: |
| The following Operator would list all the Avro files from ``sales/sales-2017`` |
| folder in ``data`` bucket. :: |
| |
| GCS_Files = GoogleCloudStorageListOperator( |
| task_id='GCS_Files', |
| bucket='data', |
| prefix='sales/sales-2017/', |
| delimiter='.avro', |
| google_cloud_storage_conn_id=google_cloud_conn_id |
| ) |
| |
| .. attribute:: template_fields |
| :annotation: :Iterable[str] = ['bucket', 'prefix', 'delimiter'] |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #f0eee4 |
| |
| |
| |
| |
| .. method:: execute(self, context) |
| |
| |
| |
| |