blob: 47016c303bda833caa31027bb69eabc57ad36a1f [file] [log] [blame]
:mod:`airflow.contrib.sensors.gcs_sensor`
=========================================
.. py:module:: airflow.contrib.sensors.gcs_sensor
Module Contents
---------------
.. py:class:: GoogleCloudStorageObjectSensor(bucket, object, google_cloud_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)
Bases::class:`airflow.sensors.base_sensor_operator.BaseSensorOperator`
Checks for the existence of a file in Google Cloud Storage.
Create a new GoogleCloudStorageObjectSensor.
:param bucket: The Google cloud storage bucket where the object is.
:type bucket: str
:param object: The name of the object to check in the Google cloud
storage bucket.
:type object: str
:param google_cloud_storage_conn_id: The connection ID to use when
connecting to Google cloud storage.
:type google_cloud_storage_conn_id: str
:param delegate_to: The account to impersonate, if any.
For this to work, the service account making the request must have
domain-wide delegation enabled.
:type delegate_to: str
.. attribute:: template_fields
:annotation: = ['bucket', 'object']
.. attribute:: ui_color
:annotation: = #f0eee4
.. method:: poke(self, context)
.. function:: ts_function(context)
Default callback for the GoogleCloudStorageObjectUpdatedSensor. The default
behaviour is check for the object being updated after execution_date +
schedule_interval.
.. py:class:: GoogleCloudStorageObjectUpdatedSensor(bucket, object, ts_func=ts_function, google_cloud_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)
Bases::class:`airflow.sensors.base_sensor_operator.BaseSensorOperator`
Checks if an object is updated in Google Cloud Storage.
Create a new GoogleCloudStorageObjectUpdatedSensor.
:param bucket: The Google cloud storage bucket where the object is.
:type bucket: str
:param object: The name of the object to download in the Google cloud
storage bucket.
:type object: str
:param ts_func: Callback for defining the update condition. The default callback
returns execution_date + schedule_interval. The callback takes the context
as parameter.
:type ts_func: function
:param google_cloud_storage_conn_id: The connection ID to use when
connecting to Google cloud storage.
:type google_cloud_storage_conn_id: str
:param delegate_to: The account to impersonate, if any.
For this to work, the service account making the request must have domain-wide
delegation enabled.
:type delegate_to: str
.. attribute:: template_fields
:annotation: = ['bucket', 'object']
.. attribute:: template_ext
:annotation: = ['.sql']
.. attribute:: ui_color
:annotation: = #f0eee4
.. method:: poke(self, context)
.. py:class:: GoogleCloudStoragePrefixSensor(bucket, prefix, google_cloud_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs)
Bases::class:`airflow.sensors.base_sensor_operator.BaseSensorOperator`
Checks for the existence of a files at prefix in Google Cloud Storage bucket.
Create a new GoogleCloudStorageObjectSensor.
:param bucket: The Google cloud storage bucket where the object is.
:type bucket: str
:param prefix: The name of the prefix to check in the Google cloud
storage bucket.
:type prefix: str
:param google_cloud_storage_conn_id: The connection ID to use when
connecting to Google cloud storage.
:type google_cloud_storage_conn_id: str
:param delegate_to: The account to impersonate, if any.
For this to work, the service account making the request must have
domain-wide delegation enabled.
:type delegate_to: str
.. attribute:: template_fields
:annotation: = ['bucket', 'prefix']
.. attribute:: ui_color
:annotation: = #f0eee4
.. method:: poke(self, context)