| |
| |
| :mod:`airflow.contrib.sensors.gcs_sensor` |
| ========================================= |
| |
| .. py:module:: airflow.contrib.sensors.gcs_sensor |
| |
| |
| |
| |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| |
| |
| |
| |
| |
| .. py:class:: GoogleCloudStorageObjectSensor(bucket, object, google_cloud_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs) |
| |
| Bases::class:`airflow.sensors.base_sensor_operator.BaseSensorOperator` |
| |
| |
| |
| Checks for the existence of a file in Google Cloud Storage. |
| Create a new GoogleCloudStorageObjectSensor. |
| |
| :param bucket: The Google cloud storage bucket where the object is. |
| :type bucket: str |
| :param object: The name of the object to check in the Google cloud |
| storage bucket. |
| :type object: str |
| :param google_cloud_storage_conn_id: The connection ID to use when |
| connecting to Google cloud storage. |
| :type google_cloud_storage_conn_id: str |
| :param delegate_to: The account to impersonate, if any. |
| For this to work, the service account making the request must have |
| domain-wide delegation enabled. |
| :type delegate_to: str |
| |
| |
| |
| |
| |
| |
| .. attribute:: template_fields |
| :annotation: = ['bucket', 'object'] |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #f0eee4 |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. method:: poke(self, context) |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. function:: ts_function(context) |
| |
| |
| Default callback for the GoogleCloudStorageObjectUpdatedSensor. The default |
| behaviour is check for the object being updated after execution_date + |
| schedule_interval. |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. py:class:: GoogleCloudStorageObjectUpdatedSensor(bucket, object, ts_func=ts_function, google_cloud_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs) |
| |
| Bases::class:`airflow.sensors.base_sensor_operator.BaseSensorOperator` |
| |
| |
| |
| Checks if an object is updated in Google Cloud Storage. |
| Create a new GoogleCloudStorageObjectUpdatedSensor. |
| |
| :param bucket: The Google cloud storage bucket where the object is. |
| :type bucket: str |
| :param object: The name of the object to download in the Google cloud |
| storage bucket. |
| :type object: str |
| :param ts_func: Callback for defining the update condition. The default callback |
| returns execution_date + schedule_interval. The callback takes the context |
| as parameter. |
| :type ts_func: function |
| :param google_cloud_storage_conn_id: The connection ID to use when |
| connecting to Google cloud storage. |
| :type google_cloud_storage_conn_id: str |
| :param delegate_to: The account to impersonate, if any. |
| For this to work, the service account making the request must have domain-wide |
| delegation enabled. |
| :type delegate_to: str |
| |
| |
| |
| |
| |
| |
| .. attribute:: template_fields |
| :annotation: = ['bucket', 'object'] |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: template_ext |
| :annotation: = ['.sql'] |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #f0eee4 |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. method:: poke(self, context) |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. py:class:: GoogleCloudStoragePrefixSensor(bucket, prefix, google_cloud_conn_id='google_cloud_default', delegate_to=None, *args, **kwargs) |
| |
| Bases::class:`airflow.sensors.base_sensor_operator.BaseSensorOperator` |
| |
| |
| |
| Checks for the existence of a files at prefix in Google Cloud Storage bucket. |
| Create a new GoogleCloudStorageObjectSensor. |
| |
| :param bucket: The Google cloud storage bucket where the object is. |
| :type bucket: str |
| :param prefix: The name of the prefix to check in the Google cloud |
| storage bucket. |
| :type prefix: str |
| :param google_cloud_storage_conn_id: The connection ID to use when |
| connecting to Google cloud storage. |
| :type google_cloud_storage_conn_id: str |
| :param delegate_to: The account to impersonate, if any. |
| For this to work, the service account making the request must have |
| domain-wide delegation enabled. |
| :type delegate_to: str |
| |
| |
| |
| |
| |
| |
| .. attribute:: template_fields |
| :annotation: = ['bucket', 'prefix'] |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #f0eee4 |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. method:: poke(self, context) |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |