| |
| |
| :mod:`airflow.sensors.base_sensor_operator` |
| =========================================== |
| |
| .. py:module:: airflow.sensors.base_sensor_operator |
| |
| |
| |
| |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| |
| |
| |
| |
| |
| .. py:class:: BaseSensorOperator(poke_interval=60, timeout=60 * 60 * 24 * 7, soft_fail=False, mode='poke', *args, **kwargs) |
| |
| Bases::class:`airflow.models.BaseOperator`, :class:`airflow.models.skipmixin.SkipMixin` |
| |
| |
| |
| Sensor operators are derived from this class and inherit these attributes. |
| |
| Sensor operators keep executing at a time interval and succeed when |
| a criteria is met and fail if and when they time out. |
| |
| :param soft_fail: Set to true to mark the task as SKIPPED on failure |
| :type soft_fail: bool |
| :param poke_interval: Time in seconds that the job should wait in |
| between each tries |
| :type poke_interval: int |
| :param timeout: Time, in seconds before the task times out and fails. |
| :type timeout: int |
| :param mode: How the sensor operates. |
| Options are: ``{ poke | reschedule }``, default is ``poke``. |
| When set to ``poke`` the sensor is taking up a worker slot for its |
| whole execution time and sleeps between pokes. Use this mode if the |
| expected runtime of the sensor is short or if a short poke interval |
| is requried. |
| When set to ``reschedule`` the sensor task frees the worker slot when |
| the criteria is not yet met and it's rescheduled at a later time. Use |
| this mode if the expected time until the criteria is met is. The poke |
| interval should be more than one minute to prevent too much load on |
| the scheduler. |
| :type mode: str |
| |
| |
| |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #e6f1f2 |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: valid_modes |
| :annotation: = ['poke', 'reschedule'] |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: reschedule |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. attribute:: deps |
| |
| |
| Adds one additional dependency for all sensor operators that |
| checks if a sensor task instance can be rescheduled. |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| .. method:: _validate_input_values(self) |
| |
| |
| |
| |
| |
| |
| |
| .. method:: poke(self, context) |
| |
| |
| Function that the sensors defined while deriving this class should |
| override. |
| |
| |
| |
| |
| |
| |
| |
| .. method:: execute(self, context) |
| |
| |
| |
| |
| |
| |
| |
| .. method:: _do_skip_downstream_tasks(self, context) |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
