| :mod:`airflow.sensors.external_task_sensor` |
| =========================================== |
| |
| .. py:module:: airflow.sensors.external_task_sensor |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: ExternalTaskSensor(external_dag_id, external_task_id=None, allowed_states=None, execution_delta=None, execution_date_fn=None, check_existence=False, *args, **kwargs) |
| |
| Bases: :class:`airflow.sensors.base_sensor_operator.BaseSensorOperator` |
| |
| Waits for a different DAG or a task in a different DAG to complete for a |
| specific execution_date |
| |
| :param external_dag_id: The dag_id that contains the task you want to |
| wait for |
| :type external_dag_id: str |
| :param external_task_id: The task_id that contains the task you want to |
| wait for. If ``None`` (default value) the sensor waits for the DAG |
| :type external_task_id: str or None |
| :param allowed_states: list of allowed states, default is ``['success']`` |
| :type allowed_states: list |
| :param execution_delta: time difference with the previous execution to |
| look at, the default is the same execution_date as the current task or DAG. |
| For yesterday, use [positive!] datetime.timedelta(days=1). Either |
| execution_delta or execution_date_fn can be passed to |
| ExternalTaskSensor, but not both. |
| :type execution_delta: datetime.timedelta |
| :param execution_date_fn: function that receives the current execution date |
| and returns the desired execution dates to query. Either execution_delta |
| or execution_date_fn can be passed to ExternalTaskSensor, but not both. |
| :type execution_date_fn: callable |
| :param check_existence: Set to `True` to check if the external task exists (when |
| external_task_id is not None) or check if the DAG to wait for exists (when |
| external_task_id is None), and immediately cease waiting if the external task |
| or DAG does not exist (default value: False). |
| :type check_existence: bool |
| |
| .. attribute:: template_fields |
| :annotation: = ['external_dag_id', 'external_task_id'] |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #19647e |
| |
| |
| |
| |
| .. method:: poke(self, context, session=None) |
| |
| |
| |
| |
| .. method:: _handle_execution_date_fn(self, context) |
| |
| This function is to handle backwards compatibility with how this operator was |
| previously where it only passes the execution date, but also allow for the newer |
| implementation to pass all context through as well, to allow for more sophisticated |
| returns of dates to return. |
| Namely, this function check the number of arguments in the execution_date_fn |
| signature and if its 1, treat the legacy way, if it's 2, pass the context as |
| the 2nd argument, and if its more, throw an exception. |
| |
| |
| |
| |
| .. py:class:: ExternalTaskMarker(external_dag_id, external_task_id, execution_date='{{ execution_date.isoformat() }}', recursion_depth=10, *args, **kwargs) |
| |
| Bases: :class:`airflow.operators.dummy_operator.DummyOperator` |
| |
| Use this operator to indicate that a task on a different DAG depends on this task. |
| When this task is cleared with "Recursive" selected, Airflow will clear the task on |
| the other DAG and its downstream tasks recursively. Transitive dependencies are followed |
| until the recursion_depth is reached. |
| |
| :param external_dag_id: The dag_id that contains the dependent task that needs to be cleared. |
| :type external_dag_id: str |
| :param external_task_id: The task_id of the dependent task that needs to be cleared. |
| :type external_task_id: str |
| :param execution_date: The execution_date of the dependent task that needs to be cleared. |
| :type execution_date: str or datetime.datetime |
| :param recursion_depth: The maximum level of transitive dependencies allowed. Default is 10. |
| This is mostly used for preventing cyclic dependencies. It is fine to increase |
| this number if necessary. However, too many levels of transitive dependencies will make |
| it slower to clear tasks in the web UI. |
| |
| .. attribute:: template_fields |
| :annotation: = ['external_dag_id', 'external_task_id', 'execution_date'] |
| |
| |
| |
| .. attribute:: ui_color |
| :annotation: = #19647e |
| |
| |
| |
| .. attribute:: __serialized_fields |
| |
| |
| |
| |
| |
| .. classmethod:: get_serialized_fields(cls) |
| |
| Serialized ExternalTaskMarker contain exactly these fields + templated_fields . |
| |
| |
| |
| |