| :py:mod:`airflow.models.mappedoperator` |
| ======================================= |
| |
| .. py:module:: airflow.models.mappedoperator |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.models.mappedoperator.OperatorPartial |
| airflow.models.mappedoperator.MappedOperator |
| |
| |
| |
| Functions |
| ~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.models.mappedoperator.validate_mapping_kwargs |
| airflow.models.mappedoperator.prevent_duplicates |
| airflow.models.mappedoperator.ensure_xcomarg_return_value |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.models.mappedoperator.ValidationSource |
| |
| |
| .. py:data:: ValidationSource |
| |
| |
| |
| |
| .. py:function:: validate_mapping_kwargs(op, func, value) |
| |
| |
| .. py:function:: prevent_duplicates(kwargs1, kwargs2, *, fail_reason) |
| |
| |
| .. py:function:: ensure_xcomarg_return_value(arg) |
| |
| |
| .. py:class:: OperatorPartial |
| |
| An "intermediate state" returned by ``BaseOperator.partial()``. |
| |
| This only exists at DAG-parsing time; the only intended usage is for the |
| user to call ``.expand()`` on it at some point (usually in a method chain) to |
| create a ``MappedOperator`` to add into the DAG. |
| |
| .. py:attribute:: operator_class |
| :annotation: :type[airflow.models.baseoperator.BaseOperator] |
| |
| |
| |
| .. py:attribute:: kwargs |
| :annotation: :dict[str, Any] |
| |
| |
| |
| .. py:method:: __attrs_post_init__() |
| |
| |
| .. py:method:: __repr__() |
| |
| Return repr(self). |
| |
| |
| .. py:method:: __del__() |
| |
| |
| .. py:method:: expand(**mapped_kwargs) |
| |
| |
| .. py:method:: expand_kwargs(kwargs, *, strict = True) |
| |
| |
| |
| .. py:class:: MappedOperator(context=None) |
| |
| Bases: :py:obj:`airflow.models.abstractoperator.AbstractOperator` |
| |
| Object representing a mapped operator in a DAG. |
| |
| .. py:attribute:: operator_class |
| :annotation: :type[BaseOperator] | dict[str, Any] |
| |
| |
| |
| .. py:attribute:: expand_input |
| :annotation: :airflow.models.expandinput.ExpandInput |
| |
| |
| |
| .. py:attribute:: partial_kwargs |
| :annotation: :dict[str, Any] |
| |
| |
| |
| .. py:attribute:: task_id |
| :annotation: :str |
| |
| |
| |
| .. py:attribute:: params |
| :annotation: :dict | None |
| |
| |
| |
| .. py:attribute:: deps |
| :annotation: :frozenset[airflow.ti_deps.deps.base_ti_dep.BaseTIDep] |
| |
| |
| |
| .. py:attribute:: operator_extra_links |
| :annotation: :Collection[airflow.models.baseoperator.BaseOperatorLink] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] |
| |
| |
| |
| .. py:attribute:: template_fields |
| :annotation: :Collection[str] |
| |
| |
| |
| .. py:attribute:: template_fields_renderers |
| :annotation: :dict[str, str] |
| |
| |
| |
| .. py:attribute:: ui_color |
| :annotation: :str |
| |
| |
| |
| .. py:attribute:: ui_fgcolor |
| :annotation: :str |
| |
| |
| |
| .. py:attribute:: dag |
| :annotation: :DAG | None |
| |
| |
| |
| .. py:attribute:: task_group |
| :annotation: :TaskGroup | None |
| |
| |
| |
| .. py:attribute:: start_date |
| :annotation: :pendulum.DateTime | None |
| |
| |
| |
| .. py:attribute:: end_date |
| :annotation: :pendulum.DateTime | None |
| |
| |
| |
| .. py:attribute:: upstream_task_ids |
| :annotation: :set[str] |
| |
| |
| |
| .. py:attribute:: downstream_task_ids |
| :annotation: :set[str] |
| |
| |
| |
| .. py:attribute:: is_mapped |
| :annotation: :ClassVar[bool] = True |
| |
| |
| |
| .. py:attribute:: subdag |
| :annotation: :None |
| |
| |
| |
| .. py:attribute:: HIDE_ATTRS_FROM_UI |
| :annotation: :ClassVar[frozenset[str]] |
| |
| |
| |
| .. py:method:: __hash__() |
| |
| Return hash(self). |
| |
| |
| .. py:method:: __repr__() |
| |
| Return repr(self). |
| |
| |
| .. py:method:: __attrs_post_init__() |
| |
| |
| .. py:method:: get_serialized_fields() |
| :classmethod: |
| |
| |
| .. py:method:: deps_for(operator_class) |
| :staticmethod: |
| |
| |
| .. py:method:: task_type() |
| :property: |
| |
| Implementing Operator. |
| |
| |
| .. py:method:: operator_name() |
| :property: |
| |
| |
| .. py:method:: inherits_from_empty_operator() |
| :property: |
| |
| Implementing Operator. |
| |
| |
| .. py:method:: roots() |
| :property: |
| |
| Implementing DAGNode. |
| |
| |
| .. py:method:: leaves() |
| :property: |
| |
| Implementing DAGNode. |
| |
| |
| .. py:method:: owner() |
| :property: |
| |
| |
| .. py:method:: email() |
| :property: |
| |
| |
| .. py:method:: trigger_rule() |
| :property: |
| |
| |
| .. py:method:: depends_on_past() |
| :property: |
| |
| |
| .. py:method:: ignore_first_depends_on_past() |
| :property: |
| |
| |
| .. py:method:: wait_for_downstream() |
| :property: |
| |
| |
| .. py:method:: retries() |
| :property: |
| |
| |
| .. py:method:: queue() |
| :property: |
| |
| |
| .. py:method:: pool() |
| :property: |
| |
| |
| .. py:method:: pool_slots() |
| :property: |
| |
| |
| .. py:method:: execution_timeout() |
| :property: |
| |
| |
| .. py:method:: max_retry_delay() |
| :property: |
| |
| |
| .. py:method:: retry_delay() |
| :property: |
| |
| |
| .. py:method:: retry_exponential_backoff() |
| :property: |
| |
| |
| .. py:method:: priority_weight() |
| :property: |
| |
| |
| .. py:method:: weight_rule() |
| :property: |
| |
| |
| .. py:method:: sla() |
| :property: |
| |
| |
| .. py:method:: max_active_tis_per_dag() |
| :property: |
| |
| |
| .. py:method:: resources() |
| :property: |
| |
| |
| .. py:method:: on_execute_callback() |
| :property: |
| |
| |
| .. py:method:: on_failure_callback() |
| :property: |
| |
| |
| .. py:method:: on_retry_callback() |
| :property: |
| |
| |
| .. py:method:: on_success_callback() |
| :property: |
| |
| |
| .. py:method:: run_as_user() |
| :property: |
| |
| |
| .. py:method:: executor_config() |
| :property: |
| |
| |
| .. py:method:: inlets() |
| :property: |
| |
| |
| .. py:method:: outlets() |
| :property: |
| |
| |
| .. py:method:: doc() |
| :property: |
| |
| |
| .. py:method:: doc_md() |
| :property: |
| |
| |
| .. py:method:: doc_json() |
| :property: |
| |
| |
| .. py:method:: doc_yaml() |
| :property: |
| |
| |
| .. py:method:: doc_rst() |
| :property: |
| |
| |
| .. py:method:: get_dag() |
| |
| Implementing Operator. |
| |
| |
| .. py:method:: output() |
| :property: |
| |
| Returns reference to XCom pushed by current operator |
| |
| |
| .. py:method:: serialize_for_task_group() |
| |
| Implementing DAGNode. |
| |
| |
| .. py:method:: unmap(resolve) |
| |
| Get the "normal" Operator after applying the current mapping. |
| |
| The *resolve* argument is only used if ``operator_class`` is a real |
| class, i.e. if this operator is not serialized. If ``operator_class`` is |
| not a class (i.e. this DAG has been deserialized), this returns a |
| SerializedBaseOperator that "looks like" the actual unmapping result. |
| |
| If *resolve* is a two-tuple (context, session), the information is used |
| to resolve the mapped arguments into init arguments. If it is a mapping, |
| no resolving happens, the mapping directly provides those init arguments |
| resolved from mapped kwargs. |
| |
| :meta private: |
| |
| |
| .. py:method:: expand_mapped_task(run_id, *, session) |
| |
| Create the mapped task instances for mapped task. |
| |
| :return: The newly created mapped TaskInstances (if any) in ascending order by map index, and the |
| maximum map_index. |
| |
| |
| .. py:method:: prepare_for_execution() |
| |
| |
| .. py:method:: iter_mapped_dependencies() |
| |
| Upstream dependencies that provide XComs used by this task for task mapping. |
| |
| |
| .. py:method:: parse_time_mapped_ti_count() |
| |
| Number of mapped TaskInstances that can be created at DagRun create time. |
| |
| :return: None if non-literal mapped arg encountered, or the total |
| number of mapped TIs this task should have. |
| |
| |
| .. py:method:: run_time_mapped_ti_count(run_id, *, session) |
| |
| Number of mapped TaskInstances that can be created at run time. |
| |
| :return: None if upstream tasks are not complete yet, or the total |
| number of mapped TIs this task should have. |
| |
| |
| .. py:method:: render_template_fields(context, jinja_env = None) |
| |
| Template all attributes listed in template_fields. |
| |
| If the operator is mapped, this should return the unmapped, fully |
| rendered, and map-expanded operator. The mapped operator should not be |
| modified. |
| |
| If the operator is not mapped, this should modify the operator in-place |
| and return either *None* (for backwards compatibility) or *self*. |
| |
| |
| |
