blob: 9c751b1aa51a355f15982d0c5a3c29d2aa588502 [file] [log] [blame]
:py:mod:`airflow.models.mappedoperator`
=======================================
.. py:module:: airflow.models.mappedoperator
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.models.mappedoperator.OperatorPartial
airflow.models.mappedoperator.MappedOperator
Functions
~~~~~~~~~
.. autoapisummary::
airflow.models.mappedoperator.get_mappable_types
airflow.models.mappedoperator.validate_mapping_kwargs
airflow.models.mappedoperator.prevent_duplicates
airflow.models.mappedoperator.ensure_xcomarg_return_value
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.models.mappedoperator.Mappable
airflow.models.mappedoperator.ValidationSource
airflow.models.mappedoperator.MAPPABLE_LITERAL_TYPES
.. py:data:: Mappable
.. py:data:: ValidationSource
.. py:data:: MAPPABLE_LITERAL_TYPES
.. py:function:: get_mappable_types()
.. py:function:: validate_mapping_kwargs(op, func, value)
.. py:function:: prevent_duplicates(kwargs1, kwargs2, *, fail_reason)
.. py:function:: ensure_xcomarg_return_value(arg)
.. py:class:: OperatorPartial
An "intermediate state" returned by ``BaseOperator.partial()``.
This only exists at DAG-parsing time; the only intended usage is for the
user to call ``.expand()`` on it at some point (usually in a method chain) to
create a ``MappedOperator`` to add into the DAG.
.. py:attribute:: operator_class
:annotation: :Type[airflow.models.baseoperator.BaseOperator]
.. py:attribute:: kwargs
:annotation: :Dict[str, Any]
.. py:method:: __attrs_post_init__(self)
.. py:method:: __repr__(self)
Return repr(self).
.. py:method:: __del__(self)
.. py:method:: expand(self, **mapped_kwargs)
.. py:class:: MappedOperator(context=None)
Bases: :py:obj:`airflow.models.abstractoperator.AbstractOperator`
Object representing a mapped operator in a DAG.
.. py:attribute:: operator_class
:annotation: :Union[Type[airflow.models.baseoperator.BaseOperator], Dict[str, Any]]
.. py:attribute:: mapped_kwargs
:annotation: :Dict[str, Mappable]
.. py:attribute:: partial_kwargs
:annotation: :Dict[str, Any]
.. py:attribute:: task_id
:annotation: :str
.. py:attribute:: params
:annotation: :Optional[dict]
.. py:attribute:: deps
:annotation: :FrozenSet[airflow.ti_deps.deps.base_ti_dep.BaseTIDep]
.. py:attribute:: operator_extra_links
:annotation: :Collection[airflow.models.baseoperator.BaseOperatorLink]
.. py:attribute:: template_ext
:annotation: :Sequence[str]
.. py:attribute:: template_fields
:annotation: :Collection[str]
.. py:attribute:: template_fields_renderers
:annotation: :Dict[str, str]
.. py:attribute:: ui_color
:annotation: :str
.. py:attribute:: ui_fgcolor
:annotation: :str
.. py:attribute:: dag
:annotation: :Optional[airflow.models.dag.DAG]
.. py:attribute:: task_group
:annotation: :Optional[airflow.utils.task_group.TaskGroup]
.. py:attribute:: start_date
:annotation: :Optional[pendulum.DateTime]
.. py:attribute:: end_date
:annotation: :Optional[pendulum.DateTime]
.. py:attribute:: upstream_task_ids
:annotation: :Set[str]
.. py:attribute:: downstream_task_ids
:annotation: :Set[str]
.. py:attribute:: is_mapped
:annotation: :ClassVar[bool] = True
.. py:attribute:: subdag
:annotation: :None
.. py:attribute:: HIDE_ATTRS_FROM_UI
:annotation: :ClassVar[FrozenSet[str]]
.. py:method:: __hash__(self)
Return hash(self).
.. py:method:: __repr__(self)
Return repr(self).
.. py:method:: __attrs_post_init__(self)
.. py:method:: get_serialized_fields(cls)
:classmethod:
.. py:method:: deps_for(operator_class)
:staticmethod:
.. py:method:: task_type(self)
:property:
Implementing Operator.
.. py:method:: inherits_from_empty_operator(self)
:property:
Implementing Operator.
.. py:method:: roots(self)
:property:
Implementing DAGNode.
.. py:method:: leaves(self)
:property:
Implementing DAGNode.
.. py:method:: owner(self)
:property:
.. py:method:: email(self)
:property:
.. py:method:: trigger_rule(self)
:property:
.. py:method:: depends_on_past(self)
:property:
.. py:method:: wait_for_downstream(self)
:property:
.. py:method:: retries(self)
:property:
.. py:method:: queue(self)
:property:
.. py:method:: pool(self)
:property:
.. py:method:: pool_slots(self)
:property:
.. py:method:: execution_timeout(self)
:property:
.. py:method:: retry_delay(self)
:property:
.. py:method:: retry_exponential_backoff(self)
:property:
.. py:method:: priority_weight(self)
:property:
.. py:method:: weight_rule(self)
:property:
.. py:method:: sla(self)
:property:
.. py:method:: max_active_tis_per_dag(self)
:property:
.. py:method:: resources(self)
:property:
.. py:method:: on_execute_callback(self)
:property:
.. py:method:: on_failure_callback(self)
:property:
.. py:method:: on_retry_callback(self)
:property:
.. py:method:: on_success_callback(self)
:property:
.. py:method:: run_as_user(self)
:property:
.. py:method:: executor_config(self)
:property:
.. py:method:: inlets(self)
:property:
.. py:method:: outlets(self)
:property:
.. py:method:: get_dag(self)
Implementing Operator.
.. py:method:: serialize_for_task_group(self)
Implementing DAGNode.
.. py:method:: unmap(self, unmap_kwargs = None)
Get the "normal" Operator after applying the current mapping.
If ``operator_class`` is not a class (i.e. this DAG has been deserialized) then this will return a
SerializedBaseOperator that aims to "look like" the real operator.
:param unmap_kwargs: Override the args to pass to the Operator constructor. Only used when
``operator_class`` is still an actual class.
:meta private:
.. py:method:: expand_mapped_task(self, run_id, *, session)
Create the mapped task instances for mapped task.
:return: The newly created mapped TaskInstances (if any) in ascending order by map index, and the
maximum map_index.
.. py:method:: prepare_for_execution(self)
.. py:method:: render_template_fields(self, context, jinja_env = None)
Template all attributes listed in template_fields.
Different from the BaseOperator implementation, this renders the
template fields on the *unmapped* BaseOperator.
:param context: Dict with values to apply on content
:param jinja_env: Jinja environment
:return: The unmapped, populated BaseOperator
.. py:method:: iter_mapped_dependencies(self)
Upstream dependencies that provide XComs used by this task for task mapping.
.. py:method:: parse_time_mapped_ti_count(self)
Number of mapped TaskInstances that can be created at DagRun create time.
:return: None if non-literal mapped arg encountered, or else total number of mapped TIs this task
should have
.. py:method:: run_time_mapped_ti_count(self, run_id, *, session)
Number of mapped TaskInstances that can be created at run time, or None if upstream tasks are not
complete yet.
:return: None if upstream tasks are not complete yet, or else total number of mapped TIs this task
should have