blob: 6f6cf5036b6d915ec48174389c3f0f226c84dd73 [file] [log] [blame]
:py:mod:`airflow.models.mappedoperator`
=======================================
.. py:module:: airflow.models.mappedoperator
Module Contents
---------------
Classes
~~~~~~~
.. autoapisummary::
airflow.models.mappedoperator.OperatorPartial
airflow.models.mappedoperator.MappedOperator
Functions
~~~~~~~~~
.. autoapisummary::
airflow.models.mappedoperator.get_mappable_types
airflow.models.mappedoperator.validate_mapping_kwargs
airflow.models.mappedoperator.prevent_duplicates
airflow.models.mappedoperator.ensure_xcomarg_return_value
Attributes
~~~~~~~~~~
.. autoapisummary::
airflow.models.mappedoperator.Mappable
airflow.models.mappedoperator.ValidationSource
airflow.models.mappedoperator.MAPPABLE_LITERAL_TYPES
.. py:data:: Mappable
.. py:data:: ValidationSource
.. py:data:: MAPPABLE_LITERAL_TYPES
.. py:function:: get_mappable_types()
.. py:function:: validate_mapping_kwargs(op, func, value)
.. py:function:: prevent_duplicates(kwargs1, kwargs2, *, fail_reason)
.. py:function:: ensure_xcomarg_return_value(arg)
.. py:class:: OperatorPartial
An "intermediate state" returned by ``BaseOperator.partial()``.
This only exists at DAG-parsing time; the only intended usage is for the
user to call ``.expand()`` on it at some point (usually in a method chain) to
create a ``MappedOperator`` to add into the DAG.
.. py:attribute:: operator_class
:annotation: :Type[airflow.models.baseoperator.BaseOperator]
.. py:attribute:: kwargs
:annotation: :Dict[str, Any]
.. py:method:: __attrs_post_init__()
.. py:method:: __repr__()
Return repr(self).
.. py:method:: __del__()
.. py:method:: expand(**mapped_kwargs)
.. py:class:: MappedOperator(context=None)
Bases: :py:obj:`airflow.models.abstractoperator.AbstractOperator`
Object representing a mapped operator in a DAG.
.. py:attribute:: operator_class
:annotation: :Union[Type[airflow.models.baseoperator.BaseOperator], Dict[str, Any]]
.. py:attribute:: mapped_kwargs
:annotation: :Dict[str, Mappable]
.. py:attribute:: partial_kwargs
:annotation: :Dict[str, Any]
.. py:attribute:: task_id
:annotation: :str
.. py:attribute:: params
:annotation: :Optional[dict]
.. py:attribute:: deps
:annotation: :FrozenSet[airflow.ti_deps.deps.base_ti_dep.BaseTIDep]
.. py:attribute:: operator_extra_links
:annotation: :Collection[airflow.models.baseoperator.BaseOperatorLink]
.. py:attribute:: template_ext
:annotation: :Sequence[str]
.. py:attribute:: template_fields
:annotation: :Collection[str]
.. py:attribute:: template_fields_renderers
:annotation: :Dict[str, str]
.. py:attribute:: ui_color
:annotation: :str
.. py:attribute:: ui_fgcolor
:annotation: :str
.. py:attribute:: dag
:annotation: :Optional[airflow.models.dag.DAG]
.. py:attribute:: task_group
:annotation: :Optional[airflow.utils.task_group.TaskGroup]
.. py:attribute:: start_date
:annotation: :Optional[pendulum.DateTime]
.. py:attribute:: end_date
:annotation: :Optional[pendulum.DateTime]
.. py:attribute:: upstream_task_ids
:annotation: :Set[str]
.. py:attribute:: downstream_task_ids
:annotation: :Set[str]
.. py:attribute:: is_mapped
:annotation: :ClassVar[bool] = True
.. py:attribute:: subdag
:annotation: :None
.. py:attribute:: HIDE_ATTRS_FROM_UI
:annotation: :ClassVar[FrozenSet[str]]
.. py:method:: __hash__()
Return hash(self).
.. py:method:: __repr__()
Return repr(self).
.. py:method:: __attrs_post_init__()
.. py:method:: get_serialized_fields()
:classmethod:
.. py:method:: deps_for(operator_class)
:staticmethod:
.. py:method:: task_type()
:property:
Implementing Operator.
.. py:method:: inherits_from_empty_operator()
:property:
Implementing Operator.
.. py:method:: roots()
:property:
Implementing DAGNode.
.. py:method:: leaves()
:property:
Implementing DAGNode.
.. py:method:: owner()
:property:
.. py:method:: email()
:property:
.. py:method:: trigger_rule()
:property:
.. py:method:: depends_on_past()
:property:
.. py:method:: wait_for_downstream()
:property:
.. py:method:: retries()
:property:
.. py:method:: queue()
:property:
.. py:method:: pool()
:property:
.. py:method:: pool_slots()
:property:
.. py:method:: execution_timeout()
:property:
.. py:method:: retry_delay()
:property:
.. py:method:: retry_exponential_backoff()
:property:
.. py:method:: priority_weight()
:property:
.. py:method:: weight_rule()
:property:
.. py:method:: sla()
:property:
.. py:method:: max_active_tis_per_dag()
:property:
.. py:method:: resources()
:property:
.. py:method:: on_execute_callback()
:property:
.. py:method:: on_failure_callback()
:property:
.. py:method:: on_retry_callback()
:property:
.. py:method:: on_success_callback()
:property:
.. py:method:: run_as_user()
:property:
.. py:method:: executor_config()
:property:
.. py:method:: inlets()
:property:
.. py:method:: outlets()
:property:
.. py:method:: get_dag()
Implementing Operator.
.. py:method:: serialize_for_task_group()
Implementing DAGNode.
.. py:method:: unmap(unmap_kwargs = None)
Get the "normal" Operator after applying the current mapping.
If ``operator_class`` is not a class (i.e. this DAG has been deserialized) then this will return a
SerializedBaseOperator that aims to "look like" the real operator.
:param unmap_kwargs: Override the args to pass to the Operator constructor. Only used when
``operator_class`` is still an actual class.
:meta private:
.. py:method:: expand_mapped_task(run_id, *, session)
Create the mapped task instances for mapped task.
:return: The newly created mapped TaskInstances (if any) in ascending order by map index, and the
maximum map_index.
.. py:method:: prepare_for_execution()
.. py:method:: render_template_fields(context, jinja_env = None)
Template all attributes listed in template_fields.
Different from the BaseOperator implementation, this renders the
template fields on the *unmapped* BaseOperator.
:param context: Dict with values to apply on content
:param jinja_env: Jinja environment
:return: The unmapped, populated BaseOperator
.. py:method:: iter_mapped_dependencies()
Upstream dependencies that provide XComs used by this task for task mapping.
.. py:method:: parse_time_mapped_ti_count()
Number of mapped TaskInstances that can be created at DagRun create time.
:return: None if non-literal mapped arg encountered, or else total number of mapped TIs this task
should have
.. py:method:: run_time_mapped_ti_count(run_id, *, session)
Number of mapped TaskInstances that can be created at run time, or None if upstream tasks are not
complete yet.
:return: None if upstream tasks are not complete yet, or else total number of mapped TIs this task
should have