| :py:mod:`airflow.models.mappedoperator` |
| ======================================= |
| |
| .. py:module:: airflow.models.mappedoperator |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.models.mappedoperator.OperatorPartial |
| airflow.models.mappedoperator.MappedOperator |
| |
| |
| |
| Functions |
| ~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.models.mappedoperator.get_mappable_types |
| airflow.models.mappedoperator.validate_mapping_kwargs |
| airflow.models.mappedoperator.prevent_duplicates |
| airflow.models.mappedoperator.ensure_xcomarg_return_value |
| |
| |
| |
| Attributes |
| ~~~~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.models.mappedoperator.Mappable |
| airflow.models.mappedoperator.ValidationSource |
| airflow.models.mappedoperator.MAPPABLE_LITERAL_TYPES |
| |
| |
| .. py:data:: Mappable |
| |
| |
| |
| |
| .. py:data:: ValidationSource |
| |
| |
| |
| |
| .. py:data:: MAPPABLE_LITERAL_TYPES |
| |
| |
| |
| |
| .. py:function:: get_mappable_types() |
| |
| |
| .. py:function:: validate_mapping_kwargs(op, func, value) |
| |
| |
| .. py:function:: prevent_duplicates(kwargs1, kwargs2, *, fail_reason) |
| |
| |
| .. py:function:: ensure_xcomarg_return_value(arg) |
| |
| |
| .. py:class:: OperatorPartial |
| |
| An "intermediate state" returned by ``BaseOperator.partial()``. |
| |
| This only exists at DAG-parsing time; the only intended usage is for the |
| user to call ``.expand()`` on it at some point (usually in a method chain) to |
| create a ``MappedOperator`` to add into the DAG. |
| |
| .. py:attribute:: operator_class |
| :annotation: :Type[airflow.models.baseoperator.BaseOperator] |
| |
| |
| |
| .. py:attribute:: kwargs |
| :annotation: :Dict[str, Any] |
| |
| |
| |
| .. py:method:: __attrs_post_init__() |
| |
| |
| .. py:method:: __repr__() |
| |
| Return repr(self). |
| |
| |
| .. py:method:: __del__() |
| |
| |
| .. py:method:: expand(**mapped_kwargs) |
| |
| |
| |
| .. py:class:: MappedOperator(context=None) |
| |
| Bases: :py:obj:`airflow.models.abstractoperator.AbstractOperator` |
| |
| Object representing a mapped operator in a DAG. |
| |
| .. py:attribute:: operator_class |
| :annotation: :Union[Type[airflow.models.baseoperator.BaseOperator], Dict[str, Any]] |
| |
| |
| |
| .. py:attribute:: mapped_kwargs |
| :annotation: :Dict[str, Mappable] |
| |
| |
| |
| .. py:attribute:: partial_kwargs |
| :annotation: :Dict[str, Any] |
| |
| |
| |
| .. py:attribute:: task_id |
| :annotation: :str |
| |
| |
| |
| .. py:attribute:: params |
| :annotation: :Optional[dict] |
| |
| |
| |
| .. py:attribute:: deps |
| :annotation: :FrozenSet[airflow.ti_deps.deps.base_ti_dep.BaseTIDep] |
| |
| |
| |
| .. py:attribute:: operator_extra_links |
| :annotation: :Collection[airflow.models.baseoperator.BaseOperatorLink] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] |
| |
| |
| |
| .. py:attribute:: template_fields |
| :annotation: :Collection[str] |
| |
| |
| |
| .. py:attribute:: template_fields_renderers |
| :annotation: :Dict[str, str] |
| |
| |
| |
| .. py:attribute:: ui_color |
| :annotation: :str |
| |
| |
| |
| .. py:attribute:: ui_fgcolor |
| :annotation: :str |
| |
| |
| |
| .. py:attribute:: dag |
| :annotation: :Optional[airflow.models.dag.DAG] |
| |
| |
| |
| .. py:attribute:: task_group |
| :annotation: :Optional[airflow.utils.task_group.TaskGroup] |
| |
| |
| |
| .. py:attribute:: start_date |
| :annotation: :Optional[pendulum.DateTime] |
| |
| |
| |
| .. py:attribute:: end_date |
| :annotation: :Optional[pendulum.DateTime] |
| |
| |
| |
| .. py:attribute:: upstream_task_ids |
| :annotation: :Set[str] |
| |
| |
| |
| .. py:attribute:: downstream_task_ids |
| :annotation: :Set[str] |
| |
| |
| |
| .. py:attribute:: is_mapped |
| :annotation: :ClassVar[bool] = True |
| |
| |
| |
| .. py:attribute:: subdag |
| :annotation: :None |
| |
| |
| |
| .. py:attribute:: HIDE_ATTRS_FROM_UI |
| :annotation: :ClassVar[FrozenSet[str]] |
| |
| |
| |
| .. py:method:: __hash__() |
| |
| Return hash(self). |
| |
| |
| .. py:method:: __repr__() |
| |
| Return repr(self). |
| |
| |
| .. py:method:: __attrs_post_init__() |
| |
| |
| .. py:method:: get_serialized_fields() |
| :classmethod: |
| |
| |
| .. py:method:: deps_for(operator_class) |
| :staticmethod: |
| |
| |
| .. py:method:: task_type() |
| :property: |
| |
| Implementing Operator. |
| |
| |
| .. py:method:: inherits_from_empty_operator() |
| :property: |
| |
| Implementing Operator. |
| |
| |
| .. py:method:: roots() |
| :property: |
| |
| Implementing DAGNode. |
| |
| |
| .. py:method:: leaves() |
| :property: |
| |
| Implementing DAGNode. |
| |
| |
| .. py:method:: owner() |
| :property: |
| |
| |
| .. py:method:: email() |
| :property: |
| |
| |
| .. py:method:: trigger_rule() |
| :property: |
| |
| |
| .. py:method:: depends_on_past() |
| :property: |
| |
| |
| .. py:method:: wait_for_downstream() |
| :property: |
| |
| |
| .. py:method:: retries() |
| :property: |
| |
| |
| .. py:method:: queue() |
| :property: |
| |
| |
| .. py:method:: pool() |
| :property: |
| |
| |
| .. py:method:: pool_slots() |
| :property: |
| |
| |
| .. py:method:: execution_timeout() |
| :property: |
| |
| |
| .. py:method:: retry_delay() |
| :property: |
| |
| |
| .. py:method:: retry_exponential_backoff() |
| :property: |
| |
| |
| .. py:method:: priority_weight() |
| :property: |
| |
| |
| .. py:method:: weight_rule() |
| :property: |
| |
| |
| .. py:method:: sla() |
| :property: |
| |
| |
| .. py:method:: max_active_tis_per_dag() |
| :property: |
| |
| |
| .. py:method:: resources() |
| :property: |
| |
| |
| .. py:method:: on_execute_callback() |
| :property: |
| |
| |
| .. py:method:: on_failure_callback() |
| :property: |
| |
| |
| .. py:method:: on_retry_callback() |
| :property: |
| |
| |
| .. py:method:: on_success_callback() |
| :property: |
| |
| |
| .. py:method:: run_as_user() |
| :property: |
| |
| |
| .. py:method:: executor_config() |
| :property: |
| |
| |
| .. py:method:: inlets() |
| :property: |
| |
| |
| .. py:method:: outlets() |
| :property: |
| |
| |
| .. py:method:: get_dag() |
| |
| Implementing Operator. |
| |
| |
| .. py:method:: serialize_for_task_group() |
| |
| Implementing DAGNode. |
| |
| |
| .. py:method:: unmap(unmap_kwargs = None) |
| |
| Get the "normal" Operator after applying the current mapping. |
| |
| If ``operator_class`` is not a class (i.e. this DAG has been deserialized) then this will return a |
| SerializedBaseOperator that aims to "look like" the real operator. |
| |
| :param unmap_kwargs: Override the args to pass to the Operator constructor. Only used when |
| ``operator_class`` is still an actual class. |
| |
| :meta private: |
| |
| |
| .. py:method:: expand_mapped_task(run_id, *, session) |
| |
| Create the mapped task instances for mapped task. |
| |
| :return: The newly created mapped TaskInstances (if any) in ascending order by map index, and the |
| maximum map_index. |
| |
| |
| .. py:method:: prepare_for_execution() |
| |
| |
| .. py:method:: render_template_fields(context, jinja_env = None) |
| |
| Template all attributes listed in template_fields. |
| |
| Different from the BaseOperator implementation, this renders the |
| template fields on the *unmapped* BaseOperator. |
| |
| :param context: Dict with values to apply on content |
| :param jinja_env: Jinja environment |
| :return: The unmapped, populated BaseOperator |
| |
| |
| .. py:method:: iter_mapped_dependencies() |
| |
| Upstream dependencies that provide XComs used by this task for task mapping. |
| |
| |
| .. py:method:: parse_time_mapped_ti_count() |
| |
| Number of mapped TaskInstances that can be created at DagRun create time. |
| |
| :return: None if non-literal mapped arg encountered, or else total number of mapped TIs this task |
| should have |
| |
| |
| .. py:method:: run_time_mapped_ti_count(run_id, *, session) |
| |
| Number of mapped TaskInstances that can be created at run time, or None if upstream tasks are not |
| complete yet. |
| |
| :return: None if upstream tasks are not complete yet, or else total number of mapped TIs this task |
| should have |
| |
| |
| |