blob: 8eedf1a962892c508577795b11f55d7ca9110547 [file] [log] [blame]
:mod:`airflow.providers.amazon.aws.sensors.emr_step`
====================================================
.. py:module:: airflow.providers.amazon.aws.sensors.emr_step
Module Contents
---------------
.. py:class:: EmrStepSensor(*, job_flow_id: str, step_id: str, target_states: Optional[Iterable[str]] = None, failed_states: Optional[Iterable[str]] = None, **kwargs)
Bases: :class:`airflow.providers.amazon.aws.sensors.emr_base.EmrBaseSensor`
Asks for the state of the step until it reaches any of the target states.
If it fails the sensor errors, failing the task.
With the default target states, sensor waits step to be completed.
:param job_flow_id: job_flow_id which contains the step check the state of
:type job_flow_id: str
:param step_id: step to check the state of
:type step_id: str
:param target_states: the target states, sensor waits until
step reaches any of these states
:type target_states: list[str]
:param failed_states: the failure states, sensor fails when
step reaches any of these states
:type failed_states: list[str]
.. attribute:: template_fields
:annotation: = ['job_flow_id', 'step_id', 'target_states', 'failed_states']
.. attribute:: template_ext
:annotation: = []
.. method:: get_emr_response(self)
Make an API call with boto3 and get details about the cluster step.
.. seealso::
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr.html#EMR.Client.describe_step
:return: response
:rtype: dict[str, Any]
.. staticmethod:: state_from_response(response: Dict[str, Any])
Get state from response dictionary.
:param response: response from AWS API
:type response: dict[str, Any]
:return: execution state of the cluster step
:rtype: str
.. staticmethod:: failure_message_from_response(response: Dict[str, Any])
Get failure message from response dictionary.
:param response: response from AWS API
:type response: dict[str, Any]
:return: failure message
:rtype: Optional[str]