| :py:mod:`airflow.providers.amazon.aws.sensors.sagemaker` |
| ======================================================== |
| |
| .. py:module:: airflow.providers.amazon.aws.sensors.sagemaker |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.amazon.aws.sensors.sagemaker.SageMakerBaseSensor |
| airflow.providers.amazon.aws.sensors.sagemaker.SageMakerEndpointSensor |
| airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTransformSensor |
| airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTuningSensor |
| airflow.providers.amazon.aws.sensors.sagemaker.SageMakerTrainingSensor |
| |
| |
| |
| |
| .. py:class:: SageMakerBaseSensor(*, aws_conn_id = 'aws_default', **kwargs) |
| |
| Bases: :py:obj:`airflow.sensors.base.BaseSensorOperator` |
| |
| Contains general sensor behavior for SageMaker. |
| |
| Subclasses should implement get_sagemaker_response() and state_from_response() methods. |
| Subclasses should also implement NON_TERMINAL_STATES and FAILED_STATE methods. |
| |
| .. py:attribute:: ui_color |
| :annotation: = #ededed |
| |
| |
| |
| .. py:method:: get_hook(self) |
| |
| Get SageMakerHook. |
| |
| |
| .. py:method:: poke(self, context) |
| |
| Function that the sensors defined while deriving this class should |
| override. |
| |
| |
| .. py:method:: non_terminal_states(self) |
| :abstractmethod: |
| |
| Placeholder for returning states with should not terminate. |
| |
| |
| .. py:method:: failed_states(self) |
| :abstractmethod: |
| |
| Placeholder for returning states with are considered failed. |
| |
| |
| .. py:method:: get_sagemaker_response(self) |
| :abstractmethod: |
| |
| Placeholder for checking status of a SageMaker task. |
| |
| |
| .. py:method:: get_failed_reason_from_response(self, response) |
| |
| Placeholder for extracting the reason for failure from an AWS response. |
| |
| |
| .. py:method:: state_from_response(self, response) |
| :abstractmethod: |
| |
| Placeholder for extracting the state from an AWS response. |
| |
| |
| |
| .. py:class:: SageMakerEndpointSensor(*, endpoint_name, **kwargs) |
| |
| Bases: :py:obj:`SageMakerBaseSensor` |
| |
| Polls the endpoint state until it reaches a terminal state. Raises an |
| AirflowException with the failure reason if a failed state is reached. |
| |
| .. seealso:: |
| For more information on how to use this sensor, take a look at the guide: |
| :ref:`howto/sensor:SageMakerEndpointSensor` |
| |
| :param endpoint_name: Name of the endpoint instance to watch. |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['endpoint_name'] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] = [] |
| |
| |
| |
| .. py:method:: non_terminal_states(self) |
| |
| Placeholder for returning states with should not terminate. |
| |
| |
| .. py:method:: failed_states(self) |
| |
| Placeholder for returning states with are considered failed. |
| |
| |
| .. py:method:: get_sagemaker_response(self) |
| |
| Placeholder for checking status of a SageMaker task. |
| |
| |
| .. py:method:: get_failed_reason_from_response(self, response) |
| |
| Placeholder for extracting the reason for failure from an AWS response. |
| |
| |
| .. py:method:: state_from_response(self, response) |
| |
| Placeholder for extracting the state from an AWS response. |
| |
| |
| |
| .. py:class:: SageMakerTransformSensor(*, job_name, **kwargs) |
| |
| Bases: :py:obj:`SageMakerBaseSensor` |
| |
| Polls the transform job until it reaches a terminal state. Raises an |
| AirflowException with the failure reason if a failed state is reached. |
| |
| .. seealso:: |
| For more information on how to use this sensor, take a look at the guide: |
| :ref:`howto/sensor:SageMakerTransformSensor` |
| |
| :param job_name: Name of the transform job to watch. |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['job_name'] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] = [] |
| |
| |
| |
| .. py:method:: non_terminal_states(self) |
| |
| Placeholder for returning states with should not terminate. |
| |
| |
| .. py:method:: failed_states(self) |
| |
| Placeholder for returning states with are considered failed. |
| |
| |
| .. py:method:: get_sagemaker_response(self) |
| |
| Placeholder for checking status of a SageMaker task. |
| |
| |
| .. py:method:: get_failed_reason_from_response(self, response) |
| |
| Placeholder for extracting the reason for failure from an AWS response. |
| |
| |
| .. py:method:: state_from_response(self, response) |
| |
| Placeholder for extracting the state from an AWS response. |
| |
| |
| |
| .. py:class:: SageMakerTuningSensor(*, job_name, **kwargs) |
| |
| Bases: :py:obj:`SageMakerBaseSensor` |
| |
| Asks for the state of the tuning state until it reaches a terminal state. |
| Raises an AirflowException with the failure reason if a failed state is reached. |
| |
| .. seealso:: |
| For more information on how to use this sensor, take a look at the guide: |
| :ref:`howto/sensor:SageMakerTuningSensor` |
| |
| :param job_name: Name of the tuning instance to watch. |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['job_name'] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] = [] |
| |
| |
| |
| .. py:method:: non_terminal_states(self) |
| |
| Placeholder for returning states with should not terminate. |
| |
| |
| .. py:method:: failed_states(self) |
| |
| Placeholder for returning states with are considered failed. |
| |
| |
| .. py:method:: get_sagemaker_response(self) |
| |
| Placeholder for checking status of a SageMaker task. |
| |
| |
| .. py:method:: get_failed_reason_from_response(self, response) |
| |
| Placeholder for extracting the reason for failure from an AWS response. |
| |
| |
| .. py:method:: state_from_response(self, response) |
| |
| Placeholder for extracting the state from an AWS response. |
| |
| |
| |
| .. py:class:: SageMakerTrainingSensor(*, job_name, print_log=True, **kwargs) |
| |
| Bases: :py:obj:`SageMakerBaseSensor` |
| |
| Polls the training job until it reaches a terminal state. Raises an |
| AirflowException with the failure reason if a failed state is reached. |
| |
| .. seealso:: |
| For more information on how to use this sensor, take a look at the guide: |
| :ref:`howto/sensor:SageMakerTrainingSensor` |
| |
| :param job_name: Name of the training job to watch. |
| :param print_log: Prints the cloudwatch log if True; Defaults to True. |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['job_name'] |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: :Sequence[str] = [] |
| |
| |
| |
| .. py:method:: init_log_resource(self, hook) |
| |
| Set tailing LogState for associated training job. |
| |
| |
| .. py:method:: non_terminal_states(self) |
| |
| Placeholder for returning states with should not terminate. |
| |
| |
| .. py:method:: failed_states(self) |
| |
| Placeholder for returning states with are considered failed. |
| |
| |
| .. py:method:: get_sagemaker_response(self) |
| |
| Placeholder for checking status of a SageMaker task. |
| |
| |
| .. py:method:: get_failed_reason_from_response(self, response) |
| |
| Placeholder for extracting the reason for failure from an AWS response. |
| |
| |
| .. py:method:: state_from_response(self, response) |
| |
| Placeholder for extracting the state from an AWS response. |
| |
| |
| |