| :py:mod:`airflow.providers.cncf.kubernetes.operators.kubernetes_pod` |
| ==================================================================== |
| |
| .. py:module:: airflow.providers.cncf.kubernetes.operators.kubernetes_pod |
| |
| .. autoapi-nested-parse:: |
| |
| Executes task in a Kubernetes POD |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.providers.cncf.kubernetes.operators.kubernetes_pod.KubernetesPodOperator |
| |
| |
| |
| |
| .. py:exception:: PodReattachFailure |
| |
| Bases: :py:obj:`airflow.exceptions.AirflowException` |
| |
| When we expect to be able to find a pod but cannot. |
| |
| |
| .. py:class:: KubernetesPodOperator(*, kubernetes_conn_id = None, namespace = None, image = None, name = None, random_name_suffix = True, cmds = None, arguments = None, ports = None, volume_mounts = None, volumes = None, env_vars = None, env_from = None, secrets = None, in_cluster = None, cluster_context = None, labels = None, reattach_on_restart = True, startup_timeout_seconds = 120, get_logs = True, image_pull_policy = None, annotations = None, container_resources = None, affinity = None, config_file = None, node_selectors = None, node_selector = None, image_pull_secrets = None, service_account_name = None, is_delete_operator_pod = True, hostnetwork = False, tolerations = None, security_context = None, container_security_context = None, dnspolicy = None, schedulername = None, full_pod_spec = None, init_containers = None, log_events_on_failure = False, do_xcom_push = False, pod_template_file = None, priority_class_name = None, pod_runtime_info_envs = None, termination_grace_period = None, configmaps = None, resources = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Execute a task in a Kubernetes Pod |
| |
| .. seealso:: |
| For more information on how to use this operator, take a look at the guide: |
| :ref:`howto/operator:KubernetesPodOperator` |
| |
| .. note:: |
| If you use `Google Kubernetes Engine <https://cloud.google.com/kubernetes-engine/>`__ |
| and Airflow is not running in the same cluster, consider using |
| :class:`~airflow.providers.google.cloud.operators.kubernetes_engine.GKEStartPodOperator`, which |
| simplifies the authorization process. |
| |
| :param kubernetes_conn_id: The :ref:`kubernetes connection id <howto/connection:kubernetes>` |
| for the Kubernetes cluster. |
| :param namespace: the namespace to run within kubernetes. |
| :param image: Docker image you wish to launch. Defaults to hub.docker.com, |
| but fully qualified URLS will point to custom repositories. (templated) |
| :param name: name of the pod in which the task will run, will be used (plus a random |
| suffix if random_name_suffix is True) to generate a pod id (DNS-1123 subdomain, |
| containing only [a-z0-9.-]). |
| :param random_name_suffix: if True, will generate a random suffix. |
| :param cmds: entrypoint of the container. (templated) |
| The docker images's entrypoint is used if this is not provided. |
| :param arguments: arguments of the entrypoint. (templated) |
| The docker image's CMD is used if this is not provided. |
| :param ports: ports for the launched pod. |
| :param volume_mounts: volumeMounts for the launched pod. |
| :param volumes: volumes for the launched pod. Includes ConfigMaps and PersistentVolumes. |
| :param env_vars: Environment variables initialized in the container. (templated) |
| :param env_from: (Optional) List of sources to populate environment variables in the container. |
| :param secrets: Kubernetes secrets to inject in the container. |
| They can be exposed as environment vars or files in a volume. |
| :param in_cluster: run kubernetes client with in_cluster configuration. |
| :param cluster_context: context that points to kubernetes cluster. |
| Ignored when in_cluster is True. If None, current-context is used. |
| :param reattach_on_restart: if the worker dies while the pod is running, reattach and monitor |
| during the next try. If False, always create a new pod for each try. |
| :param labels: labels to apply to the Pod. (templated) |
| :param startup_timeout_seconds: timeout in seconds to startup the pod. |
| :param get_logs: get the stdout of the container as logs of the tasks. |
| :param image_pull_policy: Specify a policy to cache or always pull an image. |
| :param annotations: non-identifying metadata you can attach to the Pod. |
| Can be a large range of data, and can include characters |
| that are not permitted by labels. |
| :param container_resources: resources for the launched pod. |
| :param affinity: affinity scheduling rules for the launched pod. |
| :param config_file: The path to the Kubernetes config file. (templated) |
| If not specified, default value is ``~/.kube/config`` |
| :param node_selectors: (Deprecated) A dict containing a group of scheduling rules. |
| Please use node_selector instead. |
| :param node_selector: A dict containing a group of scheduling rules. |
| :param image_pull_secrets: Any image pull secrets to be given to the pod. |
| If more than one secret is required, provide a |
| comma separated list: secret_a,secret_b |
| :param service_account_name: Name of the service account |
| :param is_delete_operator_pod: What to do when the pod reaches its final |
| state, or the execution is interrupted. If True (default), delete the |
| pod; if False, leave the pod. |
| :param hostnetwork: If True enable host networking on the pod. |
| :param tolerations: A list of kubernetes tolerations. |
| :param security_context: security options the pod should run with (PodSecurityContext). |
| :param container_security_context: security options the container should run with. |
| :param dnspolicy: dnspolicy for the pod. |
| :param schedulername: Specify a schedulername for the pod |
| :param full_pod_spec: The complete podSpec |
| :param init_containers: init container for the launched Pod |
| :param log_events_on_failure: Log the pod's events if a failure occurs |
| :param do_xcom_push: If True, the content of the file |
| /airflow/xcom/return.json in the container will also be pushed to an |
| XCom when the container completes. |
| :param pod_template_file: path to pod template file (templated) |
| :param priority_class_name: priority class name for the launched Pod |
| :param pod_runtime_info_envs: (Optional) A list of environment variables, |
| to be set in the container. |
| :param termination_grace_period: Termination grace period if task killed in UI, |
| defaults to kubernetes default |
| :param configmaps: (Optional) A list of names of config maps from which it collects ConfigMaps |
| to populate the environment variables with. The contents of the target |
| ConfigMap's Data field will represent the key-value pairs as environment variables. |
| Extends env_from. |
| |
| .. py:attribute:: BASE_CONTAINER_NAME |
| :annotation: = base |
| |
| |
| |
| .. py:attribute:: POD_CHECKED_KEY |
| :annotation: = already_checked |
| |
| |
| |
| .. py:attribute:: template_fields |
| :annotation: :Sequence[str] = ['image', 'cmds', 'arguments', 'env_vars', 'labels', 'config_file', 'pod_template_file', 'namespace'] |
| |
| |
| |
| .. py:attribute:: template_fields_renderers |
| |
| |
| |
| |
| .. py:method:: pod_manager() |
| |
| |
| .. py:method:: get_hook() |
| |
| |
| .. py:method:: hook() |
| |
| |
| .. py:method:: client() |
| |
| |
| .. py:method:: find_pod(namespace, context, *, exclude_checked = True) |
| |
| Returns an already-running pod for this task instance if one exists. |
| |
| |
| .. py:method:: get_or_create_pod(pod_request_obj, context) |
| |
| |
| .. py:method:: await_pod_start(pod) |
| |
| |
| .. py:method:: extract_xcom(pod) |
| |
| Retrieves xcom value and kills xcom sidecar container |
| |
| |
| .. py:method:: execute(context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| .. py:method:: cleanup(pod, remote_pod) |
| |
| |
| .. py:method:: process_pod_deletion(pod) |
| |
| |
| .. py:method:: patch_already_checked(pod) |
| |
| Add an "already checked" annotation to ensure we don't reattach on retries |
| |
| |
| .. py:method:: on_kill() |
| |
| Override this method to cleanup subprocesses when a task instance |
| gets killed. Any use of the threading, subprocess or multiprocessing |
| module within an operator needs to be cleaned up or it will leave |
| ghost processes behind. |
| |
| |
| .. py:method:: build_pod_request_obj(context = None) |
| |
| Returns V1Pod object based on pod template file, full pod spec, and other operator parameters. |
| |
| The V1Pod attributes are derived (in order of precedence) from operator params, full pod spec, pod |
| template file. |
| |
| |
| .. py:method:: dry_run() |
| |
| Prints out the pod definition that would be created by this operator. |
| Does not include labels specific to the task instance (since there isn't |
| one in a dry_run) and excludes all empty elements. |
| |
| |
| |
