blob: 5a8eee0466ed020d97931e6a5063af26526fd575 [file] [log] [blame]
:mod:`airflow.providers.microsoft.azure.operators.azure_container_instances`
============================================================================
.. py:module:: airflow.providers.microsoft.azure.operators.azure_container_instances
Module Contents
---------------
.. data:: Volume
.. data:: DEFAULT_ENVIRONMENT_VARIABLES
:annotation: :Dict[str, str]
.. data:: DEFAULT_SECURED_VARIABLES
:annotation: :Sequence[str] = []
.. data:: DEFAULT_VOLUMES
:annotation: :Sequence[Volume] = []
.. data:: DEFAULT_MEMORY_IN_GB
:annotation: = 2.0
.. data:: DEFAULT_CPU
:annotation: = 1.0
.. py:class:: AzureContainerInstancesOperator(*, ci_conn_id: str, registry_conn_id: Optional[str], resource_group: str, name: str, image: str, region: str, environment_variables: Optional[dict] = None, secured_variables: Optional[str] = None, volumes: Optional[list] = None, memory_in_gb: Optional[Any] = None, cpu: Optional[Any] = None, gpu: Optional[Any] = None, command: Optional[List[str]] = None, remove_on_error: bool = True, fail_if_exists: bool = True, tags: Optional[Dict[str, str]] = None, os_type: str = 'Linux', restart_policy: str = 'Never', ip_address: Optional[IpAddress] = None, ports: Optional[List[ContainerPort]] = None, **kwargs)
Bases: :class:`airflow.models.BaseOperator`
Start a container on Azure Container Instances
:param ci_conn_id: connection id of a service principal which will be used
to start the container instance
:type ci_conn_id: str
:param registry_conn_id: connection id of a user which can login to a
private docker registry. If None, we assume a public registry
:type registry_conn_id: Optional[str]
:param resource_group: name of the resource group wherein this container
instance should be started
:type resource_group: str
:param name: name of this container instance. Please note this name has
to be unique in order to run containers in parallel.
:type name: str
:param image: the docker image to be used
:type image: str
:param region: the region wherein this container instance should be started
:type region: str
:param environment_variables: key,value pairs containing environment
variables which will be passed to the running container
:type environment_variables: Optional[dict]
:param secured_variables: names of environmental variables that should not
be exposed outside the container (typically passwords).
:type secured_variables: Optional[str]
:param volumes: list of ``Volume`` tuples to be mounted to the container.
Currently only Azure Fileshares are supported.
:type volumes: list[<conn_id, account_name, share_name, mount_path, read_only>]
:param memory_in_gb: the amount of memory to allocate to this container
:type memory_in_gb: double
:param cpu: the number of cpus to allocate to this container
:type cpu: double
:param gpu: GPU Resource for the container.
:type gpu: azure.mgmt.containerinstance.models.GpuResource
:param command: the command to run inside the container
:type command: Optional[List[str]]
:param container_timeout: max time allowed for the execution of
the container instance.
:type container_timeout: datetime.timedelta
:param tags: azure tags as dict of str:str
:type tags: Optional[dict[str, str]]
:param os_type: The operating system type required by the containers
in the container group. Possible values include: 'Windows', 'Linux'
:type os_type: str
:param restart_policy: Restart policy for all containers within the container group.
Possible values include: 'Always', 'OnFailure', 'Never'
:type restart_policy: str
:param ip_address: The IP address type of the container group.
:type ip_address: IpAddress
**Example**::
AzureContainerInstancesOperator(
ci_conn_id = "azure_service_principal",
registry_conn_id = "azure_registry_user",
resource_group = "my-resource-group",
name = "my-container-name-{{ ds }}",
image = "myprivateregistry.azurecr.io/my_container:latest",
region = "westeurope",
environment_variables = {"MODEL_PATH": "my_value",
"POSTGRES_LOGIN": "{{ macros.connection('postgres_default').login }}",
"POSTGRES_PASSWORD": "{{ macros.connection('postgres_default').password }}",
"JOB_GUID": "{{ ti.xcom_pull(task_ids='task1', key='guid') }}" },
secured_variables = ['POSTGRES_PASSWORD'],
volumes = [("azure_wasb_conn_id",
"my_storage_container",
"my_fileshare",
"/input-data",
True),],
memory_in_gb=14.0,
cpu=4.0,
gpu=GpuResource(count=1, sku='K80'),
command=["/bin/echo", "world"],
task_id="start_container"
)
.. attribute:: template_fields
:annotation: = ['name', 'image', 'command', 'environment_variables']
.. method:: execute(self, context: dict)
.. method:: on_kill(self)
.. method:: _monitor_logging(self, resource_group: str, name: str)
.. method:: _log_last(self, logs: Optional[list], last_line_logged: Any)
.. staticmethod:: _check_name(name: str)