tree: ad849caf6b17297d2a66b1bdf83ab3617519f600 [path history] [tgz]
  1. emr.md
  2. index.rst
  3. kubernetes.md
  4. README.md
docs/liminal/executors/README.md

Executors

Each task in your pipelines has a reference to an executor from the executors section of your liminal.yml file. If not specified, the appropriate default executor for that task type is used.

executors:
  - executor: my_kubernetes_executor
    type: kubernetes
    resources:
      request_memory: 128Mi
pipelines:
  pipeline: my_pipeline
  tasks:
    - task: my_python_task
      type: python
      image: myorg/myrepo:mypythonapp
      executor: my_kubernetes_executor
      cmd: python -u my_module.py

In the example above we define an executor of type kubernetes with custom resources configuration.

executors is a section in the root of your liminal.yml file and is a list of executors defined by the following attributes:

executor attributes

executor: name of your executor.

type: type of the executor. The current available image types are: kubernetes and emr.

Different executor types support their own additional configuration.

task types

  1. kubernetes
  2. emr