| :mod:`airflow.executors.local_executor` |
| ======================================= |
| |
| .. py:module:: airflow.executors.local_executor |
| |
| .. autoapi-nested-parse:: |
| |
| LocalExecutor |
| |
| .. seealso:: |
| For more information on how the LocalExecutor works, take a look at the guide: |
| :ref:`executor:LocalExecutor` |
| |
| |
| |
| Module Contents |
| --------------- |
| |
| .. py:class:: LocalWorker(result_queue) |
| |
| Bases: :class:`multiprocessing.Process`, :class:`airflow.utils.log.logging_mixin.LoggingMixin` |
| |
| LocalWorker Process implementation to run airflow commands. Executes the given |
| command and puts the result into a result queue when done, terminating execution. |
| |
| |
| .. method:: execute_work(self, key, command) |
| |
| Executes command received and stores result state in queue. |
| |
| :param key: the key to identify the TI |
| :type key: tuple(dag_id, task_id, execution_date) |
| :param command: the command to execute |
| :type command: str |
| |
| |
| |
| |
| .. method:: run(self) |
| |
| |
| |
| |
| .. py:class:: QueuedLocalWorker(task_queue, result_queue) |
| |
| Bases: :class:`airflow.executors.local_executor.LocalWorker` |
| |
| LocalWorker implementation that is waiting for tasks from a queue and will |
| continue executing commands as they become available in the queue. It will terminate |
| execution once the poison token is found. |
| |
| |
| .. method:: run(self) |
| |
| |
| |
| |
| .. py:class:: LocalExecutor |
| |
| Bases: :class:`airflow.executors.base_executor.BaseExecutor` |
| |
| LocalExecutor executes tasks locally in parallel. It uses the |
| multiprocessing Python library and queues to parallelize the execution |
| of tasks. |
| |
| .. py:class:: _UnlimitedParallelism(executor) |
| |
| Bases: :class:`object` |
| |
| Implements LocalExecutor with unlimited parallelism, starting one process |
| per each command to execute. |
| |
| |
| .. method:: start(self) |
| |
| |
| |
| |
| .. method:: execute_async(self, key, command) |
| |
| :param key: the key to identify the TI |
| :type key: tuple(dag_id, task_id, execution_date) |
| :param command: the command to execute |
| :type command: str |
| |
| |
| |
| |
| .. method:: sync(self) |
| |
| |
| |
| |
| .. method:: end(self) |
| |
| |
| |
| |
| .. py:class:: _LimitedParallelism(executor) |
| |
| Bases: :class:`object` |
| |
| Implements LocalExecutor with limited parallelism using a task queue to |
| coordinate work distribution. |
| |
| |
| .. method:: start(self) |
| |
| |
| |
| |
| .. method:: execute_async(self, key, command) |
| |
| :param key: the key to identify the TI |
| :type key: tuple(dag_id, task_id, execution_date) |
| :param command: the command to execute |
| :type command: str |
| |
| |
| |
| |
| .. method:: sync(self) |
| |
| |
| |
| |
| .. method:: end(self) |
| |
| |
| |
| |
| |
| .. method:: start(self) |
| |
| |
| |
| |
| .. method:: execute_async(self, key, command, queue=None, executor_config=None) |
| |
| |
| |
| |
| .. method:: sync(self) |
| |
| |
| |
| |
| .. method:: end(self) |
| |
| |
| |
| |
