| :py:mod:`airflow.operators.bash` |
| ================================ |
| |
| .. py:module:: airflow.operators.bash |
| |
| |
| Module Contents |
| --------------- |
| |
| Classes |
| ~~~~~~~ |
| |
| .. autoapisummary:: |
| |
| airflow.operators.bash.BashOperator |
| |
| |
| |
| |
| .. py:class:: BashOperator(*, bash_command: str, env: Optional[Dict[str, str]] = None, output_encoding: str = 'utf-8', skip_exit_code: int = 99, cwd: str = None, **kwargs) |
| |
| Bases: :py:obj:`airflow.models.BaseOperator` |
| |
| Execute a Bash script, command or set of commands. |
| |
| .. seealso:: |
| For more information on how to use this operator, take a look at the guide: |
| :ref:`howto/operator:BashOperator` |
| |
| If BaseOperator.do_xcom_push is True, the last line written to stdout |
| will also be pushed to an XCom when the bash command completes |
| |
| :param bash_command: The command, set of commands or reference to a |
| bash script (must be '.sh') to be executed. (templated) |
| :type bash_command: str |
| :param env: If env is not None, it must be a dict that defines the |
| environment variables for the new process; these are used instead |
| of inheriting the current process environment, which is the default |
| behavior. (templated) |
| :type env: dict |
| :param output_encoding: Output encoding of bash command |
| :type output_encoding: str |
| :param skip_exit_code: If task exits with this exit code, leave the task |
| in ``skipped`` state (default: 99). If set to ``None``, any non-zero |
| exit code will be treated as a failure. |
| :type skip_exit_code: int |
| :param cwd: Working directory to execute the command in. |
| If None (default), the command is run in a temporary directory. |
| :type cwd: str |
| |
| Airflow will evaluate the exit code of the bash command. In general, a non-zero exit code will result in |
| task failure and zero will result in task success. Exit code ``99`` (or another set in ``skip_exit_code``) |
| will throw an :class:`airflow.exceptions.AirflowSkipException`, which will leave the task in ``skipped`` |
| state. You can have all non-zero exit codes be treated as a failure by setting ``skip_exit_code=None``. |
| |
| .. list-table:: |
| :widths: 25 25 |
| :header-rows: 1 |
| |
| * - Exit code |
| - Behavior |
| * - 0 |
| - success |
| * - `skip_exit_code` (default: 99) |
| - raise :class:`airflow.exceptions.AirflowSkipException` |
| * - otherwise |
| - raise :class:`airflow.exceptions.AirflowException` |
| |
| .. note:: |
| |
| Airflow will not recognize a non-zero exit code unless the whole shell exit with a non-zero exit |
| code. This can be an issue if the non-zero exit arises from a sub-command. The easiest way of |
| addressing this is to prefix the command with ``set -e;`` |
| |
| Example: |
| .. code-block:: python |
| |
| bash_command = "set -e; python3 script.py '{{ next_execution_date }}'" |
| |
| .. note:: |
| |
| Add a space after the script name when directly calling a ``.sh`` script with the |
| ``bash_command`` argument -- for example ``bash_command="my_script.sh "``. This |
| is because Airflow tries to apply load this file and process it as a Jinja template to |
| it ends with ``.sh``, which will likely not be what most users want. |
| |
| .. warning:: |
| |
| Care should be taken with "user" input or when using Jinja templates in the |
| ``bash_command``, as this bash operator does not perform any escaping or |
| sanitization of the command. |
| |
| This applies mostly to using "dag_run" conf, as that can be submitted via |
| users in the Web UI. Most of the default template variables are not at |
| risk. |
| |
| For example, do **not** do this: |
| |
| .. code-block:: python |
| |
| bash_task = BashOperator( |
| task_id="bash_task", |
| bash_command='echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run else "" }}\'"', |
| ) |
| |
| Instead, you should pass this via the ``env`` kwarg and use double-quotes |
| inside the bash_command, as below: |
| |
| .. code-block:: python |
| |
| bash_task = BashOperator( |
| task_id="bash_task", |
| bash_command="echo \"here is the message: '$message'\"", |
| env={"message": '{{ dag_run.conf["message"] if dag_run else "" }}'}, |
| ) |
| |
| |
| .. py:attribute:: template_fields |
| :annotation: = ['bash_command', 'env'] |
| |
| |
| |
| .. py:attribute:: template_fields_renderers |
| |
| |
| |
| |
| .. py:attribute:: template_ext |
| :annotation: = ['.sh', '.bash'] |
| |
| |
| |
| .. py:attribute:: ui_color |
| :annotation: = #f0ede4 |
| |
| |
| |
| .. py:method:: subprocess_hook(self) |
| |
| Returns hook for running the bash command |
| |
| |
| .. py:method:: get_env(self, context) |
| |
| Builds the set of environment variables to be exposed for the bash command |
| |
| |
| .. py:method:: execute(self, context) |
| |
| This is the main method to derive when creating an operator. |
| Context is the same dictionary used as when rendering jinja templates. |
| |
| Refer to get_template_context for more context. |
| |
| |
| .. py:method:: on_kill(self) -> None |
| |
| Override this method to cleanup subprocesses when a task instance |
| gets killed. Any use of the threading, subprocess or multiprocessing |
| module within an operator needs to be cleaned up or it will leave |
| ghost processes behind. |
| |
| |
| |