blob: fd4a1670907d580be877e6b22c114cda201b780c [file] [log] [blame]
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
.. _concepts:params:
Params
======
Params are how Airflow provides runtime configuration to tasks.
When you trigger a DAG manually, you can modify its Params before the dagrun starts.
If the user-supplied values don't pass validation, Airflow shows a warning instead of creating the dagrun.
(For scheduled runs, the default values are used.)
Adding Params to a DAG
----------------------
To add Params to a :class:`~airflow.models.dag.DAG`, initialize it with the ``params`` kwarg.
Use a dictionary that maps Param names to a either a :class:`~airflow.models.param.Param` or an object indicating the parameter's default value.
.. code-block::
from airflow import DAG
from airflow.models.param import Param
with DAG(
"the_dag",
params={
"x": Param(5, type="integer", minimum=3),
"y": 6
},
) as the_dag:
Referencing Params in a Task
----------------------------
Params are stored as ``params`` in the :ref:`template context <templates-ref>`.
So you can reference them in a template.
.. code-block::
PythonOperator(
task_id="from_template",
op_args=[
"{{ params.int_param + 10 }}",
],
python_callable=(
lambda x: print(x)
),
)
Even though Params can use a variety of types, the default behavior of templates is to provide your task with a string.
You can change this by setting ``render_template_as_native_obj=True`` while initializing the :class:`~airflow.models.dag.DAG`.
.. code-block::
with DAG(
"the_dag",
params={"x": Param(5, type="integer", minimum=3)},
render_template_as_native_obj=True
) as the_dag:
This way, the Param's type is respected when its provided to your task.
.. code-block::
# prints <class 'str'> by default
# prints <class 'int'> if render_template_as_native_obj=True
PythonOperator(
task_id="template_type",
op_args=[
"{{ params.int_param }}",
],
python_callable=(
lambda x: print(type(x))
),
)
Another way to access your param is via a task's ``context`` kwarg.
.. code-block::
def print_x(**context):
print(context["params"]["x"])
PythonOperator(
task_id="print_x",
python_callable=print_it,
)
Task-level Params
-----------------
You can also add Params to individual tasks.
.. code-block::
PythonOperator(
task_id="print_x",
params={"x": 10},
python_callable=print_it,
)
If there's already a dag param with that name, the task-level default will take precedence over the dag-level default.
If a user supplies their own value when the DAG was triggered, Airflow ignores all defaults and uses the user's value.
JSON Schema Validation
----------------------
:class:`~airflow.modules.param.Param` makes use of ``json-schema <https://json-schema.org/>``, so you can use the full json-schema specifications mentioned at https://json-schema.org/draft/2020-12/json-schema-validation.html to define ``Param`` objects.
.. code-block::
with DAG(
"my_dag",
params={
# a int with a default value
"int_param": Param(10, type="integer", minimum=0, maximum=20),
# a required param which can be of multiple types
"dummy": Param(type=["null", "number", "string"]),
# an enum param, must be one of three values
"enum_param": Param("foo", enum=["foo", "bar", 42]),
# a param which uses json-schema formatting
"email": Param(
default="example@example.com",
type="string",
format="idn-email",
minLength=5,
maxLength=255,
),
},
) as my_dag:
.. note::
As of now, for security reasons, one can not use Param objects derived out of custom classes. We are
planning to have a registration system for custom Param classes, just like we've for Operator ExtraLinks.
Disabling Runtime Param Modification
------------------------------------
The ability to update params while triggering a DAG depends on the flag ``core.dag_run_conf_overrides_params``.
Setting this config to ``False`` will effectively turn your default params into constants.