 .. Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at

 ..   http://www.apache.org/licenses/LICENSE-2.0

 .. Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.

Dockerfile Changelog
====================

The ``Dockerfile`` does not strictly follow the `SemVer <https://semver.org/>`_ approach of
Apache Airflow when it comes to features and backwards compatibility. While Airflow code strictly
follows it, the ``Dockerfile`` is really a way to give users a conveniently packaged Airflow
using standard container approach, so occasionally there are some changes in the building process
or in the entrypoint of the image that require slight adaptation of how it is used or built.

The Changelog below describes the changes introduced in each version of the docker images released by
the Airflow team.

:note: The Changelog below concerns only the convenience production images released at
       `Airflow DockerHub <https://hub.docker.com/r/apache/airflow>`_ . The images that are released
       there, are usually built using the ``Dockerfile`` released together with Airflow. However You are
       free to take latest released ``Dockerfile`` from Airflow and use it to build an image for
       any Airflow version from the ``Airflow 2`` line. There is no guarantee that it works, but if it does,
       then you can use latest features from that image to build the previous Airflow versions.

Changes after publishing the images
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Occasionally our images need to be regenerated using newer ``Dockerfiles`` or constraints.
This happens when an issue is found or a breaking change is released by our dependencies
that invalidates the already released image, and regenerating the image makes it usable again.
While we cannot assure 100% backwards compatibility when it happens, we at least document it
here so that users affected can find the reason for the changes.

+--------------+---------------------+-----------------------------------------+------------------------+----------------------------------------------+
| Date         | Affected images     | Potentially breaking change             | Reason                 | Link to Pull Request                         |
+==============+=====================+=========================================+========================+==============================================+
| 17 June 2022 | 2.2.5               | * The ``Authlib`` library downgraded    | Flask App Builder      | https://github.com/apache/airflow/pull/24516 |
|              |                     |   from 1.0.1 to 0.15.5 version          | not compatible with    |                                              |
|              | 2.3.0-2.3.3         |                                         | Authlib >= 1.0.0       |                                              |
+--------------+---------------------+-----------------------------------------+------------------------+----------------------------------------------+
| 18 Jan 2022  | All 2.2.\*, 2.1.\*  | * The AIRFLOW_GID 500 was removed       | MySQL changed keys     | https://github.com/apache/airflow/pull/20912 |
|              |                     | * MySQL ``apt`` repository key changed. | to sign their packages |                                              |
|              |                     |                                         | on 17 Jan 2022         |                                              |
+--------------+---------------------+-----------------------------------------+------------------------+----------------------------------------------+


Airflow 2.3
~~~~~~~~~~~

* 2.3.0

  * Airflow 2.3 ``Dockerfile`` is now better optimized for caching and "standalone" which means that you
    can copy **just** the ``Dockerfile`` to any folder and start building custom images. This
    however requires `Buildkit <https://docs.docker.com/develop/develop-images/build_enhancements/>`_
    to build the image because we started using features that are only available in ``Buildkit``.
    This can be done by setting ``DOCKER_BUILDKIT=1`` as an environment variable
    or by installing `the buildx plugin <https://docs.docker.com/buildx/working-with-buildx/>`_
    and running ``docker buildx build`` command.
  * Add Python 3.10 support
  * Add support for Bullseye Debian release (Debian Buster is deprecated)
  * Add Multi-Platform support (AMD64/ARM64) in order to accommodate MacOS M1 users
  * Build parameters which control if packages and Airflow should be installed from context file were
    unified
  * The ``INSTALL_FROM_PYPI`` arg was replaced to ``AIRFLOW_IS_IN_CONTEXT`` (with reverse meaning and
    default changed to false)
  * The ``INSTALL_FROM_DOCKER_CONTEXT_FILES`` arg changed to ``INSTALL_PACKAGES_FROM_CONTEXT``

Airflow 2.2
~~~~~~~~~~~

* 2.2.4
  * Add support for both ``.piprc`` and ``pip.conf`` customizations
  * Add ArtifactHub labels for better discovery of the images
  * Update default Python image to be 3.7
  * Build images with ``Buildkit`` (optional)
  * Fix building the image on Azure with ``text file busy`` error

* 2.2.3
  * No changes

* 2.2.2
  * No changes

* 2.2.1
  * Workaround the problem with ``libstdcpp`` TLS error

* 2.2.0
  * Remove AIRFLOW_GID (5000) from Airflow images (potentially breaking change for users using it)
  * Added warnings for Quick-start docker compose
  * Fix warm shutdown for celery worker (signal propagation)
  * Add Oauth libraries to PROD images
  * Add Python 3.9 support

Airflow 2.1
~~~~~~~~~~~

* MySQL changed the keys to sign their packages on 17 Feb 2022. This caused all released images
  to fail when being extended. As result, on 18 Feb 2021 we re-released all
  the ``2.2`` and ``2.1`` images with latest versions of ``Dockerfile``
  containing the new signing key.

  There were subtle changes in the behaviour of some 2.1 images due to that (more details below)
  Detailed `issue here <https://github.com/apache/airflow/issues/20911>`_

:note: that the changes below were valid before image refreshing on 18 Feb 2022.
  Since all the images were refreshed on 18 Feb with the same ``Dockerfile``
  as 2.1.4, the changes 2.1.1 -> 2.1.3 are
  effectively applied to all the images in 2.1.* line.
  The images refreshed have also those fixes added:

* All 2.1.* image versions refreshed on 18 Feb 2022 have those fixes applied:
  * Fix building the image on Azure with ``text file busy`` error
  * Workaround the problem with ``libstdcpp`` TLS error
  * Remove AIRFLOW_GID (5000) from Airflow images (potentially breaking change for users using it)
  * Added warnings for Quick-start docker compose
  * Add Oauth libraries to PROD images

Original image Changelog (before the refresh on 18 Feb 2022):

* 2.1.4
   * Workaround the problem with ``libstdcpp`` TLS error
   * fixed detection of port number in connection URL
   * Improve warnings for quick-start-docker compose
   * Fix warm shutdown for celery worker (signal propagation)

* 2.1.3
   * fixed auto-creation of user to use non-deprecated ``create user`` rather than ``user_create``
   * remove waiting for celery backend for ``worker`` and ``flower`` commands rather than ``scheduler`` and ``celery`` only
   * remove deprecated ``airflow upgradedb`` command from Airflow 1.10 in case upgrade is requested
   * Add Python 3.9 support

* 2.1.2
   * No changes

* 2.1.1
   * Fix failure of lack of default commands (failed when no commands were passed)
   * Added ``_PIP_ADDITIONAL_REQUIREMENTS`` development feature

* 2.1.0
   * Unset default ``PIP_USER`` variable - which caused PythonVirtualEnv to fail


Airflow 2.0
~~~~~~~~~~~

* MySQL changed the keys to sign their packages on 17 Feb 2022. This caused all released images
  to fail when being extended. As result, on 18 Feb 2021 we re-released all
  the ``2.2`` and ``2.1`` images with latest versions of ``Dockerfile``
  containing the new signing key.

  There were no changes in the behaviour of 2.0.2 image due to that
  Detailed `issue here <https://github.com/apache/airflow/issues/20911>`_ .
  Only 2.0.2 image was regenerated, as 2.0.1 and 2.0.0 versions are hardly used and it is unlikely someone
  would like to extend those images. Extending 2.0.1 and 2.0.0 images will lead to failures of "missing key".

* 2.0.2
   * Set correct PYTHONPATH for ``root`` user. Allows to run the image as root
   * Warn if the deprecated 5000 group ID was used for airflow user when running the image
     (should be 0 for the OpenShift compatibility). Fails if the group 5000 was used with any other user
     (it would not work anyway but with cryptic errors)
   * Set umask as 002 by default, so that you can actually change the user id used to run the image
     (required for OpenShift compatibility)
   * Skip checking the DB and celery backend if CONNECTION_CHECK_MAX_COUNT is equal to 0

* 2.0.1
   * Avoid reverse IP lookup when checking DB availability. This helped to solve long delays on misconfigured
     docker engines
   * Add auto-detection of redis and amqp broker ports
   * Fixed detection of all user/password combinations in URLs - helps in auto-detecting ports and testing
     connectivity
   * Add possibility to create Admin user automatically when entering the image
   * Automatically create system user when different user than ``airflow`` is used. Needed for OpenShift
     compatibility
   * Allows to exec to ``bash`` or ``python`` if specified as parameters
   * Remove ``airflow`` command if it is specified as first parameter of the ``run`` command

* 2.0.0
   * Initial release of the image based on Debian Buster