contributing-docs/03a_contributors_quick_start_beginners.rst - airflow - Git at Google

  .. Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at

  ..   http://www.apache.org/licenses/LICENSE-2.0

  .. Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
     KIND, either express or implied.  See the License for the
     specific language governing permissions and limitations
     under the License.

 Your First Airflow Pull Request — 15-Minute Guide
 =================================================

 .. contents:: On this page
    :local:
    :depth: 1


 Purpose
 -------
 This page walks **new contributors** through opening their first
 Apache Airflow pull request (PR) in about 15 minutes.  We present *one*
 local option (Breeze) and *one* fully-hosted option (GitHub Codespaces).
 Everything else lives in the advanced guides.

 Prerequisites
 -------------
 * GitHub account
 * `Fork <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo?tool=webui>`_ `apache/airflow <https://github.com/apache/airflow>`__
 * `Basic Git <https://docs.github.com/en/get-started/git-basics/set-up-git>`__ (**only** required for the Breeze path below)

 For Breeze (local development):

 * `Docker Desktop <https://www.docker.com/products/docker-desktop/>`__
 * `Podman <https://podman.io/>`__ is a license-friendly alternative to Docker Desktop, but it is
   **not yet fully supported** by Breeze. Breeze will detect Podman and print a warning, and you
   may run into issues that do not occur with Docker. Docker Desktop is recommended unless you are
   comfortable troubleshooting Podman-specific problems yourself. Track full support in
   `#54894 <https://github.com/apache/airflow/issues/54894>`_.
 * `Docker Compose <https://docs.docker.com/compose/install/>`__
 * `uv <https://github.com/astral-sh/uv>`__, which is a fast, reliable package manager that you'll use to install other developer tools to make contributing to Airflow easier.

 .. code-block:: bash

     curl -LsSf https://astral.sh/uv/install.sh | sh
 * `Prek <https://github.com/j178/prek>`__, which runs Airflow's required code-quality checks (formatting, linting, and bug-spotting) before you commit, helping save contributors and committers time during the pull request process.

 .. code-block:: bash

     uv tool install prek

 * 4GB RAM, 40GB disk space, and at least 2 CPU cores

 .. note::
    Docker installation varies by OS. See the `full guide <03_contributors_quick_start.rst#local-machine-development>`_
    for Ubuntu, macOS, and Windows instructions. Podman can be installed the same way, but since it is not
    yet fully supported by Breeze, use it at your own risk and prefer Docker if you run into problems.

 Option A – Breeze on Your Laptop
 --------------------------------
 1.  **Clone your fork**

 .. code-block:: bash

     git clone https://github.com/<you>/airflow.git
     cd airflow

 2.  **Set up Git hooks and install Breeze**

 Run these commands from the root of your cloned Airflow repository:

 .. code-block:: bash

     prek install -f
     prek install -f --hook-type pre-push
     ./scripts/tools/setup_breeze

 3. Setup your idea workspace to detect project src/ and tests/ folders as source roots.

 .. code-block:: bash

     # For IntelliJ IDEA and PyCharm
     uv run dev/ide_setup/setup_idea.py

     # For VS Code
     uv run dev/ide_setup/setup_vscode.py

 4. Create a branch for your change

 .. code-block:: bash

     git checkout -b <your-branch-name>

 5.  **Start the development container** (first run builds the image)

 .. code-block:: bash

     breeze start-airflow

 The command starts a shell and launches multiple terminals using ``mprocs`` by default
 and launches all Airflow necessary components in those terminals.
 You can also choose to use ``tmux`` via the ``--terminal-multiplexer tmux`` option.
 If you are using tmux, check out this cheat sheet to learn more about its commands: https://tmuxcheatsheet.com/.

 Now you can also access the Airflow UI on your local machine at `http://localhost:28080 <http://localhost:28080>`_ with user name ``admin`` and password ``admin``.

 To exit breeze, press ``q`` in the ``mprocs`` interface (or in any of the tmux panes) and clean up the resources by running the following command:

 .. code-block:: bash

     breeze down

 **Working with Dags in Breeze:**

 - **Adding your own Dags**: Place your Dag files in the ``/files/dags/`` directory in your local Airflow repository. This directory is automatically mounted into the Breeze container and your Dags will be visible in the Airflow UI.

 - **Loading example Dags**: Use the ``--load-example-dags`` flag to load all example Dags from the repository:

 .. code-block:: bash

     breeze start-airflow --load-example-dags

 This flag enables configuration to load example Dags when starting Airflow, which is useful for exploring Airflow's capabilities and testing.

 6. **Set up virtual environment and install dependencies**  - To run prek locally

 Install airflow dependencies from this `guide <03_contributors_quick_start.rst#setting-up-virtual-env>`__.

 7.  **Make a tiny change** – e.g. fix a typo in docs

 8.  **Run local checks**

 .. code-block:: bash

     prek --all-files

 9.  **Run tests**

 Run tests related to your change **before** pushing:

 .. code-block:: bash

     # Example: run core tests
     breeze testing core-tests

 Run ``breeze testing --help`` to see all available test groups.
 For more on testing, see the `Testing Guide <09_testing.rst>`_.

 10.  **Commit & push**

 .. code-block:: bash

     git commit -am "fix typo in README"
     git push -u origin <your-branch-name>

 11.  **Open the PR** – GitHub shows a "Compare & pull request" button.

 *Syncing your branch*

 .. code-block:: bash

     git fetch upstream && git rebase upstream/main && git push --force-with-lease

 Option B – One-Click GitHub Codespaces
 ---------------------------------------

 1. On **your fork**, click *Code → Codespaces → New codespace*.
 2. Wait for the VS Code web IDE to appear.  A terminal opens automatically.
 3. Install Docker Buildx and Docker Compose (required for Breeze)

 .. code-block:: bash

     mkdir -p ~/.docker/cli-plugins

     # Install Docker Buildx
     BUILDX_VERSION=v0.16.2
     curl -SL "https://github.com/docker/buildx/releases/download/${BUILDX_VERSION}/buildx-${BUILDX_VERSION}.linux-amd64" -o ~/.docker/cli-plugins/docker-buildx
     chmod +x ~/.docker/cli-plugins/docker-buildx
     docker buildx version

     # Install Docker Compose v2
     curl -SL "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s | tr '[:upper:]' '[:lower:]')-$(uname -m)" -o ~/.docker/cli-plugins/docker-compose
     chmod +x ~/.docker/cli-plugins/docker-compose
     docker compose version

 4. Verify Docker is accessible

 .. code-block:: bash

       docker info

    If ``docker info`` fails, try rebuilding the Codespace container
    (Command Palette → *Codespaces: Rebuild Container*) or restarting
    the Codespace from the GitHub Codespaces dashboard.

 5. Install Breeze and start the development container

 .. code-block:: bash

       curl -LsSf https://astral.sh/uv/install.sh | sh
       uv tool install prek
       prek install -f
       prek install -f --hook-type pre-push # for running mypy checks when pushing to repo
       ./scripts/tools/setup_breeze
       uv run dev/ide_setup/setup_vscode.py
       breeze start-airflow

 6. Edit a file in the editor, save, and commit via the Source Control sidebar.
    Push when prompted.

 7. Press **Create pull request** when GitHub offers.


 Review & Merge
 --------------
 Respond to reviewer comments, push updates (same commands as above).  Once
 CI is green and reviews are ✅, a committer will merge.  🎉

 Example Dag Review Checklist
 ----------------------------

 To help maintain consistency and quality across example Dags,
 please refer to the `example Dag review checklist <28_example_dag_review_checklist.rst>`_.

 Next Steps
 ----------
 * Need a full development environment? See the `Development Environments Guide <https://github.com/apache/airflow/blob/main/contributing-docs/06_development_environments.rst>`_.
 * Learn about our contribution workflow? Checkout the `Contribution Workflow Guide <https://github.com/apache/airflow/blob/main/contributing-docs/18_contribution_workflow.rst>`_.
	.. Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	.. http://www.apache.org/licenses/LICENSE-2.0

	.. Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.

	Your First Airflow Pull Request — 15-Minute Guide
	=================================================

	.. contents:: On this page
	:local:
	:depth: 1


	Purpose
	-------
	This page walks new contributors through opening their first
	Apache Airflow pull request (PR) in about 15 minutes. We present one
	local option (Breeze) and one fully-hosted option (GitHub Codespaces).
	Everything else lives in the advanced guides.

	Prerequisites
	-------------
	* GitHub account
	* `Fork <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo?tool=webui>`_ `apache/airflow <https://github.com/apache/airflow>`__
	* `Basic Git <https://docs.github.com/en/get-started/git-basics/set-up-git>`__ (only required for the Breeze path below)

	For Breeze (local development):

	* `Docker Desktop <https://www.docker.com/products/docker-desktop/>`__
	* `Podman <https://podman.io/>`__ is a license-friendly alternative to Docker Desktop, but it is
	not yet fully supported by Breeze. Breeze will detect Podman and print a warning, and you
	may run into issues that do not occur with Docker. Docker Desktop is recommended unless you are
	comfortable troubleshooting Podman-specific problems yourself. Track full support in
	`#54894 <https://github.com/apache/airflow/issues/54894>`_.
	* `Docker Compose <https://docs.docker.com/compose/install/>`__
	* `uv <https://github.com/astral-sh/uv>`__, which is a fast, reliable package manager that you'll use to install other developer tools to make contributing to Airflow easier.

	.. code-block:: bash

	curl -LsSf https://astral.sh/uv/install.sh \| sh
	* `Prek <https://github.com/j178/prek>`__, which runs Airflow's required code-quality checks (formatting, linting, and bug-spotting) before you commit, helping save contributors and committers time during the pull request process.

	.. code-block:: bash

	uv tool install prek

	* 4GB RAM, 40GB disk space, and at least 2 CPU cores

	.. note::
	Docker installation varies by OS. See the `full guide <03_contributors_quick_start.rst#local-machine-development>`_
	for Ubuntu, macOS, and Windows instructions. Podman can be installed the same way, but since it is not
	yet fully supported by Breeze, use it at your own risk and prefer Docker if you run into problems.

	Option A – Breeze on Your Laptop
	--------------------------------
	1. Clone your fork

	.. code-block:: bash

	git clone https://github.com/<you>/airflow.git
	cd airflow

	2. Set up Git hooks and install Breeze

	Run these commands from the root of your cloned Airflow repository:

	.. code-block:: bash

	prek install -f
	prek install -f --hook-type pre-push
	./scripts/tools/setup_breeze

	3. Setup your idea workspace to detect project src/ and tests/ folders as source roots.

	.. code-block:: bash

	# For IntelliJ IDEA and PyCharm
	uv run dev/ide_setup/setup_idea.py

	# For VS Code
	uv run dev/ide_setup/setup_vscode.py

	4. Create a branch for your change

	.. code-block:: bash

	git checkout -b <your-branch-name>

	5. Start the development container (first run builds the image)

	.. code-block:: bash

	breeze start-airflow

	The command starts a shell and launches multiple terminals using ``mprocs`` by default
	and launches all Airflow necessary components in those terminals.
	You can also choose to use ``tmux`` via the ``--terminal-multiplexer tmux`` option.
	If you are using tmux, check out this cheat sheet to learn more about its commands: https://tmuxcheatsheet.com/.

	Now you can also access the Airflow UI on your local machine at `http://localhost:28080 <http://localhost:28080>`_ with user name ``admin`` and password ``admin``.

	To exit breeze, press ``q`` in the ``mprocs`` interface (or in any of the tmux panes) and clean up the resources by running the following command:

	.. code-block:: bash

	breeze down

	Working with Dags in Breeze:

	- Adding your own Dags: Place your Dag files in the ``/files/dags/`` directory in your local Airflow repository. This directory is automatically mounted into the Breeze container and your Dags will be visible in the Airflow UI.

	- Loading example Dags: Use the ``--load-example-dags`` flag to load all example Dags from the repository:

	.. code-block:: bash

	breeze start-airflow --load-example-dags

	This flag enables configuration to load example Dags when starting Airflow, which is useful for exploring Airflow's capabilities and testing.

	6. Set up virtual environment and install dependencies - To run prek locally

	Install airflow dependencies from this `guide <03_contributors_quick_start.rst#setting-up-virtual-env>`__.

	7. Make a tiny change – e.g. fix a typo in docs

	8. Run local checks

	.. code-block:: bash

	prek --all-files

	9. Run tests

	Run tests related to your change before pushing:

	.. code-block:: bash

	# Example: run core tests
	breeze testing core-tests

	Run ``breeze testing --help`` to see all available test groups.
	For more on testing, see the `Testing Guide <09_testing.rst>`_.

	10. Commit & push

	.. code-block:: bash

	git commit -am "fix typo in README"
	git push -u origin <your-branch-name>

	11. Open the PR – GitHub shows a "Compare & pull request" button.

	Syncing your branch

	.. code-block:: bash

	git fetch upstream && git rebase upstream/main && git push --force-with-lease

	Option B – One-Click GitHub Codespaces
	---------------------------------------

	1. On your fork, click Code → Codespaces → New codespace.
	2. Wait for the VS Code web IDE to appear. A terminal opens automatically.
	3. Install Docker Buildx and Docker Compose (required for Breeze)

	.. code-block:: bash

	mkdir -p ~/.docker/cli-plugins

	# Install Docker Buildx
	BUILDX_VERSION=v0.16.2
	curl -SL "https://github.com/docker/buildx/releases/download/${BUILDX_VERSION}/buildx-${BUILDX_VERSION}.linux-amd64" -o ~/.docker/cli-plugins/docker-buildx
	chmod +x ~/.docker/cli-plugins/docker-buildx
	docker buildx version

	# Install Docker Compose v2
	curl -SL "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s \| tr '[:upper:]' '[:lower:]')-$(uname -m)" -o ~/.docker/cli-plugins/docker-compose
	chmod +x ~/.docker/cli-plugins/docker-compose
	docker compose version

	4. Verify Docker is accessible

	.. code-block:: bash

	docker info

	If ``docker info`` fails, try rebuilding the Codespace container
	(Command Palette → Codespaces: Rebuild Container) or restarting
	the Codespace from the GitHub Codespaces dashboard.

	5. Install Breeze and start the development container

	.. code-block:: bash

	curl -LsSf https://astral.sh/uv/install.sh \| sh
	uv tool install prek
	prek install -f
	prek install -f --hook-type pre-push # for running mypy checks when pushing to repo
	./scripts/tools/setup_breeze
	uv run dev/ide_setup/setup_vscode.py
	breeze start-airflow

	6. Edit a file in the editor, save, and commit via the Source Control sidebar.
	Push when prompted.

	7. Press Create pull request when GitHub offers.



	Review & Merge
	--------------
	Respond to reviewer comments, push updates (same commands as above). Once
	CI is green and reviews are ✅, a committer will merge. 🎉

	Example Dag Review Checklist
	----------------------------

	To help maintain consistency and quality across example Dags,
	please refer to the `example Dag review checklist <28_example_dag_review_checklist.rst>`_.

	Next Steps
	----------
	* Need a full development environment? See the `Development Environments Guide <https://github.com/apache/airflow/blob/main/contributing-docs/06_development_environments.rst>`_.
	* Learn about our contribution workflow? Checkout the `Contribution Workflow Guide <https://github.com/apache/airflow/blob/main/contributing-docs/18_contribution_workflow.rst>`_.