Contributions are welcome and are greatly appreciated! Every little bit helps, and credit will always be given.
Report bugs through Apache Jira
Please report relevant information and preferably code that exhibits the problem.
Look through the Jira issues for bugs. Anything is open to whoever wants to implement it.
Look through the GitHub issues for features. Anything tagged with “feature” is open to whoever wants to implement it.
We've created the operators, hooks, macros and executors we needed, but we made sure that this part of Airflow is extensible. New operators, hooks and operators are very welcomed!
Airflow could always use better documentation, whether as part of the official Airflow docs, in docstrings, docs/*.rst
or even on the web as blog posts or articles.
The best way to send feedback is to file an issue on Github.
If you are proposing a feature:
The latest API documentation is usually available here. To generate a local version, you need to have installed airflow with the doc
extra. In that case you can generate the doc by running:
cd docs && ./build.sh
Please install python(2.7.x or 3.4.x), mysql, and libxml by using system-level package managers like yum, apt-get for Linux, or homebrew for Mac OS at first. It is usually best to work in a virtualenv and tox. Install development requirements:
cd $AIRFLOW_HOME virtualenv env source env/bin/activate pip install -e .[devel] tox
Feel free to customize based on the extras available in setup.py
Before you submit a pull request from your forked repo, check that it meets these guidelines:
flake8 airflow tests
Tests can then be run with (see also the Running unit tests section below):
./run_unit_tests.sh
We highly recommend setting up Travis CI on your repo to automate this. It is free for open source projects. If for some reason you cannot, you can use the steps below to run tests.
Here are loose guidelines on how to get your environment to run the unit tests. We do understand that no one out there can run the full test suite since Airflow is meant to connect to virtually any external system and that you most likely have only a subset of these in your environment. You should run the CoreTests and tests related to things you touched in your PR.
To set up a unit test environment, first take a look at run_unit_tests.sh
and understand that your AIRFLOW_CONFIG
points to an alternate config file while running the tests. You shouldn't have to alter this config file but you may if need be.
From that point, you can actually export these same environment variables in your shell, start an Airflow webserver airflow webserver -d
and go and configure your connection. Default connections that are used in the tests should already have been created, you just need to point them to the systems where you want your tests to run.
Once your unit test environment is setup, you should be able to simply run ./run_unit_tests.sh
at will.
For example, in order to just execute the “core” unit tests, run the following:
./run_unit_tests.sh tests.core:CoreTest -s --logging-level=DEBUG
or a single test method:
./run_unit_tests.sh tests.core:CoreTest.test_check_operators -s --logging-level=DEBUG
For more information on how to run a subset of the tests, take a look at the nosetests docs.
See also the list of test classes and methods in tests/core.py
.
When developing features the need may arise to persist information to the the metadata database. Airflow has Alembic built-in to handle all schema changes. Alembic must be installed on your development machine before continuing.
# starting at the root of the project $ pwd ~/airflow # change to the airflow directory $ cd airflow $ alembic revision -m "add new field to db" Generating ~/airflow/airflow/migrations/versions/12341123_add_new_field_to_db.py