| # INSTALL / BUILD instructions for Apache Airflow |
| |
| This ia a generic installation method that requires a number of dependencies to be installed. |
| |
| Depending on your system you might need different prerequisites, but the following |
| systems/prerequisites are known to work: |
| |
| Linux (Debian Buster and Linux Mint Tricia): |
| |
| sudo apt install build-essentials python3.6-dev python3.7-dev python-dev openssl \ |
| sqlite sqlite-dev default-libmysqlclient-dev libmysqld-dev postgresq |
| |
| MacOS (Mojave/Catalina): |
| |
| brew install sqlite mysql postgresql |
| |
| # [required] fetch the tarball and untar the source move into the directory that was untarred. |
| |
| # [optional] run Apache RAT (release audit tool) to validate license headers |
| # RAT docs here: https://creadur.apache.org/rat/. Requires Java and Apache Rat |
| java -jar apache-rat.jar -E ./.rat-excludes -d . |
| |
| # [optional] Airflow pulls in quite a lot of dependencies in order |
| # to connect to other services. You might want to test or run Airflow |
| # from a virtual env to make sure those dependencies are separated |
| # from your system wide versions |
| |
| python3 -m venv PATH_TO_YOUR_VENV |
| source PATH_TO_YOUR_VENV/bin/activate |
| |
| # [required] building and installing by pip (preferred) |
| pip install . |
| |
| NOTE! |
| |
| On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. |
| This resolver does not yet work with Apache Airflow and might leads to errors in installation - |
| depends on your choice of extras. In order to install Airflow you need to either downgrade |
| pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option |
| ``--use-deprecated legacy-resolver`` to your pip install command. |
| |
| |
| # or you can install it directly via setup.py |
| python setup.py install |
| |
| |
| # You can also install recommended version of the dependencies by using |
| # constraint-python<PYTHON_MAJOR_MINOR_VERSION>.txt files as constraint file. This is needed in case |
| # you have problems with installing the current requirements from PyPI. |
| # There are different constraint files for different python versions and you shopuld choose the |
| # version of constraints specific for your version. |
| # For example: |
| |
| pip install . \ |
| --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-1.10.15/constraints-3.6.txt" |
| |
| |
| .. note:: |
| On 30th of November 2020, new version of PIP (20.3) has been released with a new, 2020 resolver. |
| This resolver does not yet work with Apache Airflow and might leads to errors in installation - |
| depends on your choice of extras. In order to install Airflow you need to either downgrade |
| pip to version 20.2.4 ``pip upgrade --pip==20.2.4`` or, in case you use Pip 20.3, you need to add option |
| ``--use-deprecated legacy-resolver`` to your pip install command. |
| |
| # You can also install Airflow with extras specified. The list of available extras: |
| # START EXTRAS HERE |
| |
| all, all_dbs, amazon, apache.atlas, apache.cassandra, apache.druid, apache.hdfs, apache.hive, |
| apache.pinot, apache.presto, apache.webhdfs, async, atlas, aws, azure, azure_blob_storage, |
| azure_container_instances, azure_cosmos, azure_data_lake, azure_secrets, cassandra, celery, cgroups, |
| cloudant, cncf.kubernetes, crypto, dask, databricks, datadog, devel, devel_all, devel_azure, |
| devel_ci, devel_hadoop, doc, docker, druid, elasticsearch, emr, gcp, gcp_api, github_enterprise, |
| google, google_auth, grpc, hashicorp, hdfs, hive, jdbc, jira, kerberos, kubernetes, ldap, |
| microsoft.azure, microsoft.mssql, microsoft.winrm, mongo, mssql, mysql, oracle, papermill, password, |
| pinot, postgres, presto, qds, rabbitmq, redis, s3, salesforce, samba, segment, sendgrid, sentry, |
| slack, snowflake, ssh, statsd, vertica, virtualenv, webhdfs, winrm |
| |
| # END EXTRAS HERE |
| |
| # For installing Airflow in development environments - see CONTRIBUTING.rst |