Google Git
Sign in
apache / airflow-client-python / refs/tags/2.6.0
commit1d5134ae696a71b1a91fac06439707d6aa3f2e7c[log] [tgz]
authorJarek Potiuk <jarek@potiuk.com>Sun May 07 16:35:40 2023 +0200
committerGitHub <noreply@github.com>Sun May 07 15:35:40 2023 +0100
tree2427850b212b89f373c4a6a6e35159920ad5729d
parent3057a3931ba954a8ce56d8f0498e9f0ecdf7be0b [diff]
Refresh README, examples and changelog with recent changes (#78)

This PR refreshes README, examples of our and changelog to better
emphasise recent changes in the validation/open API of ours
clarifying read-only properties in our API.

Specifically DAGRun dag_id has not been marked as read-only so far
which was a bug, and some of our "dev" examples set the read only
properties, which could mislead our users.

We've recently clarified that those properties are read-only and
we are now updating the API documentation and examples to show that
the good way of using related APIs.

Also the examples have been refrehsed and modernized a bit as well
as the dev example has been synchronized automatically (via
pre-commit) to make sure it is the same in the README and in dev
where we are using it to run the tests with the API.

Minimum version of Python have been set to 3.7 and CHANGELOG was
updated to reflect all the changes, as well as it has been
exposed in Project_URL of the package in PyPI.

Link in README have been changed to URL so that they are properly
rendered in README in `pypi` package documentation (currently
following the links to other filess in the repo there fails.
6 files changed
tree: 2427850b212b89f373c4a6a6e35159920ad5729d
  1. .github/
  2. airflow_client/
  3. dev/
  4. license-templates/
  5. .asf.yaml
  6. .gitignore
  7. .pre-commit-config.yaml
  8. .rat-excludes
  9. CHANGELOG.md
  10. INSTALL
  11. LICENSE
  12. NOTICE
  13. README.md
  14. requirements.txt
  15. setup.cfg
  16. setup.py
  17. test-requirements.txt
README.md

Apache Airflow Python Client

Requirements.

Python >= 3.7

Installation & Usage

pip install

You can install directly using pip:

pip install apache-airflow-client

Setuptools

Or install via Setuptools.

git clone git@github.com:apache/airflow-client-python.git
cd airflow-client-python
python setup.py install --user

(or sudo python setup.py install to install the package for all users)

Then import the package:

import airflow_client.client

Changelog

See CHANGELOG.md for keeping track on what has changed in the client.

Getting Started

Please follow the installation procedure and then run the following example python script:

import uuid

import airflow_client.client
try:
    # If you have rich installed, you will have nice colored output of the API responses
    from rich import print
except ImportError:
    print("Output will not be colored. Please install rich to get colored output: `pip install rich`")
    pass
from airflow_client.client.api import config_api, dag_api, dag_run_api
from airflow_client.client.model.dag_run import DAGRun

# The client must use the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.
#
# In case of the basic authentication below, make sure that Airflow is
# configured also with the basic_auth as backend additionally to regular session backend needed
# by the UI. In the `[api]` section of your `airflow.cfg` set:
#
# auth_backend = airflow.api.auth.backend.session,airflow.api.auth.backend.basic_auth
#
# Make sure that your user/name are configured properly - using the user/password that has admin
# privileges in Airflow

# Configure HTTP basic authorization: Basic
configuration = airflow_client.client.Configuration(
    host="http://localhost:8080/api/v1",
    username='admin',
    password='admin'
)

# Make sure in the [core] section, the  `load_examples` config is set to True in your airflow.cfg
# or AIRFLOW__CORE__LOAD_EXAMPLES environment variable set to True
DAG_ID = "example_bash_operator"

# Enter a context with an instance of the API client
with airflow_client.client.ApiClient(configuration) as api_client:

    errors = False

    print('[blue]Getting DAG list')
    dag_api_instance = dag_api.DAGApi(api_client)
    try:
        api_response = dag_api_instance.get_dags()
        print(api_response)
    except airflow_client.client.OpenApiException as e:
        print("[red]Exception when calling DagAPI->get_dags: %s\n" % e)
        errors = True
    else:
        print('[green]Getting DAG list successful')


    print('[blue]Getting Tasks for a DAG')
    try:
        api_response = dag_api_instance.get_tasks(DAG_ID)
        print(api_response)
    except airflow_client.client.exceptions.OpenApiException as e:
        print("[red]Exception when calling DagAPI->get_tasks: %s\n" % e)
        errors = True
    else:
        print('[green]Getting Tasks successful')


    print('[blue]Triggering a DAG run')
    dag_run_api_instance = dag_run_api.DAGRunApi(api_client)
    try:
        # Create a DAGRun object (no dag_id should be specified because it is read-only property of DAGRun)
        # dag_run id is generated randomly to allow multiple executions of the script
        dag_run = DAGRun(
            dag_run_id='some_test_run_' + uuid.uuid4().hex,
        )
        api_response = dag_run_api_instance.post_dag_run(DAG_ID, dag_run)
        print(api_response)
    except airflow_client.client.exceptions.OpenApiException as e:
        print("[red]Exception when calling DAGRunAPI->post_dag_run: %s\n" % e)
        errors = True
    else:
        print('[green]Posting DAG Run successful')

    # Get current configuration. Note, this is disabled by default with most installation.
    # You need to set `expose_config = True` in Airflow configuration in order to retrieve configuration.
    conf_api_instance = config_api.ConfigApi(api_client)
    try:
        api_response = conf_api_instance.get_config()
        print(api_response)
    except airflow_client.client.OpenApiException as e:
        print("[red]Exception when calling ConfigApi->get_config: %s\n" % e)
        errors = True
    else:
        print('[green]Config retrieved successfully')

    if errors:
        print ('\n[red]There were errors while running the script - see above for details')
    else:
        print ('\n[green]Everything went well')

See README for full client API documentation.