Google Git
Sign in
apache / airflow-client-python / refs/heads/perform-housekeeping-for-python-client
commit8eaf57be9491757ab34e1d14e6c484a0ff0f0158[log] [tgz]
authorJarek Potiuk <jarek@potiuk.com>Sat Jan 13 21:57:10 2024 +0100
committerJarek Potiuk <jarek@potiuk.com>Sat Jan 13 22:06:04 2024 +0100
tree9ac655cf012d1a7a77db3f9decbb9a883dae7598
parent4775ec82a6393ad9a59af889e492e5aabb0b01ba [diff]
Housekeeping of Python client repository

After working on reproducible package proparation for Python client
it's been found that the Python client repository had some extra
code added and docs and tests were placed in a wrong directory.

As of https://github.com/apache/airflow-client-python/pull/93, we
have duplicated airflow_client code:

* airflow_client.client - this is the one that should be used
* airflow_client.airflow_client - this is the one that is added
  accidentally (and it is a copy of the `client` one)

Also `docs` and `tests` are generated in "airflow_client" package,
but this is wrong, both docs and tests should be at the top-level
of the repository.

This PR:

* deletes the extra code
* moves "docs" and "test" folders to the top-level of the project.
401 files changed
tree: 9ac655cf012d1a7a77db3f9decbb9a883dae7598
  1. .github/
  2. airflow_client/
  3. dev/
  4. docs/
  5. license-templates/
  6. test/
  7. .asf.yaml
  8. .gitignore
  9. .pre-commit-config.yaml
  10. .rat-excludes
  11. CHANGELOG.md
  12. INSTALL
  13. LICENSE
  14. NOTICE
  15. README.md
  16. requirements.txt
  17. setup.cfg
  18. setup.py
  19. test-requirements.txt
README.md

Apache Airflow Python Client

Requirements.

Python >= 3.7

Installation & Usage

pip install

You can install directly using pip:

pip install apache-airflow-client

Setuptools

Or install via Setuptools.

git clone git@github.com:apache/airflow-client-python.git
cd airflow-client-python
python setup.py install --user

(or sudo python setup.py install to install the package for all users)

Then import the package:

import airflow_client.client

Changelog

See CHANGELOG.md for keeping track on what has changed in the client.

Getting Started

Please follow the installation procedure and then run the following example python script:

import uuid

import airflow_client.client
try:
    # If you have rich installed, you will have nice colored output of the API responses
    from rich import print
except ImportError:
    print("Output will not be colored. Please install rich to get colored output: `pip install rich`")
    pass
from airflow_client.client.api import config_api, dag_api, dag_run_api
from airflow_client.client.model.dag_run import DAGRun

# The client must use the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.
#
# In case of the basic authentication below, make sure that Airflow is
# configured also with the basic_auth as backend additionally to regular session backend needed
# by the UI. In the `[api]` section of your `airflow.cfg` set:
#
# auth_backend = airflow.api.auth.backend.session,airflow.api.auth.backend.basic_auth
#
# Make sure that your user/name are configured properly - using the user/password that has admin
# privileges in Airflow

# Configure HTTP basic authorization: Basic
configuration = airflow_client.client.Configuration(
    host="http://localhost:8080/api/v1",
    username='admin',
    password='admin'
)

# Make sure in the [core] section, the  `load_examples` config is set to True in your airflow.cfg
# or AIRFLOW__CORE__LOAD_EXAMPLES environment variable set to True
DAG_ID = "example_bash_operator"

# Enter a context with an instance of the API client
with airflow_client.client.ApiClient(configuration) as api_client:

    errors = False

    print('[blue]Getting DAG list')
    dag_api_instance = dag_api.DAGApi(api_client)
    try:
        api_response = dag_api_instance.get_dags()
        print(api_response)
    except airflow_client.client.OpenApiException as e:
        print("[red]Exception when calling DagAPI->get_dags: %s\n" % e)
        errors = True
    else:
        print('[green]Getting DAG list successful')


    print('[blue]Getting Tasks for a DAG')
    try:
        api_response = dag_api_instance.get_tasks(DAG_ID)
        print(api_response)
    except airflow_client.client.exceptions.OpenApiException as e:
        print("[red]Exception when calling DagAPI->get_tasks: %s\n" % e)
        errors = True
    else:
        print('[green]Getting Tasks successful')


    print('[blue]Triggering a DAG run')
    dag_run_api_instance = dag_run_api.DAGRunApi(api_client)
    try:
        # Create a DAGRun object (no dag_id should be specified because it is read-only property of DAGRun)
        # dag_run id is generated randomly to allow multiple executions of the script
        dag_run = DAGRun(
            dag_run_id='some_test_run_' + uuid.uuid4().hex,
        )
        api_response = dag_run_api_instance.post_dag_run(DAG_ID, dag_run)
        print(api_response)
    except airflow_client.client.exceptions.OpenApiException as e:
        print("[red]Exception when calling DAGRunAPI->post_dag_run: %s\n" % e)
        errors = True
    else:
        print('[green]Posting DAG Run successful')

    # Get current configuration. Note, this is disabled by default with most installation.
    # You need to set `expose_config = True` in Airflow configuration in order to retrieve configuration.
    conf_api_instance = config_api.ConfigApi(api_client)
    try:
        api_response = conf_api_instance.get_config()
        print(api_response)
    except airflow_client.client.OpenApiException as e:
        print("[red]Exception when calling ConfigApi->get_config: %s\n" % e)
        errors = True
    else:
        print('[green]Config retrieved successfully')

    if errors:
        print ('\n[red]There were errors while running the script - see above for details')
    else:
        print ('\n[green]Everything went well')

See README for full client API documentation.