Google Git
Sign in
apache / dolphinscheduler-sdk-python / 7fb5a335dc41a5715c38e43803f97d058650387e
commit7fb5a335dc41a5715c38e43803f97d058650387e[log] [tgz]
authorJay Chung <zhongjiajie955@gmail.com>Fri Apr 28 14:49:10 2023 +0800
committerGitHub <noreply@github.com>Fri Apr 28 14:49:10 2023 +0800
tree8324a84620e8d5c2c96dd3ce35f6f2d6d03c458e
parente166fc281d8249531a30f6c0d8f7ad60a46949de [diff]
doc: Correct support version between pydolphinscheduler and dolphinscheduler (#92)
2 files changed
tree: 8324a84620e8d5c2c96dd3ce35f6f2d6d03c458e
  1. .github/
  2. docs/
  3. examples/
  4. src/
  5. tests/
  6. .asf.yaml
  7. .dlc.json
  8. .gitignore
  9. .licenserc.yaml
  10. .pre-commit-config.yaml
  11. CONTRIBUTING.md
  12. LICENSE
  13. MANIFEST.in
  14. NOTICE
  15. README.md
  16. RELEASE.md
  17. setup.cfg
  18. setup.py
  19. UPDATING.md
README.md

PyDolphinScheduler

PyPi Version PyPi Python Versions PyPi License PyPi Status Downloads Coverage Status Code style: black Imports: isort CI Twitter Follow Slack Status

PyDolphinScheduler is python API for Apache DolphinScheduler, which allow you definition your workflow by python code, aka workflow-as-codes.

Quick Start

Version Compatibility

At Nov 7, 2022 we seperated PyDolphinScheduler from DolphinScheduler, and the version of PyDolphinScheduler 4.0.0 can match multiple versions of DolphinScheduler, for more details, please refer to version

Installation

# Install
python -m pip install apache-dolphinscheduler

# Verify installation is successful, it will show the version of apache-dolphinscheduler, here we use 0.1.0 as example
pydolphinscheduler version
# 0.1.0

NOTE: package apache-dolphinscheduler not work on above Python version 3.10(including itself) in Window operating system due to dependence py4j not work on those environments.

Here we show you how to install and run a simple example of PyDolphinScheduler

Start DolphinScheduler

There are many ways to start DolphinScheduler, here we use docker to start and run it as a standalone server.

# Change the version of dolphinscheduler to the version you want to use, here we use 3.1.1 as example
DOLPHINSCHEDULER_VERSION=3.1.1
docker run --name dolphinscheduler-standalone-server -p 12345:12345 -p 25333:25333 -d apache/dolphinscheduler-standalone-server:"${DOLPHINSCHEDULER_VERSION}"

After the container is started, you can access the DolphinScheduler UI via http://localhost:12345/dolphinscheduler. For more way to start DolphinScheduler and the more detail about DolphinScheduler, please refer to DolphinScheduler

Run a simple example

We have many examples in examples directory, we here pick up a typical one to show how to run it.

# Get the latest code of example from github 
curl https://raw.githubusercontent.com/apache/dolphinscheduler-sdk-python/main/src/pydolphinscheduler/examples/tutorial.py -o ./tutorial.py

# Change tenant to real exists tenant in the host your DolphinScheduler running, by any editor you like 

# Run the example
python ./tutorial.py

NOTICE: Since Apache DolphinScheduler's tenant is requests while running command, you have to change tenant value in tutorial.py. The default value is tenant_exists`, change it to username exists your host.

After that, a new workflow will be created by PyDolphinScheduler, and you can see it in DolphinScheduler web UI‘s Project Management page. It will trigger the workflow automatically, so you can see the workflow running in DolphinScheduler web UI’s Workflow Instance page too. For more detail about any function about DolphinScheduler Project Management, please refer to DolphinScheduler Workflow

Documentation

For full documentation visit document. This documentation is generated from this repository so please raise issues or pull requests for any additions, corrections, or clarifications.

Contributing

If you would like to contribute, check out the open issues on GitHub. You can also see the guide to contributing.

Release

Follow the release guide to release a new version of PyDolphinScheduler.