PyDolphinScheduler office release is in ASF Distribution Directory, but we also have a PyPi repository for Python package distribution for convenience.
sphinx-multiversion
dependency in setup.py
, we still can not fix this issue Distribute tarball and wheel error with direct dependencyversion_ext
about the dolphinscheduler version current support, the syntax is respect pep-044tox -e local-ci && tox -e local-integrate-test
, after you start dolphinscheduler to pass local-integrate-test
We use build to build package, and twine to upload package to PyPi. You could first install and upgrade them by:
# Install or upgrade dependencies python3 -m pip install --upgrade pip build twine # Configuration during the releasing VERSION=<VERSION> # The version of the package you want to release, e.g. 1.2.3 REMOTE=<REMOTE> # The git remote name, we usually use `origin` or `remote` # Change version # For macOS sed -i '' "s/__version__ = \".*\"/__version__ = \"${VERSION}\"/" src/pydolphinscheduler/__init__.py # For Linux sed -i "s/__version__ = \".*\"/__version__ = \"${VERSION}\"/" src/pydolphinscheduler/__init__.py git commit -am "Release v${VERSION}" # Add Tag git tag -a "${VERSION}" -m "Release v${VERSION}" git push "${REMOTE}" --tags # Build and sign according to the Apache requirements python setup.py clean && python setup.py sdist
Go to GitHub release page and create a release based to the specific tag, and set it as pre-release.
svn co https://dist.apache.org/repos/dist/dev/dolphinscheduler/ release/dolphinscheduler mkdir -p release/dolphinscheduler/python/"${VERSION}" cp dist/dolphinscheduler-python-src-"${VERSION}"* release/dolphinscheduler/python/"${VERSION}" cd release/dolphinscheduler && svn add python/"${VERSION}" && svn commit python -m "Release Apache DolphinScheduler-SDK-Python version ${VERSION}"
Create a new vote in dev@dolphinscheduler.apache.org
TITLE: [VOTE] Release Apache DolphinScheduler SDK Python <VERSION> BODY: Hello DolphinScheduler Community, This is a call for vote to release Apache DolphinScheduler SDK Python version <VERSION> Release notes: https://github.com/apache/dolphinscheduler-sdk-python/releases/tag/<VERSION> The release candidates: https://dist.apache.org/repos/dist/dev/dolphinscheduler/python/<VERSION>/ Git tag for the release: https://github.com/apache/dolphinscheduler-sdk-python/tree/<VERSION> Release Commit ID: https://github.com/apache/dolphinscheduler-sdk-python/commit/<commit-SHA> Keys to verify the Release Candidate: https://dist.apache.org/repos/dist/dev/dolphinscheduler/KEYS The vote will be open for at least 72 hours or until necessary number of votes are reached. Please vote accordingly: [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove with the reason Checklist for reference: [ ] Download links are valid. [ ] Checksums and PGP signatures are valid. [ ] Source code artifacts have correct names matching the current release. [ ] LICENSE and NOTICE files are correct for each DolphinScheduler repo. [ ] All files have license headers if necessary. [ ] No compiled archives bundled in source archive.
.asc
, and sha512
files.LICENSE
and NOTICE
are in source codes and distribution package.sha512
and gpg
signature with following command:VERSION=<VERSION> # The version of the package you want to release, e.g. 1.2.3 # Download source code curl https://dist.apache.org/repos/dist/dev/dolphinscheduler/python/"${VERSION}"/dolphinscheduler-python-src-"${VERSION}".tar.gz > dolphinscheduler-python-src-"${VERSION}".tar.gz curl https://dist.apache.org/repos/dist/dev/dolphinscheduler/python/"${VERSION}"/dolphinscheduler-python-src-"${VERSION}".tar.gz.asc > dolphinscheduler-python-src-"${VERSION}".tar.gz.asc curl https://dist.apache.org/repos/dist/dev/dolphinscheduler/python/"${VERSION}"/dolphinscheduler-python-src-"${VERSION}".tar.gz.sha512 > dolphinscheduler-python-src-"${VERSION}".tar.gz.sha512 # Verify sha512 sha512sum --check dolphinscheduler-python-src-"${VERSION}".tar.gz.sha512 # Verify gpg signature curl https://dist.apache.org/repos/dist/release/dolphinscheduler/KEYS > KEYS gpg --import KEYS gpg --verify dolphinscheduler-python-src-"${VERSION}".tar.gz.asc
Vote result should follow these:
PMC vote is +1 binding, all others is +1 no binding.
Within 72 hours, you get at least 3 (+1 binding), and have more +1 than -1. Vote pass.
Send the closing vote mail to announce the result. When count the binding and no binding votes, please list the names of voters. An example like this:
[RESULT][VOTE] Release Apache SkyWalking Python version $VERSION 72+ hours passed, we’ve got ($NUMBER) +1 bindings (and ... +1 non-bindings): (list names) +1 bindings: xxx ... +1 non-bindings: xxx ... Thank you for voting, I’ll continue the release process.
Move source codes tar balls and distributions to https://dist.apache.org/repos/dist/release/dolphinscheduler/, you can do this only if you are a PMC member.
# Move from dev to release svn mv https://dist.apache.org/repos/dist/dev/dolphinscheduler/python/"${VERSION}" https://dist.apache.org/repos/dist/release/dolphinscheduler/python/"${VERSION}" # Delete the old version svn delete -m "remove old release Python SDK" https://dist.apache.org/repos/dist/release/dolphinscheduler/python/<PREVIOUS-RELEASE-VERSION>
Update GitHub release page and set it as formal release.
Send ANNOUNCE email to dev@dolphinscheduler.apache.org
and announce@apache.org
.
Subject: [ANNOUNCE] Apache DolphinScheduler SDK Python $VERSION Released Content: Hi Community, We are glad to announce the release of Apache DolphinScheduler SDK Python $VERSION. Apache DolphinScheduler SDK Python is an API for Apache DolphinScheduler which allow you definition your workflow by Python code, aka workflow-as-codes. DolphinScheduler is a distributed and easy-to-extend visual workflow scheduler system, dedicated to solving the complex task dependencies in data processing, making the scheduler system out of the box for data processing. Download Links: https://dolphinscheduler.apache.org/#/en-us/download Release Notes: https://github.com/apache/dolphinscheduler-sdk-python/releases/tag/$VERSION DolphinScheduler Resources: - Issue: https://github.com/apache/dolphinscheduler-sdk-python/issues/ - Mailing list: dev@dolphinscheduler.apache.org - Website: https://dolphinscheduler.apache.org - Documents: https://dolphinscheduler.apache.org/python/$VERSION
PyPI, Python Package Index, is a repository of software for the Python programming language.
TestPyPi is a test environment of PyPi, you could release to it to test whether the package is work or not.
dist
directory, and build package python3 setup.py clean
.python3 -m build
, and you will see two new files in dist
directory, with extension .tar.gz
and .whl
.python3 -m twine upload --repository testpypi dist/*
.python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps apache-dolphinscheduler
to test whether it is work or not.PyPi is the official repository of Python packages, it is highly recommended releasing package to TestPyPi first to test whether the package is correct.
dist
directory, and build package python3 setup.py clean
.python3 -m build
, and you will see two new files in dist
directory, with extension .tar.gz
and .whl
.python3 -m twine upload dist/*
.python3 -m pip install apache-dolphinscheduler
to install it.