PyPI CD Pipeline

The Jenkins pipelines for continuous delivery of the PyPI MXNet packages. The pipelines for each variant are run, and fail, independently. Each depends on a successful build of the statically linked libmxet library.

The pipeline relies on the scripts and resources located in tools/pip to build the PyPI packages.

Credentials

The pipeline depends on the following environment variables in order to successfully retrieve the credentials for the PyPI account:

  • CD_PYPI_SECRET_NAME
  • DOCKERHUB_SECRET_ENDPOINT_URL
  • DOCKERHUB_SECRET_ENDPOINT_REGION

The credentials are stored in the Secrets Manager of the AWS account hosting Jenkins. The pypi_publish.py script is in charge of retrieving the credentials.

Mock publishing

Because of space limitations on PyPI, we don't want to push test packages from Jenkins Dev everytime the pipeline is run. Therefore, the pypi_publish.sh script will fake publishing packages if the username is skipPublish.