Google Git
Sign in
apache / couchdb-ci / f4805b90738b724551020ef50112bfc2c796a83e
commitf4805b90738b724551020ef50112bfc2c796a83e[log] [tgz]
authorJoan Touzet <wohali@apache.org>Thu May 02 20:14:21 2019 -0700
committerJoan Touzet <wohali@apache.org>Thu May 02 20:14:21 2019 -0700
tree18485230fa82c6c152e4d166ab51785373118d2c
parentc0e51f6925f5e83eb29f2e583dbdcdb30f424c6b [diff]
Clean up .travis.yml
1 file changed
tree: 18485230fa82c6c152e4d166ab51785373118d2c
  1. bin/
  2. dockerfiles/
  3. files/
  4. .dockerignore
  5. .gitignore
  6. .travis.yml
  7. build.sh
  8. freebsd-jenkins-setup.md
  9. README.md
  10. TODO.md
README.md

CouchDB Continuous Integration (CI) support repo

The main purpose of this repository is to provide scripts that:

  • Install the necessary build-time dependencies for CouchDB on a number of platforms, either inside or outside of a container or VM
  • Build Docker containers with those dependencies necessary to build binary JavaScript (SpiderMonkey 1.8.5) packages
  • Build Docker containers with all dependencies necessary to build CouchDB, including Erlang and JavaScript

It intends to cover a range of both operating systems (Linux, macOS, BSD, Windows) and Erlang versions (17.x, 18.x, 19.x, etc.)

These images are used by Apache Jenkins CI to build CouchDB with every checkin to master or a release branch (e.g., 2.3.0).

CouchDB's CI build philosophy is to use Travis (with kerl) to validate CouchDB against different Erlang versions, and to use Jenkins to validate CouchDB against different OSes and architectures. Where possible, Jenkins also auto-builds convenience binaries or packages. The eventual goal is that these auto-built binaries/packages/Docker images will be auto-pushed to our distribution repos for downstream consumption.

Supported Configurations (updated 2018-12-05)

OS / distroVersionErlang VersionArchitectureDocker?
debianjessie17.5.3x86_64:heavy_check_mark:
debianjessie19.3.6x86_64:heavy_check_mark:
debianstretch19.3.6x86_64:heavy_check_mark:
debianstretch19.3.6aarch64:heavy_check_mark:
ubuntutrusty19.3.6x86_64:heavy_check_mark:
ubuntuxenial19.3.6x86_64:heavy_check_mark:
ubuntubionic19.3.6x86_64:heavy_check_mark:
centos619.3.6x86_64:heavy_check_mark:
centos719.3.6x86_64:heavy_check_mark:
freebsd11.xdefaultx86_64:x:
freebsd12.0defaultx86_64:x:

Docker

For those OSes that support Docker, we run builds inside of Docker containers. These containers are built using the build.sh command at the root level.

Building a “base image”

The base images include all of the build dependencies necessary to build CouchDB except for Erlang and SpiderMonkey 1.8.5. These images are typically used to build the CouchDB SpiderMonkey 1.8.5 binaries for a given OS/version/architecture combination.

Build a base image with:

./build.sh base <distro>-<version>

Building a “platform image”

The platform images include all of the build dependencies necessary to build and full test CouchDB on a given OS/version/architecture combination.

Build a platform image with:

./build.sh platform <distro>-<version>

Building the special Debian Jessie 17.5.3 x86_64 image

We use this image to build the initial tarball, before running the build test on other platforms. We do this because we want to generate a rebar binary compatible with all versions of Erlang we support. If we do this on too new a version, older Erlangs won't recognize it. At present, Erlang 17 is the oldest version we still support.

The build command is:

ERLANGVERSION=17.5.3 ./build.sh platform debian-jessie

Publishing a container

  1. You need a Docker Cloud account with access to the couchdbdev organization. Ask the CouchDB PMC for assistance with this.
  2. export DOCKER_ID_USER="username"
  3. docker login and enter your password.
  4. ./build.sh publish <distro>-<version> just as above.

Useful things you can do

Full build.sh options

./build.sh <command> [OPTIONS]

Recognized commands:
  clean <plat>          Removes all images for <plat>.
  clean-all             Removes all images for all platforms & base images.

  base <plat>           Builds the base (no JS/Erlang) image for <plat>.
  base-all              Builds all base (no JS/Erlang) images.
  *base-upload          Uploads the specified couchdbdev/*-base image
                        to Docker Hub.
  *base-upload-all      Uploads all the couchdbdev/*-base images.

  platform <plat>       Builds the image for <plat> with Erlang & JS support.
  platform-all          Builds all images with Erlang and JS support.
  *platform-upload      Uploads the couchdbdev/*-erlang-* images to Docker Hub.
  *platform-upload-all  Uploads all the couchdbdev/*-erlang-* images to Docker.

  couch <plat>          Builds and tests CouchDB for <plat>.
  couch-all             Builds and tests CouchDB on all platforms.

  Commands marked with * require appropriate Docker Hub credentials.

Interactively working in a built container

After building the image as above:

docker run -it couchdbdev/<tag>

where <tag> is of the format <distro>-<version>-<type>, such as debian-stretch-erlang-19.3.6.

Running the CouchDB build in a published container

./build.sh couch <distro>-<version>

Building SpiderMonkey 1.8.5 convenience packages

After building the base image as above, head over to the apache/couchdb-pkg repository and follow the instructions there.

Adding support for a new release/platform/architecture

  1. Update the build scripts in the bin/ directory to install the dependencies correctly on your new OS/version/platform. Push a PR with these changes.
  2. Copy and customize an appropriate Dockerfile in the dockerfiles directory for your new OS.
  3. Build a base image using ./build.sh base <distro>-<version>. Solve any problems with the build process here. Add your new platform combination to the .travis.yml file, then push a PR with these changes.
  4. Using the apache/couchdb-pkg repository, validate you can build the JS package. Fix any problems in that repo that arise and raise a new PR. Open a new issue on that PR requesting that the JS packages be made available through the CouchDB repository/download infrastructure.
  5. Build a full platform image with ./build.sh platform <distro>-<version>. Solve any problems with the build process here. Add your new platform combination to the .travis.yml file, then push a PR with these changes.
  6. Submit a PR against the apache/couchdb repository, adding the new platform to the top level Jenkinsfile. Ask if you need help.

Other platforms

We are eager for contributions to enhance the build scripts to support setting up machines with the necessary build environment for:

as well as alternative architectures for the already supported image types (arm, ppc64le, s390x, sparc, etc).

We know that Docker won't support some of these, but we should be able to at least expand the install scripts for all of these platforms.

Background

See: