Container image to provide runtime environment for developing and testing Apache Ozone

Clone this repo:
  1. 6500bcd HDDS-8843. Bump RocksDB to 7.7.3 (#20) by Swaminathan Balachandran · 6 months ago master 20230615-1
  2. 75bcf89 HDDS-8628. Install lsof and netstat in ozone-runner (#19) by Doroszlai, Attila · 7 months ago 20230515-1
  3. e239441 HDDS-8488. Install openssl in runner image (#16) by Galsza · 7 months ago 20230503-1
  4. 4f43387 HDDS-8523. Upgrade Github Actions runner to ubuntu-latest (#17) by Galsza · 7 months ago
  5. a076c8e Create httpfs user and group (#15) by Zita Dombi · 11 months ago 20230104-1

Apache Ozone runner base image

This is the base image to run Apache Hadoop Ozone in docker containers. This is only for test/develop and not for production.

The container doesn't include any Ozone specific jar files or release artifacts just an empty environment which includes all the specific tools to run and test Apache Ozone inside containers.

The image is available as apache/ozone-runner. Build is managed by Docker Hub.

Development

To build the image, please use:

DOCKER_BUILDKIT=1 docker build -t apache/ozone-runner:dev .

To test it, build Apache Ozone:

mvn clean verify -DskipTests -Dskip.npx -DskipShade -Ddocker.ozone-runner.version=dev

And start the compose cluster:

cd hadoop-ozone/dist/target/ozone-*/compose/ozone
docker-compose up -d

*After merging PR, a new tag should pushed to the repository to create a new image. Use the convention: YYYYMMDD-N for tags where N is a daily counter (see the existing tags as an example).

After tag is published (and built by Docker Hub), the used runner version can be updated by modifying the docker.ozone-runner.version version in hadoop-ozone/dist/pom.xml

Building multi-architecture images

To build images with multiple architectures, use docker buildx.

For example, to build images for both linux/amd64 and linux/arm64, run:

docker buildx build --platform linux/amd64,linux/arm64 -t apache/ozone-runner:dev . --progress=plain

It might be slow when building the non-native architecture image due to QEMU emulation.