Add support for Junit5 tests and fix CI that has been broken (#4197)

* Add support for Junit5 tests

- upgrade surefire plugin
- replace <forkMode>always</forkMode> with supported way in newer maven-surefire-plugin
- maven-failsafe-plugin version must match maven-surefire-plugin version
- remove invalid test case. Log4j is in the classpath for other reasons
  checked by running
  "mvn -pl tests/integration-tests-utils/pom.xml test -X"
  also verified by debugging that it's not in the system classpath, it is loaded by the
  MavenClassLoader
- add assertj-core for fluent assertions in tests

* Upgrade arquillian-junit-standalone

* Fix error in pom.xml

* Fix java.lang.ClassNotFoundException: javax.ws.rs.ProcessingException

* Pick docker-java-api version that is compatible with arquillian-cube-docker

* Fix snakeyaml issue

* Fix running of bookkeeper in Docker in tests

* Improve GitHub Actions CI workflow logging

* Increase client tests timeout

* Add script for picking fast ubuntu mirror for docker build

* upgrade maven-dependency-plugin version (used for copying .tar.gz for docker image building)

* Fix building of the docker image

* Increase timeouts for integration tests

* Use eclipse-temurin base image

* Replace "FROM ubuntu:22.10" with "FROM ubuntu:22.04" to use LTS version of ubuntu

22.10 (kinetic) is EOL since July 20 2023

* Build docker image also for backward-compatibility-tests

* Install gpg that is required by the scripts

* Add UBUNTU_MIRROR

* Remove redundancy from integration tests

* Fix missing dependency

* Fix classpath issues for arquillian-cube

* Upgrade actions versions

* Fix invalid references in native-io/pom.xml

* Fix circe-checksum native library compilation

* Migrate Gradle specific settings to Maven

* Another attempt to disable backward-compat tests in integration test build job

* Add working solution for skipping backward compat tests in the int tests

* Attempt to fix issue in locating the file

* Fix tar file extraction

* Improve solution

* Backward compat tests build commands without int tests

* Copy tune-runner-vm and clean-disk from Pulsar CI and clean disk

* Revisit skipping of backward compat tests once again

* Use similar MAVEN_OPTS as there is in Pulsar

* Fix directory

* Add way to find git root directory

* Revisit one more time

* Add current-version-image back

* Enable tests for backwardCompatTests

* Don't create /ledgers in ZK if it already exists

* Add uncaughtexceptionhandler

* Add exclusions for reload4j

* Use maven-surefire 2.8.1 for certain tests

* Add failing test back in, also fails with surefire 2.8.1

* Exclude logging implementations

* Revert "Use maven-surefire 2.8.1 for certain tests"

This reverts commit 94e2b8eb959c8d7559edd13d7ebe2c0637bf8ab9.

* Exclude log4j

* Add instructions how to run on Mac Apple Silicon

* Find slf4j version

* Block loading some classes

* Change context classloader while creating new bookkeeper

* Route commons logging to slf4j

* Update shrinkwrap.version

* Wait for ZK client to shutdown before returning from close

* Upgrade Groovy version

* Remove arquillian-junit-container which causes tests to be run twice

* Skip backward compat tests unless -DbackwardCompatTests is passed

* Delay closing of classloaders by 5 seconds

* Fix checkstyle

* Improve instructions for running tests

* Upgrade Testcontainers and import the bom in dependencyManagement

* Fix synchronization bug pointed out by spotbugs

* upgrade markdown-link-check

* Ignore checking Google calendar for dead links

* Ignore anchors

* Add 2 more patterns to dead link checker exclusions

* Collect logs

* Log integration tests to console to observe progress, disable retries

* Use maven version provided by GitHub Action runner

* Improve instructions for docker socket proxy

* Add test timeout of 5 minutes for BookieShellTestBase

* Fix test to match changes made in e41d6728

* Upgrade mockito to 4.11.0

* Align hamcrest with junit

* Revert "Upgrade mockito to 4.11.0"

This reverts commit 55b7fcd4ca3d83534f2ce328bc35d3a666a4b622.

* Make JAVA_HOME optional

* Tune Java's DNS cache timeouts for docker images

* Use different directory for JDK 8

* Add tooling to containers that help debugging

* Upgrade JMH version and configuration

* Specify maven.compiler.release to fix issues with building with Java 11

* Remove gmavenplus plugin since there's already groovy-eclipse compiler in use

* Upgrade apache-rat-plugin

* Fix validation step

* Don't import shaded class

* Move freebuilder out of distributed libraries (there was invalid config for the annotation processor)

* Remove use of shaded class in test

* Upgrade junit5, assertj-core & awaitility to latest stable

* Fix simpletest logic

- LAC is updated asynchronously. Entries might be unconfirmed when read before ledger is closed.
- The last entry id passed to the read method was invalid

* Fix SimpleTestCommandTest

---------

Co-authored-by: Matteo Merli <mmerli@apache.org>
70 files changed
tree: 31e3337538eb4b9c879d2c1962699c24dcef1478
  1. .github/
  2. .test-infra/
  3. bin/
  4. bookkeeper-benchmark/
  5. bookkeeper-common/
  6. bookkeeper-common-allocator/
  7. bookkeeper-dist/
  8. bookkeeper-http/
  9. bookkeeper-proto/
  10. bookkeeper-server/
  11. bookkeeper-slogger/
  12. buildtools/
  13. circe-checksum/
  14. conf/
  15. cpu-affinity/
  16. deploy/
  17. dev/
  18. docker/
  19. metadata-drivers/
  20. microbenchmarks/
  21. native-io/
  22. shaded/
  23. site3/
  24. src/
  25. stats/
  26. stream/
  27. tests/
  28. testtools/
  29. tools/
  30. .asf.yaml
  31. .dlc.json
  32. .gitignore
  33. CONTRIBUTING.md
  34. LICENSE
  35. NOTICE
  36. pom.xml
  37. README.md
README.md

Maven Central

Apache BookKeeper

Apache BookKeeper is a scalable, fault-tolerant and low latency storage service optimized for append-only workloads.

It is suitable for being used in following scenarios:

  • WAL (Write-Ahead-Logging), e.g. HDFS NameNode, Pravega.
  • Message Store, e.g. Apache Pulsar.
  • Offset/Cursor Store, e.g. Apache Pulsar.
  • Object/Blob Store, e.g. storing state machine snapshots.

Get Started

  • Checkout the project website.
  • Concepts: Start with the basic concepts of Apache BookKeeper. This will help you to fully understand the other parts of the documentation.
  • Follow the Installation guide to set up BookKeeper.

Documentation

Please visit the Documentation from the project website for more information.

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a GitHub issue.

Need Help?

Subscribe or mail the user@bookkeeper.apache.org list - Ask questions, find answers, and also help other users.

Subscribe or mail the dev@bookkeeper.apache.org list - Join development discussions, propose new ideas and connect with contributors.

Join us on Slack - This is the most immediate way to connect with Apache BookKeeper committers and contributors.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See our installation guide to get your local environment setup.

  2. Take a look at our open issues: GitHub Issues.

  3. Review our coding style and follow our pull requests to learn more about our conventions.

  4. Make your changes according to our contributing guide