Microbenchmarks for Apache Pulsar

This module contains microbenchmarks for Apache Pulsar.

Running the benchmarks

The benchmarks are written using JMH. To compile & run the benchmarks, use the following command:

# Compile everything including the shaded microbenchmarks jar
./gradlew :microbench:shadowJar

# run the benchmarks using the standalone shaded jar in any environment
java -jar microbench/build/libs/microbench-*-benchmarks.jar

Running specific benchmarks

Display help:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -h

Listing all benchmarks:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -l

Running specific benchmarks:

java -jar microbench/build/libs/microbench-*-benchmarks.jar ".*BenchmarkName.*"

Running specific benchmarks with machine-readable output and saving the output to a file:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -rf json -rff jmh-result-$(date +%s).json ".*BenchmarkName.*" | tee jmh-result-$(date +%s).txt

The jmh-result-*.json file can be used to visualize the results using JMH Visualizer.

Checking what benchmarks match the pattern:

java -jar microbench/build/libs/microbench-*-benchmarks.jar ".*BenchmarkName.*" -lp

Profiling benchmarks with async-profiler:

Set LIBASYNCPROFILER_PATH to the path of the async-profiler library.

Corretto JDK ships with async-profiler (asprof binary and libasyncProfiler dynamic library)

LIBASYNCPROFILER_PATH=$(ls $JAVA_HOME/lib/libasyncProfiler.*)

Alternatively, download async-profiler from https://github.com/async-profiler/async-profiler/releases and install to ~/async-profiler directory.

Mac OS example:

LIBASYNCPROFILER_PATH=$HOME/async-profiler/lib/libasyncProfiler.dylib

Linux example:

# macos example
LIBASYNCPROFILER_PATH=$HOME/async-profiler/lib/libasyncProfiler.dylib

Then run the benchmarks with the -prof argument:

java -jar microbench/build/libs/microbench-*-benchmarks.jar -prof async:libPath=$LIBASYNCPROFILER_PATH\;output=flamegraph\;dir=profile-results ".*BenchmarkName.*"

When profiling on Mac OS, you might need to add \;event=itimer to the -prof argument since it's the only async profiler CPU sampling engine that supports Mac OS. The default value for event is cpu.

It‘s possible to add options to the async-profiler that aren’t supported by the JMH async-profiler plugin. This can be done by adding rawCommand option to the -prof argument. This example shows how to add all (new in Async Profiler 4.1), jfrsync (record JFR events such as garbage collection) and cstack=vmx options.

java -jar microbench/build/libs/microbench-*-benchmarks.jar -prof async:libPath=$LIBASYNCPROFILER_PATH\;output=jfr\;dir=profile-results\;rawCommand=all,jfrsync,cstack=vmx ".*BenchmarkName.*"