XmlPlexusConfiguration Performance Benchmarks

This directory contains JMH (Java Microbenchmark Harness) benchmarks to measure the performance improvements in the optimized XmlPlexusConfiguration implementation.

Overview

The benchmarks compare the old implementation (XmlPlexusConfigurationOld) with the new optimized implementation (XmlPlexusConfiguration) across several key performance metrics:

  1. Constructor Performance - Measures the impact of eliminating deep copying
  2. Memory Allocation - Compares memory usage patterns and allocation rates
  3. Thread Safety - Tests concurrent access performance and safety
  4. Lazy vs Eager Loading - Measures the benefits of lazy child creation

Benchmark Classes

1. XmlPlexusConfigurationBenchmark

  • Purpose: Core performance comparison between old and new implementations
  • Metrics: Constructor speed, child access performance, memory allocation
  • Test Cases: Simple, complex, and deep XML structures

2. XmlPlexusConfigurationConcurrencyBenchmark

  • Purpose: Thread safety and concurrent performance testing
  • Metrics: Throughput under concurrent load, race condition detection
  • Test Cases: Multi-threaded child access, concurrent construction

3. XmlPlexusConfigurationMemoryBenchmark

  • Purpose: Memory efficiency and garbage collection impact
  • Metrics: Allocation rates, memory sharing vs copying
  • Test Cases: Small, medium, and large XML documents

Running the Benchmarks

Prerequisites

  • Java 11 or higher
  • Maven 3.6 or higher
  • At least 2GB of available memory

Quick Start

  1. Compile the test classes:

    mvn test-compile -pl impl/maven-xml

  2. Run all benchmarks:

    mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="org.apache.maven.internal.xml.*Benchmark" \
    -pl impl/maven-xml

Running Specific Benchmarks

Constructor Performance:

mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="XmlPlexusConfigurationBenchmark.constructor.*" \
    -pl impl/maven-xml

Memory Allocation:

mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="XmlPlexusConfigurationMemoryBenchmark" \
    -pl impl/maven-xml

Thread Safety:

mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="XmlPlexusConfigurationConcurrencyBenchmark" \
    -pl impl/maven-xml

Advanced Options

Generate detailed reports:

mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="-rf json -rff benchmark-results.json org.apache.maven.internal.xml.*Benchmark" \
    -pl impl/maven-xml

Profile memory allocation:

mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="-prof gc XmlPlexusConfigurationMemoryBenchmark" \
    -pl impl/maven-xml

Profile with async profiler (if available):

mvn test-compile exec:java -Dexec.mainClass="org.openjdk.jmh.Main" \
    -Dexec.classpathScope=test \
    -Dexec.args="-prof async:output=flamegraph XmlPlexusConfigurationBenchmark" \
    -pl impl/maven-xml

Expected Results

Based on the optimizations implemented, you should see:

Constructor Performance

  • 50-80% faster initialization for complex XML structures
  • Dramatic improvement for deep XML hierarchies due to eliminated deep copying

Memory Usage

  • 60-80% reduction in memory allocation for typical XML documents
  • Linear scaling instead of exponential growth with document complexity

Thread Safety

  • Zero race conditions in the new implementation
  • Consistent performance under concurrent load
  • No infinite loops or exceptions during parallel access

Lazy Loading Benefits

  • Faster startup when not all children are accessed
  • Lower memory footprint for partially used configurations
  • Better scalability for large XML documents

Actual Benchmark Results

Here are real performance measurements from the benchmark suite:

Constructor Performance (Simple XML)

Benchmark                                             Mode  Cnt   Score    Error  Units
XmlPlexusConfigurationBenchmark.constructorNewSimple  avgt    3   4.666 ±  7.721  ns/op
XmlPlexusConfigurationBenchmark.constructorOldSimple  avgt    3  41.361 ± 14.438  ns/op

Result: 8.9x faster (88% improvement)

Constructor Performance (Complex XML)

Benchmark                                              Mode  Cnt    Score    Error  Units
XmlPlexusConfigurationBenchmark.constructorNewComplex  avgt    3    4.887 ± 15.716  ns/op
XmlPlexusConfigurationBenchmark.constructorOldComplex  avgt    3  657.163 ± 94.225  ns/op

Result: 134x faster (99.3% improvement)

These results demonstrate the dramatic performance benefits of the optimization, especially for complex XML structures where the old implementation's deep copying becomes extremely expensive.

Interpreting Results

  • Lower numbers are better for average time benchmarks
  • Higher numbers are better for throughput benchmarks
  • Error margins indicate measurement confidence
  • GC profiling shows allocation reduction in the new implementation

Troubleshooting

Out of Memory Errors:

  • Increase heap size: -Dexec.args="-jvmArgs -Xmx4g"
  • Reduce benchmark iterations: -Dexec.args="-wi 1 -i 3"

Long Execution Times:

  • Run specific benchmarks instead of all
  • Reduce warmup and measurement iterations
  • Use shorter time periods: -Dexec.args="-w 1s -r 1s"

Contributing

When adding new benchmarks:

  1. Follow the existing naming convention
  2. Include both old and new implementation tests
  3. Add appropriate JMH annotations
  4. Document the benchmark purpose and expected results
  5. Update this README with new benchmark information