test.py

Builds Juneau, runs the Maven tests, then checks timings, runs the scripts' own pytest suite, and optionally the browser tests.

When to use

  • You want the standard local verification: a clean build plus the full test run.
  • You want only the build (--build-only) or only the tests (--test-only).
  • You want to profile one module's tests (--profile) or refresh the timing baselines (--rebaseline).
  • push.py runs this script for its build and test steps.

Usage

python3 scripts/test.py
python3 scripts/test.py --test-only --no-container
python3 scripts/test.py --build-only
python3 scripts/test.py --enforce-perf
python3 scripts/test.py --console board --warn-traces
python3 scripts/test.py --profile juneau-core/juneau-marshall
python3 scripts/test.py --rebaseline --window 20 --min-runs 5

Options

OptionArgumentDefaultMeaning
--build-only, -bnoneoffOnly build; skip the tests.
--test-only, -tnoneoffOnly run the tests; no build. Cannot be combined with --build-only.
--full, -fnoneon when no other mode is givenClean build, then the tests.
--verbose, -vnoneoffAccepted for compatibility. Maven output always streams live.
--no-containernoneoffExclude tests tagged container (-DexcludedGroups=container).
--timing-logpathnoneIgnored, with a warning. Timings are in the run history; the option goes away in a later release.
--enforce-perfnoneoffFail on a perf-guard breach instead of only warning.
--rebaselinenoneoffRewrite the suite line and [modules] section of perf-baseline.txt from the run history, then exit without building.
--windowN (integer)20With --rebaseline: runs per key whose median is taken. A module whose window spreads by more than 50% takes the 75th percentile instead.
--min-runsN (integer)5With --rebaseline: skip keys with fewer runs.
--js-testsnoneautoAlso run the headless-browser tests (mvn -Pjs-tests). Fails if Node or npm are missing.
--no-js-testsnoneoffNever run the browser tests. Cannot be combined with --js-tests.
--warn-tracesnoneoffReport stack traces in the test output as a warning instead of failing the Traces row. Allowlist errors still fail.
--consolefull, board or nonefullHow Maven output appears on the console.
--detailsummary, actionable, modules or allactionableConsole detail level.
--full-logpathnoneAlso append everything Maven prints to this file.
--board-logpathnoneAlso append the board one-line-per-event stream to this file.
--profilemodule pathnoneRun one module's tests under a one-shot JFR profile, then exit.
--help, -hnoneoffPrint the usage text and exit 0.

What it does

  1. Parses the options. Unknown options print the usage text and exit 1 (_main).
  2. Build: mvn clean install -T1C -DskipTests (build).
  3. Tests: mvn test -T1C -Drat.skip=true, plus -DexcludedGroups=container with --no-container (test). Per-module Surefire times and Maven's reactor times are recorded in the run history (_record_buckets).
  4. Perf guard: compares the suite wall clock with the suite line of perf-baseline.txt, and each module's reactor time with its [modules] entry. The report is written to target/perf-guard.txt (run_perf_guard).
  5. Python tests: runs scripts/tests with python3 -m pytest, or uv run --native-tls --with pytest --with pyyaml pytest when pytest is not importable (script_tests).
  6. Browser tests: with -Pjs-tests in the views, petstore-core and terminal modules (maybe_run_js_tests). Without --js-tests or --no-js-tests, they run automatically when a .js, .cjs, .css or .ftl file under a src/ tree differs from origin/master.
  7. Traces: scans what steps 3 to 6 printed (read back from the run's full log, from a mark taken before the build) for stack traces, and fails on any whose header line is not matched by trace-allowlist.txt (run_trace_guard). With the default --console full there is no full log, so the row is skipped with a warning; use --console board. A full log that cannot be read fails the row. The report of every trace, allowed or not, is target/trace-guard.txt.
  8. --profile runs mvn test -Drat.skip=true -Djacoco.skip=true from the module‘s directory, with the JFR flags (including stackdepth=1024) appended to the module’s own -Dsurefire.extraArgs. It writes one hotspot-pid-<pid>-id-<n>-<time>.jfr per forked JVM under <repo>/target/profile-results/<module>-<timestamp>/, and fails if Maven passes but no file exists (profile). Read a recording with jfr print --json --stack-depth 300 <file>; the default depth of 5 hides every Juneau frame.

Output and exit codes

Maven output, a status line per step, and the perf-guard report. The run header and markers come from the shared run toolkit. A standalone run rings the terminal bell at the end; JUNEAU_PUSH_NO_SOUND=1 silences it.

CodeMeaning
0Every requested step passed.
1Conflicting or unknown options, a failed build or test run, or --rebaseline with history turned off.
2An invalid --detail value.
130Cancelled with Ctrl-C or SIGTERM.
otherThe exit code of the failing Maven, perf-guard or pytest step.

Gotchas

  • The perf guard only warns unless --enforce-perf is given. A suite breaches above baseline times 1 + JUNEAU_CI_PERF_THRESHOLD (default 0.20). A module breaches only when it is over that and more than JUNEAU_CI_PERF_MIN_SECONDS (default 3) over its baseline. A module with no baseline entry warns and never fails.
  • JUNEAU_MVN_WRAPPER adds a prefix to every mvn command, for example a lock script that serializes runs. The value is a command plus arguments, not a shell fragment: it is split like a shell would (so '/a b/lock.sh' --x works), then re-quoted word by word, so a path with spaces or $/; is never interpreted. An unquoted path with spaces also works, taken as the longest leading run of words that names an existing file. A word that is exactly &&, ||, ; or | stays a shell operator, so lock.sh --x && still chains.
  • scripts/trace-allowlist.txt holds one header regex per line followed by # <reason> (the reason is everything after the first #, so it may hold # itself); a trace joined from two JavaScript errors in one console block is allowed only when every one of its headers is matched; a line with no reason, a bad regex or a duplicate regex fails the Traces row even with --warn-traces. An entry that matched nothing is reported as stale (a warning) only when the browser tests ran and neither --no-container nor --no-js-tests was given.
  • The pytest step is skipped with a notice when neither pytest nor uv is available. Nothing is installed.
  • The browser tests download Playwright and Chromium on the first run, so they need network access.
  • JUNEAU_TERMINAL_HISTORY=0 turns off the run history, which --rebaseline and push-timings.py depend on.
  • --profile skips JaCoCo with -Djacoco.skip=true, because its prepare-agent goal rewrites argLine after any -DargLine. Upstream modules are not built or tested: the run starts in the module directory, not from the root with -pl.
  • The default mode runs mvn clean install, which replaces the Juneau artifacts in ~/.m2.

See also

  • push.py, which runs this script
  • push-timings.py, which reads the run history
  • perf-required.py
  • coverage.py