| Testing |
| ======= |
| |
| Examples below assume cwd at the gradlew script in the top directory of |
| the project's checkout. |
| |
| |
| Generic test/ checkup commands |
| ------------------------------ |
| |
| Run all unit tests: |
| |
| gradlew test |
| |
| Run all verification tasks, including tests: |
| |
| gradlew check |
| |
| Run all verification tasks, excluding tests (-x is gradle's generic task |
| exclusion mechanism): |
| |
| gradlew check -x test |
| |
| Run verification for a selected module only: |
| |
| gradlew :lucene:core:check # By full gradle project path |
| gradlew -p lucene/core check # By folder designation + task |
| |
| |
| Randomization |
| ------------- |
| |
| To run tests with the given starting seed pass 'tests.seed' |
| property: |
| |
| gradlew :lucene:misc:test -Ptests.seed=DEADBEEF |
| |
| There are a lot of other test randomization properties |
| available. To list them, their defaults and current values |
| run the testOpts task against a project that has tests. |
| For example: |
| |
| gradlew -p lucene/core testOpts |
| |
| |
| Filtering |
| --------- |
| |
| Run tests of lucene-core module: |
| |
| gradlew -p lucene/core test |
| |
| Run a single test case (from a single module). Uses gradle's built-in filtering |
| (https://docs.gradle.org/current/userguide/java_testing.html#test_filtering): |
| |
| gradlew -p lucene/core test --tests TestDemo |
| |
| Run all tests in a package: |
| |
| gradlew -p lucene/core test --tests "org.apache.lucene.document.*" |
| |
| Run all test classes/ methods that match this pattern: |
| |
| gradlew -p lucene/core test --tests "*testFeatureMissing*" |
| |
| |
| Test groups |
| ----------- |
| |
| Tests can be filtered by an annotation they're marked with. |
| Some test group annotations include: @AwaitsFix, @Nightly, @Slow |
| |
| This uses filtering infrastructure on the *runner* (randomizedtesting), |
| not gradle's built-in mechanisms (but it can be combined with "--tests"). |
| For example, run all lucene-core tests annotated as @Slow: |
| |
| gradlew -p lucene/core test -Ptests.filter=@Slow |
| |
| Test group filters can be combined into Boolean expressions: |
| |
| gradlew -p lucene/core test "default and not(@awaitsfix or @slow)" |
| |
| |
| Reiteration ("beasting") |
| ------------------------ |
| |
| Multiply each test case N times (this works by repeating the same test |
| within the same JVM; it also works in IDEs): |
| |
| gradlew -p lucene/core test --tests TestDemo -Ptests.iters=5 |
| |
| Tests tasks will be (by default) re-executed on each invocation because |
| we pick a random global tests.seed. If you run the same tests twice |
| with the same seed, the test task will be skipped (as it is up-to-date |
| with respect to source code): |
| |
| gradlew -p lucene/core test -Ptests.seed=deadbeef |
| |
| to force re-execution of tests, even for the same master seed, apply |
| cleanTest task: |
| |
| gradlew -p lucene/core cleanTest test -Ptests.seed=deadbeef |
| |
| The 'tests.iters' option should be sufficient for individual test cases |
| and is *much* faster than trying to duplicate re-runs of the entire |
| test suites. When it is absolutely needed to re-run an entire suite (because |
| of randomization in the static initialization, for example), you can do it |
| by running the 'beast' task with 'tests.dups' option: |
| |
| gradlew -p lucene/core beast -Ptests.dups=10 --tests TestPerFieldDocValuesFormat |
| |
| Note the filter (--tests) used to narrow down test reiterations to a particular |
| class. You can use any filter, including no filter at all, but it rarely makes |
| sense (will take ages). By default the test tasks generated by the 'beast' mode |
| use a random starting seed for randomization. If you pass an explicit seed, this |
| won't be the case (all tasks will use exactly the same starting seed): |
| |
| gradlew -p lucene/core beast -Ptests.dups=10 --tests TestPerFieldDocValuesFormat -Dtests.seed=deadbeef |
| |
| Verbose mode and debugging |
| -------------------------- |
| |
| The "tests.verbose" mode switch enables standard streams from tests |
| to be dumped directly to the console. Run your verbose tests explicitly |
| specifying the project and test task or a fully qualified task path. Example: |
| |
| gradlew -p lucene/core test -Ptests.verbose=true --tests "TestDemo" |
| |
| |
| Profiling slow tests |
| -------------------- |
| |
| The "tests.profile" mode switch turns on a sampling profiler during test execution, |
| and prints a simple summary at the end. |
| |
| For example, top-10 histogram of methods (cpu samples): |
| |
| gradlew -p lucene/core test -Ptests.profile=true |
| |
| Alternatively, you can profile heap allocations instead: |
| |
| gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.mode=heap |
| |
| By default, results are computed (deduplicated) on just the method name, folding |
| together all events from the same method. To drill down further, you can increase the |
| stack size from the default of 1, to get a histogram of stacktraces instead: |
| |
| gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.stacksize=8 |
| |
| For big methods, it can also be helpful to include line numbers for more granularity: |
| |
| gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.linenumbers=true |
| |
| Using these additional options will make the results more sparse, so it may be useful |
| to increase the top-N count: |
| |
| gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.count=100 |