| Running the tests |
| ----------------- |
| |
| - All tests reside in the tests/ directory tree. |
| |
| - Running the tests is as simple as: |
| $ make check |
| |
| - Alternatively, the test scripts can be run directly: |
| $ cd tests |
| $ ./run_tests.py |
| |
| - To run the only the tests associated with language X ("cpp", "java", etc.): |
| $ cd tests |
| $ ./run_tests.py X |
| |
| Adding new tests |
| ---------------- |
| |
| - A test is defined by adding a line in one of the tests/*.test |
| files: you must pick the .test file for the proper programming |
| language, i.e. pick the X.test file which defines tests which use |
| inputs in language X. |
| |
| - A test has a (unique) number (a.k.a. test name) -- it does not have |
| to be a sequential number, but it might be handy to treat it like |
| that. |
| |
| - The X.test entry (line) has this format: |
| |
| <test.name> <config.file> <input.file> |
| |
| where the fields are separated by an arbitrary non-zero amount of |
| whitespace. |
| |
| - As each test references a config file and an input file (the latter |
| written in programming language X), these are assumed to exist in |
| the paths tests/config/<config.file> and tests/<input.file> |
| respectively. It is _strongly_ advised to put input files in their |
| own subdirectory, so a 'C' source formatting test entry might look |
| like this (in 'c.test'): |
| |
| 99902 ger-full-cmt-reflow.cfg c/cmt-not-a-boxed-cmt.c |
| |
| where 'ger-full-cmt-reflow.cfg' is stored at |
| tests/config/ger-full-cmt-reflow.cfg and 'c/cmt-not-a-boxed-cmt.c' |
| will be fetched from tests/c/cmt-not-a-boxed-cmt.c when you run |
| the tests. |
| |
| |
| Features |
| -------- |
| |
| - You can re-use config files and input files for multiple tests. So |
| one might observe test sets like these: |
| |
| # clark's style - blank lines before and after flow control, |
| indented comments: |
| |
| 20100 clark.cfg c/i2c-core.c |
| 20101 clark.cfg c/comment-indent.c |
| 20102 clark.cfg c/indent.cpp |
| 20103 clark.cfg c/output.cpp |
| |
| and |
| |
| 21051 return-1.cfg c/nl_return_expr.c |
| 21052 return-2.cfg c/nl_return_expr.c |
| 21053 return-3.cfg c/nl_return_expr.c |
| 21054 return-4.cfg c/nl_return_expr.c |
| |
| - Reference output files (the reference against which the uncrustify |
| test run output is compared) are to be stored in the path |
| tests/output/<X>/<NNN-TTTTT.TTT> where <X> is the 'language |
| directory' part of the input file, e.g. 'c' for input file |
| 'c/nl_return_expr.c', <NNN> is the test name (number), e.g. '21051', |
| and <TTTTT.TTT> is the file name part of the input test filespec, |
| e.g. 'nl_return_expr.c' for input file spec 'c/nl_return_expr.c'. |
| |
| This means that for test |
| |
| 21051 return-1.cfg c/nl_return_expr.c |
| |
| the accompanying reference output is: |
| |
| tests/output/c/21051-nl_return_expr.c |
| |
| - When starting out with a new test, you don't need to have a |
| 'reference output' yet: the test will simply be reported as a |
| 'fail'ed test until you do. |
| |
| - Tip: the easiest way to produce 'reference output' is to copy the |
| test output (from tests/results/...etc... ) to |
| tests/output/...etc... once you've ascertained that those tests |
| produce the desired (correct) output. The helper script |
| tests/fixtest.sh will copy the results file to the output folder: |
| |
| $ fixtest.sh 30014 00110 # copy tests 30014 and 00110 |
| |
| - Tip: the helper script tests/difftest.sh runs 'diff' on the result |
| and output versions to point out what changed. |
| |
| - Tip: There are also a few options to run_tests.py that can |
| help. Most interesting is '-d' will run 'diff' if the test fails. |
| |