src/test/regression/README.txt

               Directions for Regression Test Developers
               =========================================

    This file describes conventions accepted for VM regression test suite
    and steps needed to integrate new tests into the suite.
    ---------------------------------------------------------------------


Test Format and Test Naming conventions
---------------------------------------

 1. The tests are supposed to be in JUnit test format

 2. The source files of each regression test should be placed in a directory
    named H{NUMBER}, where {NUMBER} (here and below) is an ID
    of JIRA issue the test is related to.
        For example: H1234

 3. The name of the main class should end with Test suffix
        For example: SomethingTest.java

 4. The package of the test should be
    org.apache.harmony.drlvm.tests.regression.h{NUMBER}
       For example: org.apache.harmony.drlvm.tests.regression.h1234

 5. The source files for the tests can be in Java, Java Assembler, and
    C/C++ programming languages.


New Test Integration
--------------------

To integrate new test in the regression test suite:

 1. Make sure it complies with conventions described above.

 2. Put the test sources into H{NUMBER} directory.

 3. By default test execution entry point is a class ending with Test
    suffix. It will be automatically launched by regression test
    infrastructure as a JUnit test. If there are several classes ending
    with Test suffix all of them will be launched.

After that regression test infrastructure will be able to compile and
execute your test.


Custom Test Launchers
---------------------

If there is a need in some more sophisticated test launching mechanism
(JVM parameters or environment variables need to be specified in order to
reproduce the problem, etc.) it can be done by using CUSTOM LAUNCHER.
It is an Ant build file named run.test.xml and placed near the test
sources. The target named "run-test" should specify the way in which the
test is launched. This Ant's file is executed in context of DRLVM build
system and inherits all the properties, references, and macrodefinitions
defined there. The custom launcher is started from
            build/make/targets/reg.test.run.xml
file which contains some useful macrodefinitions to simplify custom
launchers.

 Note:  If there are tests with Test suffix along with the custom launcher
 -----  they won't be automatically launched by regression testing
        infrastructure.


Some useful properties for custom launchers:
-------------------------------------------

    ${reg.test.class.path} - the path to where java and
        java assembler sources were compiled

    ${reg.test.2launch.native.path} - the path to where native
        sources were compiled


Examples
--------
The best way to learn something is to use examples. Please, use existing 
regression tests for this purpose. The following tests can act as a starting 
points:

    H1694 - Simple JUnit test automatically compiled and 
            launched by regression testing framework.

    H2151 - The test consisting of java and native code.
            It uses custom launcher for execution.

    H788  - The test consisting of java and java assembler
            sources. Source files are automatically compiled by
            regression testing framework. The entry point for the test
            is JUnit test implemented by Test.java source file.


Test Exclusion
--------------

If there is a serious reason to exclude some test from regression testing
it can be done by means of exclude lists placed under 'excludes' directory.
To exclude the test, say H1234, from regression testing on operating
system OS running on top of platform PLATFORM just put line 'H1234'
(without quotes) in file exclude.OS.PLATFORM . That's all. The possible
values for OS are linux and windows, for PLATFORM are x86 and x86_64 .


Test Execution
--------------
To execute the Regression Test Suite type

    > ant reg.test

in top level directory of DRLVM workspace. This will execute all non-excluded
regression tests for your platform configuration.

Also by providing the value for 'test.case' property you can choose 
which particular regression tests to execute. 
For example, to run only H1234 regression test type the following 
on your command line:

    > ant -Dtest.case=H1234 reg.test

To check for several regressions use coma or space separated list as a value for
the property. So the following command:

    > ant -Dtest.case="H1234,H4321" reg.test

will execute two regression tests - H1234 and H4321.
 
 Note:  If some of the tests selected by test.case property are in
 -----  exclude list, they will be executed anyway. So you can use this property
        to go around exclude lists.