maven-surefire-plugin/src/site/apt/usage.apt.vm

  ------
  Usage
  ------
  Brett Porter
  Allan Ramirez
  Stephen Connolly
  ------
  2011-06-27
  ------
  
~~ Licensed to the Apache Software Foundation (ASF) under one
~~ or more contributor license agreements.  See the NOTICE file
~~ distributed with this work for additional information
~~ regarding copyright ownership.  The ASF licenses this file
~~ to you under the Apache License, Version 2.0 (the
~~ "License"); you may not use this file except in compliance
~~ with the License.  You may obtain a copy of the License at
~~
~~   http://www.apache.org/licenses/LICENSE-2.0
~~
~~ Unless required by applicable law or agreed to in writing,
~~ software distributed under the License is distributed on an
~~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~~ KIND, either express or implied.  See the License for the
~~ specific language governing permissions and limitations
~~ under the License.

~~ NOTE: For help with the syntax of this file, see:
~~ http://maven.apache.org/doxia/references/apt-format.html

Usage

#{if}(${project.artifactId}=="maven-surefire-plugin")
  Best practice is to define the version of the Surefire Plugin that you want to use in either your <<<pom.xml>>>
  or a parent <<<pom.xml>>>:

+---+
<project>
  [...]
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>${project.groupId}</groupId>
          <artifactId>${project.artifactId}</artifactId>
          <version>${project.version}</version>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  [...]
</project>
+---+

#{else}
  To use the ${thisPlugin} Plugin, you need to add the following configuration to
  your <<<pom.xml>>>:

+---+
<project>
  [...]
  <build>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  [...]
</project>
+---+

#{end}
#{if}(${project.artifactId}=="maven-surefire-plugin")
  The ${thisPlugin} Plugin can be invoked by calling the <<<test>>> phase of the
  build lifecycle.

+---+
mvn test
+---+

#{else}
  The ${thisPlugin} Plugin can be invoked by calling the <<<verify>>> phase of the
  build lifecycle.

+---+
mvn verify
+---+
#{end}

* Using Different Testing Providers

  Tests in your test source directory can be any combination of the following:

   * TestNG

   * JUnit (3.8, 4.x or 5.x)

   * POJO

  Which providers are available is controlled simply by the inclusion of the
  appropriate dependencies (i.e., <<<junit:junit>>> or <<<junit:junit-dep>>> for JUnit4, <<junit-jupiter-engine>> or
  <<junit-vintage-engine>> for JUnit5 and <<<org.testng:testng>>> 4.7+ for TestNG). Since this is required to compile the test
  classes anyway, no additional configuration is required.

  Note that any normal Surefire integration works identically no matter which
  providers are in use - so you can still produce a Cobertura report and a
  Surefire results report on your project web site for your TestNG tests,
  for example.

  The POJO provider above allows you to write tests that do not depend on either of
  JUnit and TestNG. It behaves in the same way, running all <<<test*>>> methods that are
  public in the class, but the API dependency is not required. To perform
  assertions, the JDK 1.4 <<<assert>>> keyword can be used.
  See {{{./examples/pojo-test.html} Using POJO Tests}} for more information.

  All of the providers support the Surefire Plugin parameter configurations.
  However, there are additional options available if you are running TestNG
  tests (including if you are using TestNG to execute your JUnit tests, which
  occurs by default if both are present in Surefire).

  See {{{./examples/testng.html} Using TestNG}} for more information.

#{if}(${project.artifactId}=="maven-failsafe-plugin")
* Using jetty and ${project.artifactId}

  You need to bind one of <<<jetty:start>>>, <<<jetty:run>>>, <<<jetty:run-exploded>>> or <<<jetty:run-war>>>
  to the <<<pre-integration-test>>> phase with <<<daemon>>> set to true, bind
  <<<failsafe:integration-test>>> to the <<<integration-test>>> phase, bind <<<jetty:stop>>>
  to the <<<post-integration-test>>> phase and finally bind <<<failsafe:verify>>> to
  the <<<verify>>> phase.  Here is an example:

+---+
<project>
  [...]
  <build>
    [...]
    <plugins>
      [...]
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
        <executions>
          <execution>
            <id>integration-test</id>
            <goals>
              <goal>integration-test</goal>
            </goals>
          </execution>
          <execution>
            <id>verify</id>
            <goals>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>9.2.2.v20140723</version>
        [...]
        <configuration>
          [...]
          <scanIntervalSeconds>10</scanIntervalSeconds>
          <stopPort>8005</stopPort>
          <stopKey>STOP</stopKey>
          [...]
        </configuration>
        [...]
        <executions>
          [...]
          <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start</goal>
            </goals>
            <configuration>
              <scanIntervalSeconds>0</scanIntervalSeconds>
              <daemon>true</daemon>
            </configuration>
          </execution>
          <execution>
            <id>stop-jetty</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
          [...]
        </executions>
        [...]
      </plugin>
      [...]
    </plugins>
    [...]
  </build>
  [...]
</project>
+---+

  You then invoke Maven with a phase of <<<verify>>> or later in order to run
  the integration tests.  <<DO NOT>> directly invoke any of the phases
  <<<pre-integration-test>>>, <<<integration-test>>>, or <<<post-integration-test>>> as
  these are too long to type and they will likely leave a jetty container running.

+---+
mvn verify
+---+

  Note: during test development, you will likely run a jetty instance in the background.
  to help running the integration tests, it can be handy to bind <<<jetty:stop>>> to
  the <<<pre-integration-test>>> phase before <<<jetty:run>>> to flush out any
  running jetty instance before starting the integration test jetty instance, e.g.

+---+
<project>
  [...]
  <build>
    [...]
    <plugins>
      [...]
      <plugin>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-maven-plugin</artifactId>
        <version>9.2.2.v20140723</version>
        [...]
        <executions>
          [...]
          <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
              <!-- stop any previous instance to free up the port -->
              <goal>stop</goal>
              <goal>start</goal>
            </goals>
            [...]
          </execution>
          [...]
        </executions>
        [...]
      </plugin>
      [...]
    </plugins>
    [...]
  </build>
  [...]
</project>
+---+

* Reporting integration test results

  The ${thisPlugin} Plugin uses the exact same format as the ${thatPlugin} Plugin, so to generate a report you just add a second
  Surefire Report Plugin report set using the ${thisPlugin} reports directory, e.g.

+---+
<project>
  [...]
  <reporting>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>maven-surefire-report-plugin</artifactId>
        <version>${project.version}</version>
        <reportSets>
          <reportSet>
            <id>integration-tests</id>
            <reports>
              <report>failsafe-report-only</report>
            </reports>
          </reportSet>
        </reportSets>
      </plugin>
    </plugins>
  </reporting>
  [...]
</project>
+---+

* Running integration tests multiple times

  If you need to run your integration tests multiple times, just use multiple executions of the <<<integration-test>>>
  goal.  You will need to specify a different summary file for each execution, and then configure the <<<verify>>> goal
  with the multiple summary files in order to fail the build when any one execution has failures, e.g.

+---+
<project>
  [...]
  <reporting>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
        <executions>
          <execution>
            <id>integration-test-red-bevels</id>
            <goals>
              <goal>integration-test</goal>
            </goals>
            <configuration>
              <systemPropertyVariables>
                <bevels>red</bevels>
              </systemPropertyVariables>
              <summaryFile>target/failsafe-reports/failsafe-summary-red-bevels.xml</summaryFile>
            </configuration>
          </execution>
          <execution>
            <id>integration-test-no-bevels</id>
            <goals>
              <goal>integration-test</goal>
            </goals>
            <configuration>
              <systemPropertyVariables>
                <bevels>none</bevels>
              </systemPropertyVariables>
              <summaryFile>target/failsafe-reports/failsafe-summary-no-bevels.xml</summaryFile>
            </configuration>
          </execution>
          <execution>
            <id>verify</id>
            <goals>
              <goal>verify</goal>
            </goals>
            <configuration>
              <summaryFiles>
                <summaryFile>target/failsafe-reports/failsafe-summary-red-bevels.xml</summaryFile>
                <summaryFile>target/failsafe-reports/failsafe-summary-no-bevels.xml</summaryFile>
              </summaryFiles>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </reporting>
  [...]
</project>
+---+

* Usage in multi-module projects

  The recommendations for using the ${thisPlugin} Plugin listed at the top of this page are fine for 95% of use cases.
  When you are defining a shared definition of the ${thisPlugin} Plugin in a parent pom, it is considered best practice
  to define an execution id in order to allow child projects to override the configuration. Thus you might have the
  following in your parent <<<pom.xml>>>:

+---+
<project>
  [...]
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>${project.groupId}</groupId>
          <artifactId>${project.artifactId}</artifactId>
          <version>${project.version}</version>
          <executions>
            <execution>
              <id>integration-test</id>
              <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  [...]
</project>
+---+

  The child projects can then trigger usage of the ${thisPlugin} Plugin with

+---+
<project>
  [...]
  <build>
    <plugins>
      <plugin>
        <groupId>${project.groupId}</groupId>
        <artifactId>${project.artifactId}</artifactId>
        <version>${project.version}</version>
      </plugin>
    </plugins>
  </build>
  [...]
</project>
+---+

  For very complex builds, it may be better to separate the executions for the <<<integration-test>>> and <<<verify>>>
  goals.

+---+
<project>
  [...]
  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>${project.groupId}</groupId>
          <artifactId>${project.artifactId}</artifactId>
          <version>${project.version}</version>
          <executions>
            <execution>
              <id>integration-test</id>
              <goals>
                <goal>integration-test</goal>
              </goals>
            </execution>
            <execution>
              <id>verify</id>
              <goals>
                <goal>verify</goal>
              </goals>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
  [...]
</project>
+---+

#{end}