<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html>
    <head>
        <!-- -*- xhtml -*- -->
        <title>NetBeans Platform Test Infrastructure for NetBeans Platform 7.0</title>
        <meta name="AUDIENCE" content="NBUSER"/>
        <meta name="TYPE" content="ARTICLE"/>
        <meta name="EXPIRES" content="N"/>
        <meta name="developer" content="geertjan.wielenga@sun.com"/>
        <meta name="indexed" content="y"/>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
        <meta name="description"
              content="A guide describing how to test NetBeans Platform applications."/>
        <link rel="stylesheet" type="text/css" href="https://netbeans.org/netbeans.css"/>
    </head>

    <!--      Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved. -->
    <!--     Use is subject to license terms.-->

    <body>
        <h1><a name="top"></a>NetBeans Platform Test Infrastructure Tutorial</h1>

        <p>During development of NetBeans Platform 6.5, an effort was
            made to improve the testing infrastructure
            provided for NetBeans Platform applications.
            Prior to that, many small magical build scripts and other
            configuration files were needed when setting up the test
            infrastructure for NetBeans Platform applications.
            Since then, however, there is inherent support for
            testing NetBeans Platform application within the
            NetBeans Platform's build harness scripts.
            Therefore, unit and functional tests for NetBeans Platform
            applications are now supported out of the box.
            This simplification of the testing infrastructure is
            sometimes referred to as "simpletests", since
            it greatly simplifies the work necessary to set
            up such tests for your application.</p>

        <p>In this tutorial, you are introduced to setting up
            the "simpletests" infrastructure and to using
            the NetBeans Platform's unit testing and
            functional testing frameworks. These are:</p>

        <ul>
            <li><b><a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/overview-summary.html">NB Junit.</a></b> NetBeans Platform extension to the <a href="http://www.junit.org/">JUnit</a> testing framework.</li>
            <li><b><a href="http://www.java2s.com/Open-Source/Java-Document/IDE-Netbeans/jellytools/org.netbeans.jellytools.htm">Jelly Tools.</a></b> NetBeans Platform extension to the <a href="https://jemmy.dev.java.net/">Jemmy</a> testing framework.</li>
        </ul>

        <p>By the end of this tutorial, you should know how
            to set up the NetBeans Platform testing infrastructure,
            how to create unit tests, and how to create functional
            tests.</p>

        <p><b>Contents</b></p>

        <p><img src="../../images/articles/70/netbeans-stamp.gif" class="stamp" width="114" height="114" alt="Content on this page applies to NetBeans IDE 7.0" title="Content on this page applies to NetBeans IDE 7.0"/></p>
        <ul class="toc">
            <li><a href="#setup">Setting Up the Testing Infrastructure</a></li>
            <li><a href="#unit">Unit Testing on the NetBeans Platform</a></li>
            <li><a href="#functional">Functional Testing on the NetBeans Platform</a></li>
            <li><a href="#coverage">Code Coverage on the NetBeans Platform</a></li>
        </ul>

        <p><b>To follow this tutorial, you need the software and resources listed in the following
                table.</b></p>

        <table>
            <tbody>
                <tr>
                    <th class="tblheader" scope="col">Software or Resource</th>
                    <th class="tblheader" scope="col">Version Required</th>
                </tr>
                <tr>
                    <td class="tbltd1"><a href="https://netbeans.org/downloads/index.html">NetBeans IDE</a></td>
                    <td class="tbltd1">version 7.0 or above</td>
                </tr>
                <tr>
                    <td class="tbltd1"><a href="http://java.sun.com/javase/downloads/index.jsp">Java Developer Kit (JDK)</a></td>
                    <td class="tbltd1">version 6 or above</td>
                </tr>
            </tbody>
        </table>

        <!-- ===================================================================================== -->

        <h2 class="tutorial"><a name="setup"></a>Setting Up the Testing Infrastructure</h2>

        <p>When setting up the testing frameworks, you need
            to enable certain modules that are disabled by
            default. Then you need to create folders and files
            in your source structure, where the libraries and
            tests will live.</p>

        <p class="tips">If you want to try out these instructions on an actual
            application prior to trying them out on your own
            sources, you can use the NetBeans Platform Paint
            Application, which you can get from the Samples
            category in the New Project wizard (Ctrl-Shift-N).</p>

        <ol>
            <li>In the Projects window, right-click your application and choose Properties.
                In the Project Properties dialog, click "Libraries".</li>
            <li><p>Check the "harness" box, adding the entire harness cluster,
                    which provides all the testing modules provided by
                    the NetBeans Platform:</p>
                <p><img alt="enable harness" src="../../images/tutorials/test/70/harness-cluster-2.png"/></p>
            </li>
            <li><p>Click the "Resolve" button, shown above, which will add
                    modules from the "platform" cluster
                    to your application, as needed
                    by the modules in the "harness" cluster. Notice
                    that there are now no messages about excluded
                    modules and that the Resolve button has
                    disappeared:</p>
                <p><img alt="enable harness" src="../../images/tutorials/test/70/harness-cluster-3.png"/></p>
            </li>
        </ol>

        <p>Now you must set up a source structure for unit testing
            and functional testing in the modules that need to
            be tested. In other words,
            the steps below do not apply to the application's source
            structure, but to the modules that you need to test.</p>
        <ol>
            <li>Switch to the Files window (Ctrl-2), and expand the module's main node.</li>
            <li>If it does not exist, create a new folder named "test", within the
                module's main folder.</li>
            <li><p>Within the "test" folder, create a folder named "qa-functional".
                    Underneath "qa-functional", create a folder named "src". You can
                    use the New Folder wizard for these purposes, as follows:
                </p>
                <p><img alt="create folders" src="../../images/tutorials/test/create-folder.png"/></p>
            </li>
            <li>Within the "test" folder, create a folder named "unit".
                Underneath "unit", create a folder named "src".
            </li>
            <li><p>Check that the Files window shows the "test" folder
                    structure as shown below:</p>
                <p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/test-folders.png"/></p>
            </li>
            <li>Restart the IDE.</li>
            <li><p>In the Projects window notice
                    that there are now two new nodes for
                    your test packages and two new nodes
                    for adding the test libraries
                    to the classpath of the module:</p>
                <p><img style="border: 1px solid" alt="new nodes" src="../../images/tutorials/test/new-nodes.png"/></p>
            </li>
            <li><p>Right-click the "Unit Test Libraries" node
                and choose "Add Missing Test Dependencies". Then add
                "JUnit 4" and "NB JUnit". Right-click the "Functional Test Libraries" node
                and choose "Add Missing Test Dependencies".
                Then add "JUnit 4", "NB JUnit", "Jemmy", and
                "Jelly Tools Platform".</p>

                <p class="tips">If you use "Add Missing Test Dependencies"
                 instead of "Add Unit Test Dependency"
                 and "Add Functional Test Dependency",
                 NB JUnit's recursive dependencies are properly
                 configured. Otherwise <a href="http://performance.netbeans.org/insane/index.html">INSANE</a> will not be
                 available, which can cause linkage errors when
                 running tests.</p>

            </li>
            <li><p>Check that the Projects window shows the
                    test library dependencies as shown below:</p>
                <p><img style="border: 1px solid" alt="test libs" src="../../images/tutorials/test/new-libs-in-nodes.png"/></p>
            </li>
        </ol>

        <p>You have now set up everything needed for creating
            unit tests and functional tests on the NetBeans Platform.</p>

        <!-- ======================================================================================== -->

        <h2><a name="unit"></a>Unit Testing on the NetBeans Platform</h2>

        <p>The NetBeans Platform's extension to JUnit is named "<a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/overview-summary.html">NB JUnit</a>".
            It is a separate library with additional support for memory
            leak tests, deadlock tests, and extended use of logging.
            More info and motivation
            can be found
            at <a href="http://openide.netbeans.org/tutorial/test-patterns.html">Test Patterns in Java</a>.
            The basic test class for NB JUnit is <a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/org/netbeans/junit/NbTestCase.html">NbTestCase</a>.</p>

        <p>An example unit test for the NetBeans Platform Paint sample:</p>

        <pre class="examplecode">import org.netbeans.junit.*;
import org.netbeans.paint.PaintCanvas;

public class PaintCanvasTest extends <a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/org/netbeans/junit/NbTestCase.html">NbTestCase</a> {

    public PaintCanvasTest(java.lang.String testName) {
        super(testName);
    }

    public void testSetDiam() {
        PaintCanvas paintCanvas = new PaintCanvas();
        paintCanvas.setBrush(10);
        assertEquals("Brush diameter should be set.", 10, paintCanvas.getBrushDiameter());
    }

}</pre>

        <p>Create the class above in a package in the
            Unit Test Packages node in the Projects window.
            Then right-click the "Paint" project node
            and choose "Test". If the test is successful,
            you will see this:</p>

<p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/result-1.png"/></p>
<p>If the test fails, you will see this:</p>
<p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/result-2.png"/></p>
<p>In the Files window (Ctrl-2), you can find the test results:</p>
<p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/result-3.png"/></p>

        <p>Typical questions relating to unit testing on the NetBeans Platform
             relate to specific NetBeans API objects and the central registry
             (also known as "System FileSystem"). For information on how to
             test these parts of your application, see the following resources:</p>
        <ul>
            <li><a href="http://openide.netbeans.org/tutorial/test-patterns.html">Typical Test Patterns</a></li>
            <li><a href="http://wiki.netbeans.org/TestingThingsThatUseFileObjectDataObjectDataFolder">Writing Tests with FileObjects, DataObjects, and DataFolders</a></li>
            <li><a href="http://wiki.netbeans.org/DevFaqTestDataObject">Writing Tests with DataObjects and DataLoaders</a></li>
            <li><a href="http://wiki.netbeans.org/InitializationOfDefaultLookup">Writing Tests with Lookup</a></li>
            <li><a href="http://wiki.netbeans.org/DevFaqTestUsingSystemFileSystem" title="DevFaqTestUsingSystemFileSystem">How do I test something which uses the System Filesystem?</a></li>
        </ul>

        <!-- ======================================================================================== -->

        <h2><a name="functional"></a>Functional Testing on the NetBeans Platform</h2>

        <p>The NetBeans Platform's extension to Jemmy is named
             <a href="http://www.java2s.com/Open-Source/Java-Document/IDE-Netbeans/jellytools/org.netbeans.jellytools.htm">Jelly</a>.
             It provides a set of operators that are tailored to UI
             components used specifically in the NetBeans Platform,
             such as <tt>TopComponentOperator</tt>.</p>

        <p>An example functional test for the NetBeans Platform Paint sample.</p>

        <pre class="examplecode">import junit.framework.Test;
import org.netbeans.jellytools.JellyTestCase;
import org.netbeans.jellytools.MainWindowOperator;
import org.netbeans.jellytools.TopComponentOperator;
import org.netbeans.jellytools.actions.Action;
import org.netbeans.jemmy.operators.JButtonOperator;
import org.netbeans.jemmy.operators.JSliderOperator;
import org.netbeans.junit.NbModuleSuite;
import org.netbeans.junit.NbModuleSuite.Configuration;

/**
 * A Test based on JellyTestCase. JellyTestCase redirects Jemmy output
 * to a log file provided by NbTestCase. It can be inspected in results.
 * It also sets timeouts necessary for NetBeans GUI testing.
 *
 * Any JemmyException (which is normally thrown as a result of an unsuccessful
 * operation in Jemmy) going from a test is treated by JellyTestCase as a test
 * failure; any other exception - as a test error.
 *
 * Additionally it:
 *    - closes all modal dialogs at the end of the test case (property jemmy.close.modal - default true)
 *    - generates component dump (XML file containing components information) in case of test failure (property jemmy.screen.xmldump - default false)
 *    - captures screen into a PNG file in case of test failure (property jemmy.screen.capture - default true)
 *    - waits at least 1000 ms between test cases (property jelly.wait.no.event - default true)
 *
 */

public class OverallTest extends JellyTestCase {

    /** Constructor required by JUnit */
    public OverallTest(String name) {
        super(name);
    }

    /** Creates suite from particular test cases. You can define order of testcases here. */
    public static Test suite() {
        Configuration testConfig = NbModuleSuite.createConfiguration(OverallTest.class);
        testConfig = testConfig.addTest("testBrushSize", "testPainting", "testClear", "testColorChooser");
        testConfig = testConfig.clusters(".*").enableModules(".*");
        return NbModuleSuite.create(testConfig);
    }

    /** Called before every test case. */
    public void setUp() {
        System.out.println("########  "+getName()+"  #######");
    }

    // Add test methods here, they have to start with 'test' name:

    /** Test brush size setting. */
    public void testBrushSize() {
        new Action("Window|New Canvas", null).perform();
        JSliderOperator slider = new JSliderOperator(MainWindowOperator.getDefault());
        slider.scrollToMaximum();
        slider.scrollToMinimum();
        slider.scrollToMaximum();
    }

    /** Test painting. */
    public void testPainting() {
        TopComponentOperator tcOper = new TopComponentOperator("Image");
        int x = tcOper.getCenterX();
        int y = tcOper.getCenterY();
        tcOper.clickMouse(x, y, 1);
        for (int i = 0; i < 50; i++) {
            tcOper.dragNDrop(x, y, x + 1, y + 1);
            x++;
            y++;
        }
        for (int i = 0; i<50; i++) {
            tcOper.dragNDrop(x, y, x - 1, y);
            x--;
        }
        for (int i = 0; i<50; i++) {
            tcOper.dragNDrop(x, y, x, y - 1);
            y--;
        }
    }

    /** Test clear button. */
    public void testClear() {
        new JButtonOperator(new TopComponentOperator("Image"), "Clear").push();
    }

    public void testColorChooser() {
        fail("Not yet implemented.");
    }

}</pre>

              <p>Create the class above in a package in the
            Functional Test Packages node in the Projects window.
            Then right-click the "Paint" project node
            and choose "Test". The application starts up
            and the specified functional tests are performed:</p>

<p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/result-4.png"/></p>
<p>Then the test results are shown:</p>
<p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/result-5.png"/></p>
<p>In the Files window (Ctrl-2), you can find the test results:</p>
<p><img style="border: 1px solid" alt="test folders" src="../../images/tutorials/test/result-6.png"/></p>


   <!-- ======================================================================================== -->

        <h2><a name="coverage"></a>Code Coverage on the NetBeans Platform</h2>

        <p>Via a plugin, it is easy to
            use the NetBeans Platform with
            <a href="http://cobertura.sourceforge.net/">Cobertura</a>.</p>

        <p>Take the following steps to use Cobertura for
             calculating the percentage of code accessed
             during unit testing and functional testing:</p>

        <ol>
            <li>In the IDE, go to Tools | Plugins and
                install "Cobertura Module Test Coverage".</li>
            <li>Go to the command line and browse to the "Paint" project folder, that is,
             browse to the module project folder where the tests have been run.</li>

            <li><p>Enter the following on the command line:</p>
<pre class="examplecode">ant coverage-report</pre></li>

            <li>You will see output such as the following:

<pre class="examplecode">...
...
...
 [junit] Cobertura: Loaded information on 8 classes.
    [junit] Cobertura: Saved information on 8 classes.
    [junit] Test tests.PaintCanvasTest FAILED

test-coverage:

coverage-report:
[cobertura-report] Cobertura 1.9.3 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
[cobertura-report] Cobertura: Loaded information on 8 classes.
[cobertura-report] Report time: 448ms

BUILD SUCCESSFUL
Total time: 9 seconds</pre></li>

            <li><p>In the IDE, within the Paint project,
                expand the "Important Files" node and then
                expand the "Build Script" node. A list of
                nodes is displayed for the targets defined
                in the build script. Find the target named
                "display-coverage-report", right-click it,
                and choose "Run Target". The following is
                displayed in the browser:</p>

<p><img style="border: 1px solid" alt="code coverage" src="../../images/tutorials/test/coverage.png"/></p>
            </li>
 <li><p>Enter the following on the command line:</p>
<pre class="examplecode">ant coverage-report-qa-functional</pre></li>


            <li>You will see output such as the following:

<pre class="examplecode">...
...
...
 testcoverage-restore-default:
   [delete] Deleting: /home/geertjan/PaintApp/build/cluster/modules/org-netbeans-paint.jar
     [copy] Copying 1 file to /home/geertjan/PaintApp/build/cluster

testcoverage-restore-specified:

test-coverage-restore:

test-coverage-qa-functional:

coverage-report-qa-functional:
[cobertura-report] Cobertura 1.9.3 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file
[cobertura-report] Cobertura: Loaded information on 8 classes.
[cobertura-report] Report time: 563ms

BUILD SUCCESSFUL
Total time: 23 seconds</pre></li>

 <li><p>The results can be found in the Files window (Ctrl-2):</p>

 <p><img style="border: 1px solid" alt="code coverage" src="../../images/tutorials/test/result-7.png"/></p>
 </li>
 <li><p>Open the index file to see the result:</p>

 <p><img style="border: 1px solid" alt="code coverage" src="../../images/tutorials/test/result-8.png"/></p>


 </li>

        </ol>
        
 <p>Further reading, advice, and warnings related to code
      coverage can be found <a href="http://wiki.netbeans.org/CodeCoverage">here</a>.</p>


        <!-- ======================================================================================== -->

        <h2><a name="nextsteps"></a>See Also</h2>

        <p>This concludes the NetBeans Platform Testing Tutorial.
            This document has described
            how to add unit testing and functional testing
            functionality to a NetBeans Platform application.
            For more information about testing on the NetBeans Platform,
            see the following resources:</p>

        <ul>
            <li><a href="http://openide.netbeans.org/tutorial/test-patterns.html">Typical Test Patterns</a></li>
            <li><a href="http://wiki.netbeans.org/NetBeansDeveloperTestFAQ" title="NetBeansDeveloperTestFAQ"> NetBeans Developer Test FAQ</a></li>
            <li><a href="http://blogs.oracle.com/coreqa/entry/xtest_is_dead_long_live">XTest is Dead Long Live Simpletests</a></li>
            <li><a href="http://wiki.netbeans.org/FitnessTestsWithoutX">FitnessTestsWithoutX</a></li>
            <li><a href="http://wiki.netbeans.org/DevFaqUsingSimpletests">Using Simpletests</a></li>
            <li><a href="http://forums.netbeans.org/topic10210.html">http://forums.netbeans.org/topic10210.html</a></li>
            <li><a href="http://wiki.netbeans.org/JellyTools">JellyTools</a></li>
            <li><a href="http://wiki.netbeans.org/DevRunningTestsPlatformApp" title="DevRunningTestsPlatformApp">Running tests on a platform application</a></li>
            <li><a href="http://wiki.netbeans.org/TestingThingsThatUseFileObjectDataObjectDataFolder" title="TestingThingsThatUseFileObjectDataObjectDataFolder"> Testing things that use FileObjects</a></li>
            <li><a href="http://wiki.netbeans.org/DevFaqTestDataObject" title="DevFaqTestDataObject"> Writing Tests for DataObjects and DataLoaders</a></li>
            <li><a href="http://wiki.netbeans.org/DevFaqTestUsingSystemFileSystem" title="DevFaqTestUsingSystemFileSystem"> How do I test something which uses the System Filesystem?</a></li>
            <li><a href="http://performance.netbeans.org/insane/index.html">INSANE</a></li>
            <li><a href="http://wiki.netbeans.org/CodeCoverage">Code Coverage</a></li>
        </ul>

        <!-- ======================================================================================== -->


    </body>
</html>