| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> |
| |
| <html> |
| <head> |
| <!-- -*- xhtml -*- --> |
| <title>NetBeans Platform Test Infrastructure for the NetBeans Platform</title> |
| <meta name="AUDIENCE" content="NBUSER"/> |
| <meta name="TYPE" content="ARTICLE"/> |
| <meta name="EXPIRES" content="N"/> |
| <meta name="developer" content="geertjan.wielenga@sun.com"/> |
| <meta name="indexed" content="y"/> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/> |
| <meta name="description" |
| content="A guide describing how to test NetBeans Platform applications."/> |
| <link rel="stylesheet" type="text/css" href="https://netbeans.org/netbeans.css"/> |
| </head> |
| |
| <!-- Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved. --> |
| <!-- Use is subject to license terms.--> |
| |
| <body> |
| <h1><a name="top"></a>NetBeans Platform Test Infrastructure Tutorial</h1> |
| |
| <p>During development of NetBeans Platform 6.5, an effort was |
| made to improve the testing infrastructure |
| provided for NetBeans Platform applications. |
| Prior to that, many small magical build scripts and other |
| configuration files were needed when setting up the test |
| infrastructure for NetBeans Platform applications. |
| Since then, however, there is inherent support for |
| testing NetBeans Platform application within the |
| NetBeans Platform's build harness scripts. |
| Therefore, unit and functional tests for NetBeans Platform |
| applications are now supported out of the box. |
| This simplification of the testing infrastructure is |
| sometimes referred to as "simpletests", since |
| it greatly simplifies the work necessary to set |
| up such tests for your application.</p> |
| |
| <p>In this tutorial, you are introduced to setting up |
| the "simpletests" infrastructure and to using |
| the NetBeans Platform's unit testing and |
| functional testing frameworks. These are:</p> |
| |
| <ul> |
| <li><b><a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/overview-summary.html">NB Junit.</a></b> NetBeans Platform extension to the <a href="http://www.junit.org/">JUnit</a> testing framework.</li> |
| <li><b><a href="http://wiki.netbeans.org/JellyTools">Jelly Tools.</a></b> NetBeans Platform extension to the <a href="https://jemmy.dev.java.net/">Jemmy</a> testing framework.</li> |
| </ul> |
| |
| <p>By the end of this tutorial, you should know how |
| to set up the NetBeans Platform testing infrastructure, |
| how to create unit tests, and how to create functional |
| tests.</p> |
| |
| <p><b>Contents</b></p> |
| |
| <p><img src="../images/articles/74/netbeans_stamp_74_73_72.png" class="stamp" width="114" height="114" alt="Content on this page applies to NetBeans IDE 7.0" title="Content on this page applies to NetBeans IDE 7.0"/></p> |
| <ul class="toc"> |
| <li><a href="#setup">Setting Up the Testing Infrastructure</a></li> |
| <li><a href="#unit">Unit Testing on the NetBeans Platform</a></li> |
| <li><a href="#functional">Functional Testing on the NetBeans Platform</a></li> |
| <li><a href="#coverage">Code Coverage on the NetBeans Platform</a></li> |
| </ul> |
| |
| <p><b>To follow this tutorial, you need the software and resources listed in the following |
| table.</b></p> |
| |
| <table> |
| <tbody> |
| <tr> |
| <th class="tblheader" scope="col">Software or Resource</th> |
| <th class="tblheader" scope="col">Version Required</th> |
| </tr> |
| <tr> |
| <td class="tbltd1"><a href="https://netbeans.org/downloads/index.html">NetBeans IDE</a></td> |
| <td class="tbltd1">version 7.0 or above</td> |
| </tr> |
| <tr> |
| <td class="tbltd1"><a href="http://java.sun.com/javase/downloads/index.jsp">Java Developer Kit (JDK)</a></td> |
| <td class="tbltd1">version 7 or above</td> |
| </tr> |
| </tbody> |
| </table> |
| |
| <!-- ===================================================================================== --> |
| |
| <h2 class="tutorial"><a name="setup"></a>Setting Up the Testing Infrastructure</h2> |
| |
| <p>When setting up the testing frameworks, you need |
| to enable certain modules that are disabled by |
| default. Then you need to create folders and files |
| in your source structure, where the libraries and |
| tests will live.</p> |
| |
| <p class="tips">If you want to try out these instructions on an actual |
| application prior to trying them out on your own |
| sources, you can use the NetBeans Platform Paint |
| Application, which you can get from the Samples |
| category in the New Project wizard (Ctrl-Shift-N).</p> |
| |
| <div class="indent"> |
| |
| <ol> |
| <li>In the Projects window, right-click your application and choose Properties. |
| In the Project Properties dialog, click "Libraries".</li> |
| <li><p>Check the "harness" box, adding the entire harness cluster, |
| which provides all the testing modules provided by |
| the NetBeans Platform:</p> |
| <p><img alt="enable harness" src="../images/tutorials/test/70/harness-cluster-2.png"/></p> |
| </li> |
| <li><p>Click the "Resolve" button, shown above, which will add |
| modules from the "platform" cluster |
| to your application, as needed |
| by the modules in the "harness" cluster. Notice |
| that there are now no messages about excluded |
| modules and that the Resolve button has |
| disappeared:</p> |
| <p><img alt="enable harness" src="../images/tutorials/test/70/harness-cluster-3.png"/></p> |
| </li> |
| </ol> |
| |
| </div> |
| |
| <p>Now you must set up a source structure for unit testing |
| and functional testing in the modules that need to |
| be tested. In other words, |
| the steps below do not apply to the application's source |
| structure, but to the modules that you need to test.</p> |
| |
| <div class="indent"> |
| |
| <ol> |
| <li>Switch to the Files window (Ctrl-2), and expand the module's main node.</li> |
| <li>If it does not exist, create a new folder named "test", within the |
| module's main folder.</li> |
| <li><p>Within the "test" folder, create a folder named "qa-functional". |
| Underneath "qa-functional", create a folder named "src". You can |
| use the New Folder wizard for these purposes, as follows: |
| </p> |
| <p><img alt="create folders" src="../images/tutorials/test/create-folder.png"/></p> |
| </li> |
| <li>Within the "test" folder, create a folder named "unit". |
| Underneath "unit", create a folder named "src". |
| </li> |
| <li><p>Check that the Files window shows the "test" folder |
| structure as shown below:</p> |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/test-folders.png"/></p> |
| </li> |
| <li>Restart the IDE.</li> |
| <li><p>In the Projects window notice |
| that there are now two new nodes for |
| your test packages and two new nodes |
| for adding the test libraries |
| to the classpath of the module:</p> |
| <p><img style="border: 1px solid" alt="new nodes" src="../images/tutorials/test/new-nodes.png"/></p> |
| </li> |
| <li><p>Right-click the "Unit Test Libraries" node |
| and choose "Add Missing Test Dependencies". Then add |
| "JUnit 4" and "NB JUnit". Right-click the "Functional Test Libraries" node |
| and choose "Add Missing Test Dependencies". |
| Then add "JUnit 4", "NB JUnit", "Jemmy", and |
| "Jelly Tools Platform".</p> |
| |
| <p class="tips">If you use "Add Missing Test Dependencies" |
| instead of "Add Unit Test Dependency" |
| and "Add Functional Test Dependency", |
| NB JUnit's recursive dependencies are properly |
| configured. Otherwise <a href="http://performance.netbeans.org/insane/index.html">INSANE</a> will not be |
| available, which can cause linkage errors when |
| running tests.</p> |
| |
| </li> |
| <li><p>Check that the Projects window shows the |
| test library dependencies as shown below:</p> |
| <p><img style="border: 1px solid" alt="test libs" src="../images/tutorials/test/new-libs-in-nodes.png"/></p> |
| </li> |
| </ol> |
| |
| </div> |
| |
| <p>You have now set up everything needed for creating |
| unit tests and functional tests on the NetBeans Platform.</p> |
| |
| <!-- ======================================================================================== --> |
| |
| <h2><a name="unit"></a>Unit Testing on the NetBeans Platform</h2> |
| |
| <p>The NetBeans Platform's extension to JUnit is named "<a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/overview-summary.html">NB JUnit</a>". |
| It is a separate library with additional support for memory |
| leak tests, deadlock tests, and extended use of logging. |
| More info and motivation |
| can be found |
| at <a href="http://openide.netbeans.org/tutorial/test-patterns.html">Test Patterns in Java</a>. |
| The basic test class for NB JUnit is <a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/org/netbeans/junit/NbTestCase.html">NbTestCase</a>.</p> |
| |
| <p>An example unit test for the NetBeans Platform Paint sample:</p> |
| |
| <pre class="examplecode">import org.netbeans.junit.*; |
| import org.netbeans.paint.PaintCanvas; |
| |
| public class PaintCanvasTest extends <a href="http://bits.netbeans.org/dev/javadoc/org-netbeans-modules-nbjunit/org/netbeans/junit/NbTestCase.html">NbTestCase</a> { |
| |
| public PaintCanvasTest(java.lang.String testName) { |
| super(testName); |
| } |
| |
| public void testSetDiam() { |
| PaintCanvas paintCanvas = new PaintCanvas(); |
| paintCanvas.setBrush(10); |
| assertEquals("Brush diameter should be set.", 10, paintCanvas.getBrushDiameter()); |
| } |
| |
| }</pre> |
| |
| <p>Create the class above in a package in the |
| Unit Test Packages node in the Projects window. |
| Then right-click the "Paint" project node |
| and choose "Test". If the test is successful, |
| you will see this:</p> |
| |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/result-1.png"/></p> |
| <p>If the test fails, you will see this:</p> |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/result-2.png"/></p> |
| <p>In the Files window (Ctrl-2), you can find the test results:</p> |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/result-3.png"/></p> |
| |
| <p>Typical questions relating to unit testing on the NetBeans Platform |
| relate to specific NetBeans API objects and the central registry |
| (also known as "System FileSystem"). For information on how to |
| test these parts of your application, see the following resources:</p> |
| <ul> |
| <li><a href="http://openide.netbeans.org/tutorial/test-patterns.html">Typical Test Patterns</a></li> |
| <li><a href="http://wiki.netbeans.org/TestingThingsThatUseFileObjectDataObjectDataFolder">Writing Tests with FileObjects, DataObjects, and DataFolders</a></li> |
| <li><a href="http://wiki.netbeans.org/DevFaqTestDataObject">Writing Tests with DataObjects and DataLoaders</a></li> |
| <li><a href="http://wiki.netbeans.org/InitializationOfDefaultLookup">Writing Tests with Lookup</a></li> |
| <li><a href="http://wiki.netbeans.org/DevFaqTestUsingSystemFileSystem" title="DevFaqTestUsingSystemFileSystem">How do I test something which uses the System Filesystem?</a></li> |
| </ul> |
| |
| <!-- ======================================================================================== --> |
| |
| <h2><a name="functional"></a>Functional Testing on the NetBeans Platform</h2> |
| |
| <p>The NetBeans Platform's extension to Jemmy is named |
| <a href="http://wiki.netbeans.org/JellyTools">JellyTools</a>. |
| It provides a set of operators that are tailored to UI |
| components used specifically in the NetBeans Platform, |
| such as <tt>TopComponentOperator</tt>.</p> |
| |
| <p>An example functional test for the NetBeans Platform Paint sample.</p> |
| |
| <pre class="examplecode">import junit.framework.Test; |
| import org.netbeans.jellytools.JellyTestCase; |
| import org.netbeans.jellytools.MainWindowOperator; |
| import org.netbeans.jellytools.TopComponentOperator; |
| import org.netbeans.jellytools.actions.Action; |
| import org.netbeans.jemmy.operators.JButtonOperator; |
| import org.netbeans.jemmy.operators.JSliderOperator; |
| import org.netbeans.junit.NbModuleSuite; |
| import org.netbeans.junit.NbModuleSuite.Configuration; |
| |
| /** |
| * A Test based on JellyTestCase. JellyTestCase redirects Jemmy output |
| * to a log file provided by NbTestCase. It can be inspected in results. |
| * It also sets timeouts necessary for NetBeans GUI testing. |
| * |
| * Any JemmyException (which is normally thrown as a result of an unsuccessful |
| * operation in Jemmy) going from a test is treated by JellyTestCase as a test |
| * failure; any other exception - as a test error. |
| * |
| * Additionally it: |
| * - closes all modal dialogs at the end of the test case (property jemmy.close.modal - default true) |
| * - generates component dump (XML file containing components information) in case of test failure (property jemmy.screen.xmldump - default false) |
| * - captures screen into a PNG file in case of test failure (property jemmy.screen.capture - default true) |
| * - waits at least 1000 ms between test cases (property jelly.wait.no.event - default true) |
| * |
| */ |
| |
| public class OverallTest extends JellyTestCase { |
| |
| /** Constructor required by JUnit */ |
| public OverallTest(String name) { |
| super(name); |
| } |
| |
| /** Creates suite from particular test cases. You can define order of testcases here. */ |
| public static Test suite() { |
| Configuration testConfig = NbModuleSuite.createConfiguration(OverallTest.class); |
| testConfig = testConfig.addTest("testBrushSize", "testPainting", "testClear", "testColorChooser"); |
| testConfig = testConfig.clusters(".*").enableModules(".*"); |
| return NbModuleSuite.create(testConfig); |
| } |
| |
| /** Called before every test case. */ |
| public void setUp() { |
| System.out.println("######## "+getName()+" #######"); |
| } |
| |
| // Add test methods here, they have to start with 'test' name: |
| |
| /** Test brush size setting. */ |
| public void testBrushSize() { |
| new Action("Window|New Canvas", null).perform(); |
| JSliderOperator slider = new JSliderOperator(MainWindowOperator.getDefault()); |
| slider.scrollToMaximum(); |
| slider.scrollToMinimum(); |
| slider.scrollToMaximum(); |
| } |
| |
| /** Test painting. */ |
| public void testPainting() { |
| TopComponentOperator tcOper = new TopComponentOperator("Image"); |
| int x = tcOper.getCenterX(); |
| int y = tcOper.getCenterY(); |
| tcOper.clickMouse(x, y, 1); |
| for (int i = 0; i < 50; i++) { |
| tcOper.dragNDrop(x, y, x + 1, y + 1); |
| x++; |
| y++; |
| } |
| for (int i = 0; i<50; i++) { |
| tcOper.dragNDrop(x, y, x - 1, y); |
| x--; |
| } |
| for (int i = 0; i<50; i++) { |
| tcOper.dragNDrop(x, y, x, y - 1); |
| y--; |
| } |
| } |
| |
| /** Test clear button. */ |
| public void testClear() { |
| new JButtonOperator(new TopComponentOperator("Image"), "Clear").push(); |
| } |
| |
| public void testColorChooser() { |
| fail("Not yet implemented."); |
| } |
| |
| }</pre> |
| |
| <p>Create the class above in a package in the |
| Functional Test Packages node in the Projects window. |
| Then right-click the "Paint" project node |
| and choose "Test". The application starts up |
| and the specified functional tests are performed:</p> |
| |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/result-4.png"/></p> |
| <p>Then the test results are shown:</p> |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/result-5.png"/></p> |
| <p>In the Files window (Ctrl-2), you can find the test results:</p> |
| <p><img style="border: 1px solid" alt="test folders" src="../images/tutorials/test/result-6.png"/></p> |
| |
| |
| <!-- ======================================================================================== --> |
| |
| <h2><a name="coverage"></a>Code Coverage on the NetBeans Platform</h2> |
| |
| <p>Via a plugin, it is easy to |
| use the NetBeans Platform with |
| <a href="http://cobertura.sourceforge.net/">Cobertura</a>.</p> |
| |
| <p>Take the following steps to use Cobertura for |
| calculating the percentage of code accessed |
| during unit testing and functional testing:</p> |
| |
| <div class="indent"> |
| |
| <ol> |
| <li>In the IDE, go to Tools | Plugins and |
| install "Cobertura Module Test Coverage".</li> |
| <li>Go to the command line and browse to the "Paint" project folder, that is, |
| browse to the module project folder where the tests have been run.</li> |
| |
| <li><p>Enter the following on the command line:</p> |
| <pre class="examplecode">ant coverage-report</pre></li> |
| |
| <li>You will see output such as the following: |
| |
| <pre class="examplecode">... |
| ... |
| ... |
| [junit] Cobertura: Loaded information on 8 classes. |
| [junit] Cobertura: Saved information on 8 classes. |
| [junit] Test tests.PaintCanvasTest FAILED |
| |
| test-coverage: |
| |
| coverage-report: |
| [cobertura-report] Cobertura 1.9.3 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file |
| [cobertura-report] Cobertura: Loaded information on 8 classes. |
| [cobertura-report] Report time: 448ms |
| |
| BUILD SUCCESSFUL |
| Total time: 9 seconds</pre></li> |
| |
| <li><p>In the IDE, within the Paint project, |
| expand the "Important Files" node and then |
| expand the "Build Script" node. A list of |
| nodes is displayed for the targets defined |
| in the build script. Find the target named |
| "display-coverage-report", right-click it, |
| and choose "Run Target". The following is |
| displayed in the browser:</p> |
| |
| <p><img style="border: 1px solid" alt="code coverage" src="../images/tutorials/test/coverage.png"/></p> |
| </li> |
| <li><p>Enter the following on the command line:</p> |
| <pre class="examplecode">ant coverage-report-qa-functional</pre></li> |
| |
| |
| <li>You will see output such as the following: |
| |
| <pre class="examplecode">... |
| ... |
| ... |
| testcoverage-restore-default: |
| [delete] Deleting: /home/geertjan/PaintApp/build/cluster/modules/org-netbeans-paint.jar |
| [copy] Copying 1 file to /home/geertjan/PaintApp/build/cluster |
| |
| testcoverage-restore-specified: |
| |
| test-coverage-restore: |
| |
| test-coverage-qa-functional: |
| |
| coverage-report-qa-functional: |
| [cobertura-report] Cobertura 1.9.3 - GNU GPL License (NO WARRANTY) - See COPYRIGHT file |
| [cobertura-report] Cobertura: Loaded information on 8 classes. |
| [cobertura-report] Report time: 563ms |
| |
| BUILD SUCCESSFUL |
| Total time: 23 seconds</pre></li> |
| |
| <li><p>The results can be found in the Files window (Ctrl-2):</p> |
| |
| <p><img style="border: 1px solid" alt="code coverage" src="../images/tutorials/test/result-7.png"/></p> |
| </li> |
| <li><p>Open the index file to see the result:</p> |
| |
| <p><img style="border: 1px solid" alt="code coverage" src="../images/tutorials/test/result-8.png"/></p> |
| |
| |
| </li> |
| |
| </ol> |
| |
| </div> |
| |
| <p>Further reading, advice, and warnings related to code |
| coverage can be found <a href="http://wiki.netbeans.org/CodeCoverage">here</a>.</p> |
| |
| <!-- ======================================================================================== --> |
| |
| <h2><a name="nextsteps"></a>See Also</h2> |
| |
| <p>This concludes the NetBeans Platform Testing Tutorial. |
| This document has described |
| how to add unit testing and functional testing |
| functionality to a NetBeans Platform application. |
| For more information about testing on the NetBeans Platform, |
| see the following resources:</p> |
| |
| <ul> |
| <li><a href="http://openide.netbeans.org/tutorial/test-patterns.html">Typical Test Patterns</a></li> |
| <li><a href="http://wiki.netbeans.org/NetBeansDeveloperTestFAQ" title="NetBeansDeveloperTestFAQ"> NetBeans Developer Test FAQ</a></li> |
| <li><a href="http://blogs.oracle.com/coreqa/entry/xtest_is_dead_long_live">XTest is Dead Long Live Simpletests</a></li> |
| <li><a href="http://wiki.netbeans.org/FitnessTestsWithoutX">FitnessTestsWithoutX</a></li> |
| <li><a href="http://wiki.netbeans.org/DevFaqUsingSimpletests">Using Simpletests</a></li> |
| <li><a href="http://forums.netbeans.org/topic10210.html">http://forums.netbeans.org/topic10210.html</a></li> |
| <li><a href="http://wiki.netbeans.org/JellyTools">JellyTools</a></li> |
| <li><a href="http://wiki.netbeans.org/DevRunningTestsPlatformApp" title="DevRunningTestsPlatformApp">Running tests on a platform application</a></li> |
| <li><a href="http://wiki.netbeans.org/TestingThingsThatUseFileObjectDataObjectDataFolder" title="TestingThingsThatUseFileObjectDataObjectDataFolder"> Testing things that use FileObjects</a></li> |
| <li><a href="http://wiki.netbeans.org/DevFaqTestDataObject" title="DevFaqTestDataObject"> Writing Tests for DataObjects and DataLoaders</a></li> |
| <li><a href="http://wiki.netbeans.org/DevFaqTestUsingSystemFileSystem" title="DevFaqTestUsingSystemFileSystem"> How do I test something which uses the System Filesystem?</a></li> |
| <li><a href="http://performance.netbeans.org/insane/index.html">INSANE</a></li> |
| <li><a href="http://wiki.netbeans.org/CodeCoverage">Code Coverage</a></li> |
| </ul> |
| |
| <!-- ======================================================================================== --> |
| |
| |
| </body> |
| </html> |