src/documentation/content/xdocs/dev/testing.xml - xmlgraphics-fop - Git at Google

 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
 <!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
 -->
 <!-- $Id$ -->
 <!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.3//EN" "http://forrest.apache.org/dtd/document-v13.dtd">
 <document>
   <header>
     <title>Apache™ FOP Development: Testing</title>
     <version>$Revision$</version>
   </header>

     <body>
     <section id="build">
       <title>"Build" Testing</title>
       <p>Apache™ projects use an automated build tool called "gump" to create nightly builds from the SVN repository. Gump sends "nag" messages if the build fails. This can be considered a sort of basic test of the build process. To view the most recent logs of the gump builds:</p>
       <ul>
         <li><link href="http://gump.cocoondev.org/xml-fop.html">Gump build for the Trunk</link></li>
         <li><link href="http://gump.cocoondev.org/xml-fop-maintenance.html">Gump build for the Maintenance Branch</link></li>
       </ul>
     </section>
     <section id="basic">
       <title>Basic/API Testing</title>
       <p>There is a group of basic API tests that are included in the build process.
 For these tests to occur, JUnit must be available to Ant (simply copy junit.jar into Ant's lib directory).
 The build will then report error(s) if the high-level APIs for Driver and the Transcoders fail.
 The tests do not check the output, but only ensure that something is generated and without exceptions.</p>
     </section>
     <section id="layoutengine">
       <title>Layout Engine Testing</title>
       <p>
         The <link href="http://svn.apache.org/viewcvs.cgi/xmlgraphics/fop/trunk/test/layoutengine/"><code>test/layoutengine</code></link>
         directory in the repository contains a test suite for checking the functionality of Apache� FOP's
         layout engine. For information on how to create test cases for the layout engine, please
         visit the <link href="http://wiki.apache.org/xmlgraphics-fop/HowToCreateLayoutEngineTests">Wiki page</link>.
       </p>
     </section>
 <section id="functional">
   <title>Functional Testing</title>
   <warning>The "functional" testing section on this page is currently inoperative.</warning>
   <section>
     <title>Running and Using Tests</title>
     <p>
 Testing is an important part of getting FOP to operate correctly and conform to the
 necessary standards.
     </p>
     <p>
 A testing system has been set up that works with as a build target when developing
 with FOP. A developer can run the tests after making changes to the code, the aim
 is to have the tests run to verfiy that nothing working has been broken.
     </p>
     <p>
 To setup the testing the developer must place a reference fop.jar in the
 "&lt;cvs_repository&gt;/test/reference/" directory. This jar will be dynamically
 loaded to create the reference output.
     </p>
   </section>

   <section>
     <title>W3C TestSuite</title>
     <p>
 The testing is set up so that you can download the testsuite from
 <jump href="http://www.w3.org/Style/XSL/TestSuite/">http://www.w3.org/Style/XSL/TestSuite/</jump>,
 unzip the file into the base directory of FOP.
 Then you can uncomment the lines in the build.xml file in the test target and itwill run through all the tests in the testsuite distribution.
     </p>
   </section>

   <section>
     <title>Writing a Test</title>
     <p>
 A test belongs to one of a few catagories. A basic test should excercise one
 element in a number of situations such as changing a property. This should have
 at least one normal value, one border value and one invalid value. If the property
 can be of different types then this should also be included.
     </p>
     <p>
 A bug test is a test that is specifically aimed at a problem with FOP. That is, the test
 is not excercising the specification but rather a problem with FOP in handling a particular
 situation that is not exposed with the other testing.
     </p>
     <p>
 A system test is one that tests the abitlity of FOP to handle a number of different
 elements together.
     </p>
     <p>
 A test can consist of a complete fo document or a part of the document such as
 some elements that will be placed into the flow of a standard document.
     </p>

   </section>
   <section>
     <title>Submitting a Test</title>
     <p>
 If you have a test which you think would be useful you should supply the
 test and a diff to the appropriate test suite xml file. Make sure that the
 test works as would be expected against the current build.
     </p>
   </section>

   <section>
     <title>How Testing Works</title>
     <p>
 The tests are stored in the "&lt;svn_repository&gt;/test" directory.
     </p>
     <p>
 You can run the tests by specifying the build target "test" ie: <br/>
 <code>ant.sh test</code> (Unix)<br/>
 <code>ant test</code> (Windows)<br/>
     </p>
     <p>
 This will then compare the current code in the local src directory to a specified
 release of FOP. Any differences between the current code and the output from
 the reference version will be reported. If the test previously passed then the
 test run will have failed.
     </p>
     <p>
 The testing is done by reading a test suite xml file, which corresponds to the
 standard testsuite.dtd supplied from w3c. This xml file contains a test xml
 file and an xsl file (which may simply copy the file). It also contains information
 such as if the test has passed and any comments.
     </p>
     <p>
 For FOP the testing is done by rendering all the testing documents using the
 XML renderer. The XML files are then compared to see if there are any differences.
     </p>
   </section>

   <section>
     <title>SVG Testing</title>
     <p>
 The testing of SVG is not part of this testing system. SVG is tested for its rendering
 accuracy by using the transcoding mechanism via
 <link href="ext:batik">Apache Batik</link>. So that the only part that needs
 testing is how the SVG image is embedded inside the flow of the fo document.
     </p>
   </section>
 </section>
 </body>
 </document>
	<?xml version="1.0" encoding="UTF-8" standalone="no"?>
	<!--
	Licensed to the Apache Software Foundation (ASF) under one or more
	contributor license agreements. See the NOTICE file distributed with
	this work for additional information regarding copyright ownership.
	The ASF licenses this file to You under the Apache License, Version 2.0
	(the "License"); you may not use this file except in compliance with
	the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing, software
	distributed under the License is distributed on an "AS IS" BASIS,
	WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
	See the License for the specific language governing permissions and
	limitations under the License.
	-->
	<!-- $Id$ -->
	<!DOCTYPE document PUBLIC "-//APACHE//DTD Documentation V1.3//EN" "http://forrest.apache.org/dtd/document-v13.dtd">
	<document>
	<header>
	<title>Apache™ FOP Development: Testing</title>
	<version>$Revision$</version>
	</header>

	<body>
	<section id="build">
	<title>"Build" Testing</title>
	<p>Apache™ projects use an automated build tool called "gump" to create nightly builds from the SVN repository. Gump sends "nag" messages if the build fails. This can be considered a sort of basic test of the build process. To view the most recent logs of the gump builds:</p>
	<ul>
	<li><link href="http://gump.cocoondev.org/xml-fop.html">Gump build for the Trunk</link></li>
	<li><link href="http://gump.cocoondev.org/xml-fop-maintenance.html">Gump build for the Maintenance Branch</link></li>
	</ul>
	</section>
	<section id="basic">
	<title>Basic/API Testing</title>
	<p>There is a group of basic API tests that are included in the build process.
	For these tests to occur, JUnit must be available to Ant (simply copy junit.jar into Ant's lib directory).
	The build will then report error(s) if the high-level APIs for Driver and the Transcoders fail.
	The tests do not check the output, but only ensure that something is generated and without exceptions.</p>
	</section>
	<section id="layoutengine">
	<title>Layout Engine Testing</title>
	<p>
	The <link href="http://svn.apache.org/viewcvs.cgi/xmlgraphics/fop/trunk/test/layoutengine/"><code>test/layoutengine</code></link>
	directory in the repository contains a test suite for checking the functionality of Apache� FOP's
	layout engine. For information on how to create test cases for the layout engine, please
	visit the <link href="http://wiki.apache.org/xmlgraphics-fop/HowToCreateLayoutEngineTests">Wiki page</link>.
	</p>
	</section>
	<section id="functional">
	<title>Functional Testing</title>
	<warning>The "functional" testing section on this page is currently inoperative.</warning>
	<section>
	<title>Running and Using Tests</title>
	<p>
	Testing is an important part of getting FOP to operate correctly and conform to the
	necessary standards.
	</p>
	<p>
	A testing system has been set up that works with as a build target when developing
	with FOP. A developer can run the tests after making changes to the code, the aim
	is to have the tests run to verfiy that nothing working has been broken.
	</p>
	<p>
	To setup the testing the developer must place a reference fop.jar in the
	"<cvs_repository>/test/reference/" directory. This jar will be dynamically
	loaded to create the reference output.
	</p>
	</section>

	<section>
	<title>W3C TestSuite</title>
	<p>
	The testing is set up so that you can download the testsuite from
	<jump href="http://www.w3.org/Style/XSL/TestSuite/">http://www.w3.org/Style/XSL/TestSuite/</jump>,
	unzip the file into the base directory of FOP.
	Then you can uncomment the lines in the build.xml file in the test target and itwill run through all the tests in the testsuite distribution.
	</p>
	</section>

	<section>
	<title>Writing a Test</title>
	<p>
	A test belongs to one of a few catagories. A basic test should excercise one
	element in a number of situations such as changing a property. This should have
	at least one normal value, one border value and one invalid value. If the property
	can be of different types then this should also be included.
	</p>
	<p>
	A bug test is a test that is specifically aimed at a problem with FOP. That is, the test
	is not excercising the specification but rather a problem with FOP in handling a particular
	situation that is not exposed with the other testing.
	</p>
	<p>
	A system test is one that tests the abitlity of FOP to handle a number of different
	elements together.
	</p>
	<p>
	A test can consist of a complete fo document or a part of the document such as
	some elements that will be placed into the flow of a standard document.
	</p>

	</section>
	<section>
	<title>Submitting a Test</title>
	<p>
	If you have a test which you think would be useful you should supply the
	test and a diff to the appropriate test suite xml file. Make sure that the
	test works as would be expected against the current build.
	</p>
	</section>

	<section>
	<title>How Testing Works</title>
	<p>
	The tests are stored in the "<svn_repository>/test" directory.
	</p>
	<p>
	You can run the tests by specifying the build target "test" ie: <br/>
	<code>ant.sh test</code> (Unix)<br/>
	<code>ant test</code> (Windows)<br/>
	</p>
	<p>
	This will then compare the current code in the local src directory to a specified
	release of FOP. Any differences between the current code and the output from
	the reference version will be reported. If the test previously passed then the
	test run will have failed.
	</p>
	<p>
	The testing is done by reading a test suite xml file, which corresponds to the
	standard testsuite.dtd supplied from w3c. This xml file contains a test xml
	file and an xsl file (which may simply copy the file). It also contains information
	such as if the test has passed and any comments.
	</p>
	<p>
	For FOP the testing is done by rendering all the testing documents using the
	XML renderer. The XML files are then compared to see if there are any differences.
	</p>
	</section>

	<section>
	<title>SVG Testing</title>
	<p>
	The testing of SVG is not part of this testing system. SVG is tested for its rendering
	accuracy by using the transcoding mechanism via
	<link href="ext:batik">Apache Batik</link>. So that the only part that needs
	testing is how the SVG image is embedded inside the flow of the fo document.
	</p>
	</section>
	</section>
	</body>
	</document>