netbeans.apache.org/src/content/wiki/SignatureTest.asciidoc - netbeans-website - Git at Google

 //
 //     Licensed to the Apache Software Foundation (ASF) under one
 //     or more contributor license agreements.  See the NOTICE file
 //     distributed with this work for additional information
 //     regarding copyright ownership.  The ASF licenses this file
 //     to you under the Apache License, Version 2.0 (the
 //     "License"); you may not use this file except in compliance
 //     with the License.  You may obtain a copy of the License at
 //
 //       http://www.apache.org/licenses/LICENSE-2.0
 //
 //     Unless required by applicable law or agreed to in writing,
 //     software distributed under the License is distributed on an
 //     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 //     KIND, either express or implied.  See the License for the
 //     specific language governing permissions and limitations
 //     under the License.
 //

 = SignatureTest
 :jbake-type: wiki
 :jbake-tags: wiki, devfaq, needsreview
 :markup-in-source: verbatim,quotes,macros
 :jbake-status: published
 :syntax: true
 :description: Use from Maven
 :icons: font
 :source-highlighter: pygments
 :toc: left
 :toc-title:
 :experimental:

 Signature tests verify that public and protected members of non-private classes
 in each module do not change or remain compatible with previous version. These
 tests are part of the build or the
 link:http://deadlock.netbeans.org/hudson/[continuous build process]. The
 testing infrastructure is currently based on link:APITest.asciidoc[APITest], an
 extension and temporary fork of link:http://sigtest.java.net/[sigtest tool]
 which is used as part of the link:http://www.jcp.org/[JCP] test compatibility
 kits to verify that the API exposed by a JSR implementation does not diverge
 from the reference implementation.

 [[Use_from_Maven]]
 == Use from Maven

 See link:SigTest.asciidoc[SigTest] for the details.

 [[Consistency_Check]]
 == Consistency Check

 *Turned on Nov 7, 2012*:

 For a while the link:VersioningPolicy.asciidoc[VersioningPolicy] prescribes how
 to properly version modules and their API changes. However the policy has for a
 long time only been applied to modules with stable API. From time to time this
 leads to _no such method or class_ errors like {{iz|186233 which need to be
 evaluated by _platform/module system_ component owner.

 Moreover the introduction of incremental binary build
 link:AutoUpdateTask.asciidoc[AutoUpdateTask] requires more proper attitude to
 versioning. Only then one can be sure that incremental binary build really
 links properly.

 For the above reasons ita consistency check that verifies minimal versioning
 consistency has been introduced. The
 link:HgParallelProjectIntegration.html[HgParallelProjectIntegration] runs:

 * Check on all modules exposing non-private packages
 * Compares the spec version.
 ** If the new spec is the same as previous, the check fails if there is some API change (method or class)
 ** If the new spec is higher, the check will pass regardless of any API changes.

 The check is integral part of push-* jobs and if it detects a failure, the push job fails. *To recover* you are supposed to increment spec version of your module whenever you change its API. Increment spec versions of modules that depend on your API change as well (not enforced right now). Run following command to simulate the check locally:

 [source,bash,subs="{markup-in-source}"]
 ----
 # first of all make sure your system is built OK
 # Then either:
 # generate appropriate JUnit testing report for all modules:
 $ ant check-sigtests-version
 #
 # or run the 'version' check for a single module
 $ ant -f your.module/build.xml check-sigtest -Dsigtest.check.type=versioncheck

 ----


 [[Continuous_Check_for_Changes]]
 == Continuous Check for Changes

 The link:http://deadlock.netbeans.org/hudson/[continuous build process] runs a sigtest build that compares APIs in a new build with APIs of the one of the previous build. Whenever there is a change (either compatible or incompatible) a mail is sent:

 * if the module has public packages - to the link:mailto:api-changes@netbeans.org[api-changes@netbeans.org] mailing list
 * if the module has friends in other clusters - to the link:mailto:api-changes@netbeans.org[api-changes@netbeans.org] mailing list
 * if the module has friends only in the same cluster - to appropriate *cvs@<module>.netbeans.org* mailing list
 * if the module has neither friends, neither public packages - it is skipped from the check

 Appropriate owners of each module and tech leads of each area are expected to review the change and prevent or revert unintentional changes.

 The link:http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastSuccessfulBuild/artifact/nbbuild/build/sigtest/snapshot/[signature golden files] are generated by

 [source,bash,subs="{markup-in-source}"]
 ----

 $ ant -f nbbuild/build.xml gen-sigtests
 ----

 and compared by

 [source,bash,subs="{markup-in-source}"]
 ----

 $ ant -f nbbuild/build.xml gen-sigtests
 ----

 A module can configure sigtest behavior by modifying its *project.properties*:

 [source,bash,subs="{markup-in-source}"]
 ----

 # redefine the email where reports are sent
 sigtest.mail=your.name@your.org
 # don't fail if the generation of the report fails
 sigtest.gen.fail.on.error=false

 ----


 [[At_the_time_of_release]]
 == At the time of release

 When doing a release it is important to record the actual signatures of _stable_ modules (and to some extend also _under development_ ones), so we know what to keep link:apidesign::BackwardCompatibility.html[backward compatibility] with. To generate these signatures we used to do following, but since link:https://github.com/apache/incubator-netbeans/pull/1064[Apache NetBeans is in Git], it is better to follow the link:https://github.com/apache/incubator-netbeans/pull/1064[PR-1064 steps]. Anyway here is the old recipe:

 [source,bash,subs="{markup-in-source}"]
 ----

 $ hg update -C releaseXY_fcs
 $ ant clean
 $ ant build
 $ ant gen-sigtests-release
 $ hg st|cut -c 3-| xargs zip /tmp/sig.zip

 ----

 The last target will modify a lot of module/nbproject/*.sig files and possibly generate some new. These files shall be integrated into trunk build. For example by:

 [source,bash,subs="{markup-in-source}"]
 ----

 $ cd <team-repository>
 $ hg update -c default
 $ unzip /tmp/sig.zip
 $ hg add .
 $ hg ci -m "Signatures as of release X.Y"
 $ hg push

 ----


 [[Check_for_Incompatible_Changes_Since_Previous_Version]]
 == Check for Incompatible Changes Since Previous Version

 There is another check for API signature compliance: A check that the APIs are compatible with a previous released version. This check is available to every module, which can make it part of its build by generating its API snapshot by:

 [source,bash,subs="{markup-in-source}"]
 ----

 ant netbeans gen-sigtest

 ----

 This creates a file nbproject/org-netbeans-module-name.sig which can be checked into the version control system as a reference point. This check is part of continuous build as well: every module with its golden file in nbproject/org-netbeans-module-name.sig is checked for backward compatibility and failures are included in continuous build link:http://deadlock.netbeans.org/hudson/job/trunk/lastBuild/testReport/[test results]. To run the same test for one module on local computer just use:

 [source,bash,subs="{markup-in-source}"]
 ----

 $ ant netbeans check-sigtest

 ----

 in your module directory and API of your module will be verified. Of course, only if the signature file exists.


 [[Who_needs_the_Golden_Signature_File]]
 === Who needs the Golden Signature File

 Not every module with an API needs this golden file; currently only those modules that produced a stable API in release 6.1 have it. The plan is to update these golden files with a new version as soon as it is released, so future checks are done against latest released version.

 If an incompatible change is made (discouraged and subject to link:http://openide.netbeans.org/tutorial/reviews[API review process]) the golden file needs to be edited by hand to remove the element that is no longer present in the API.


 [[Checking_for_all_changes_from_previous_release]]
 == Checking for all changes from previous release

 In case you have the sig file in nbproject directory generated at the time of previour release (all public APIs have it), you can see all changes since that time by:

 [source,bash,subs="{markup-in-source}"]
 ----

 $ ant -f yourmodule/build.xml check-sigtest -Dsigtest.check.type=strictcheck

 ----


 [[apitest_tool]]
 === apitest tool

 This check for incompatibilities is done by link:APITest.asciidoc[NetBeans' own version] of an OpenJDK tools. This adds a standard Ant task to check for binary backward compatibility, mutual signature compatibility. The binary compatibility check ignores generic types. Its link:http://hg.netbeans.org/apitest/[sources] are distributed together with its Mercurial repository and are available under GPL version 2. Builds are available from our link:http://deadlock.netbeans.org/hudson/job/apitest/[hudson builder].


 [[Other_NetBeans_Tests]]
 === Other NetBeans Tests

 * link:FindBugsTest.asciidoc[FindBugsTest]


 [NOTE]
 ====

 The content in this page was kindly donated by Oracle Corp. to the
 Apache Software Foundation.

 This page was exported from link:http://wiki.netbeans.org/SignatureTest[http://wiki.netbeans.org/SignatureTest] ,
 that was last modified by NetBeans user Jtulach
 on 2018-12-31T04:42:32Z.


 This document was automatically converted to the AsciiDoc format on 2020-03-12, and needs to be reviewed.
 ====
	//
	// Licensed to the Apache Software Foundation (ASF) under one
	// or more contributor license agreements. See the NOTICE file
	// distributed with this work for additional information
	// regarding copyright ownership. The ASF licenses this file
	// to you under the Apache License, Version 2.0 (the
	// "License"); you may not use this file except in compliance
	// with the License. You may obtain a copy of the License at
	//
	// http://www.apache.org/licenses/LICENSE-2.0
	//
	// Unless required by applicable law or agreed to in writing,
	// software distributed under the License is distributed on an
	// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	// KIND, either express or implied. See the License for the
	// specific language governing permissions and limitations
	// under the License.
	//

	= SignatureTest
	:jbake-type: wiki
	:jbake-tags: wiki, devfaq, needsreview
	:markup-in-source: verbatim,quotes,macros
	:jbake-status: published
	:syntax: true
	:description: Use from Maven
	:icons: font
	:source-highlighter: pygments
	:toc: left
	:toc-title:
	:experimental:

	Signature tests verify that public and protected members of non-private classes
	in each module do not change or remain compatible with previous version. These
	tests are part of the build or the
	link:http://deadlock.netbeans.org/hudson/[continuous build process]. The
	testing infrastructure is currently based on link:APITest.asciidoc[APITest], an
	extension and temporary fork of link:http://sigtest.java.net/[sigtest tool]
	which is used as part of the link:http://www.jcp.org/[JCP] test compatibility
	kits to verify that the API exposed by a JSR implementation does not diverge
	from the reference implementation.

	[[Use_from_Maven]]
	== Use from Maven

	See link:SigTest.asciidoc[SigTest] for the details.

	[[Consistency_Check]]
	== Consistency Check

	Turned on Nov 7, 2012:

	For a while the link:VersioningPolicy.asciidoc[VersioningPolicy] prescribes how
	to properly version modules and their API changes. However the policy has for a
	long time only been applied to modules with stable API. From time to time this
	leads to _no such method or class_ errors like {{iz\|186233 which need to be
	evaluated by _platform/module system_ component owner.

	Moreover the introduction of incremental binary build
	link:AutoUpdateTask.asciidoc[AutoUpdateTask] requires more proper attitude to
	versioning. Only then one can be sure that incremental binary build really
	links properly.

	For the above reasons ita consistency check that verifies minimal versioning
	consistency has been introduced. The
	link:HgParallelProjectIntegration.html[HgParallelProjectIntegration] runs:

	* Check on all modules exposing non-private packages
	* Compares the spec version.
	** If the new spec is the same as previous, the check fails if there is some API change (method or class)
	** If the new spec is higher, the check will pass regardless of any API changes.

	The check is integral part of push-* jobs and if it detects a failure, the push job fails. To recover you are supposed to increment spec version of your module whenever you change its API. Increment spec versions of modules that depend on your API change as well (not enforced right now). Run following command to simulate the check locally:

	[source,bash,subs="{markup-in-source}"]
	----
	# first of all make sure your system is built OK
	# Then either:
	# generate appropriate JUnit testing report for all modules:
	$ ant check-sigtests-version
	#
	# or run the 'version' check for a single module
	$ ant -f your.module/build.xml check-sigtest -Dsigtest.check.type=versioncheck

	----


	[[Continuous_Check_for_Changes]]
	== Continuous Check for Changes

	The link:http://deadlock.netbeans.org/hudson/[continuous build process] runs a sigtest build that compares APIs in a new build with APIs of the one of the previous build. Whenever there is a change (either compatible or incompatible) a mail is sent:

	* if the module has public packages - to the link:mailto:api-changes@netbeans.org[api-changes@netbeans.org] mailing list
	* if the module has friends in other clusters - to the link:mailto:api-changes@netbeans.org[api-changes@netbeans.org] mailing list
	* if the module has friends only in the same cluster - to appropriate cvs@<module>.netbeans.org mailing list
	* if the module has neither friends, neither public packages - it is skipped from the check

	Appropriate owners of each module and tech leads of each area are expected to review the change and prevent or revert unintentional changes.

	The link:http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastSuccessfulBuild/artifact/nbbuild/build/sigtest/snapshot/[signature golden files] are generated by

	[source,bash,subs="{markup-in-source}"]
	----

	$ ant -f nbbuild/build.xml gen-sigtests
	----

	and compared by

	[source,bash,subs="{markup-in-source}"]
	----

	$ ant -f nbbuild/build.xml gen-sigtests
	----

	A module can configure sigtest behavior by modifying its project.properties:

	[source,bash,subs="{markup-in-source}"]
	----

	# redefine the email where reports are sent
	sigtest.mail=your.name@your.org
	# don't fail if the generation of the report fails
	sigtest.gen.fail.on.error=false

	----


	[[At_the_time_of_release]]
	== At the time of release

	When doing a release it is important to record the actual signatures of _stable_ modules (and to some extend also _under development_ ones), so we know what to keep link:apidesign::BackwardCompatibility.html[backward compatibility] with. To generate these signatures we used to do following, but since link:https://github.com/apache/incubator-netbeans/pull/1064[Apache NetBeans is in Git], it is better to follow the link:https://github.com/apache/incubator-netbeans/pull/1064[PR-1064 steps]. Anyway here is the old recipe:

	[source,bash,subs="{markup-in-source}"]
	----

	$ hg update -C releaseXY_fcs
	$ ant clean
	$ ant build
	$ ant gen-sigtests-release
	$ hg st\|cut -c 3-\| xargs zip /tmp/sig.zip

	----

	The last target will modify a lot of module/nbproject/*.sig files and possibly generate some new. These files shall be integrated into trunk build. For example by:

	[source,bash,subs="{markup-in-source}"]
	----

	$ cd <team-repository>
	$ hg update -c default
	$ unzip /tmp/sig.zip
	$ hg add .
	$ hg ci -m "Signatures as of release X.Y"
	$ hg push

	----


	[[Check_for_Incompatible_Changes_Since_Previous_Version]]
	== Check for Incompatible Changes Since Previous Version

	There is another check for API signature compliance: A check that the APIs are compatible with a previous released version. This check is available to every module, which can make it part of its build by generating its API snapshot by:

	[source,bash,subs="{markup-in-source}"]
	----

	ant netbeans gen-sigtest

	----

	This creates a file nbproject/org-netbeans-module-name.sig which can be checked into the version control system as a reference point. This check is part of continuous build as well: every module with its golden file in nbproject/org-netbeans-module-name.sig is checked for backward compatibility and failures are included in continuous build link:http://deadlock.netbeans.org/hudson/job/trunk/lastBuild/testReport/[test results]. To run the same test for one module on local computer just use:

	[source,bash,subs="{markup-in-source}"]
	----

	$ ant netbeans check-sigtest

	----

	in your module directory and API of your module will be verified. Of course, only if the signature file exists.


	[[Who_needs_the_Golden_Signature_File]]
	=== Who needs the Golden Signature File

	Not every module with an API needs this golden file; currently only those modules that produced a stable API in release 6.1 have it. The plan is to update these golden files with a new version as soon as it is released, so future checks are done against latest released version.

	If an incompatible change is made (discouraged and subject to link:http://openide.netbeans.org/tutorial/reviews[API review process]) the golden file needs to be edited by hand to remove the element that is no longer present in the API.


	[[Checking_for_all_changes_from_previous_release]]
	== Checking for all changes from previous release

	In case you have the sig file in nbproject directory generated at the time of previour release (all public APIs have it), you can see all changes since that time by:

	[source,bash,subs="{markup-in-source}"]
	----

	$ ant -f yourmodule/build.xml check-sigtest -Dsigtest.check.type=strictcheck

	----


	[[apitest_tool]]
	=== apitest tool

	This check for incompatibilities is done by link:APITest.asciidoc[NetBeans' own version] of an OpenJDK tools. This adds a standard Ant task to check for binary backward compatibility, mutual signature compatibility. The binary compatibility check ignores generic types. Its link:http://hg.netbeans.org/apitest/[sources] are distributed together with its Mercurial repository and are available under GPL version 2. Builds are available from our link:http://deadlock.netbeans.org/hudson/job/apitest/[hudson builder].


	[[Other_NetBeans_Tests]]
	=== Other NetBeans Tests

	* link:FindBugsTest.asciidoc[FindBugsTest]


	[NOTE]
	====

	The content in this page was kindly donated by Oracle Corp. to the
	Apache Software Foundation.

	This page was exported from link:http://wiki.netbeans.org/SignatureTest[http://wiki.netbeans.org/SignatureTest] ,
	that was last modified by NetBeans user Jtulach
	on 2018-12-31T04:42:32Z.


	This document was automatically converted to the AsciiDoc format on 2020-03-12, and needs to be reviewed.
	====