portlet-tck_3.0/README.txt - portals-pluto - Git at Google


 This project contains the JSR 362 Portlet Specification 3.0 TCK.

 Overview:
 =========

 The TCK contains portlets and a test driver that are used to perform the
 compatibility test. The portlets contain the code that actually exercises the
 portlet API. The driver uses Selenium 2 driven by Junit to access the system
 under test through a selenium webdriver browser.

 You can click through the tests on the portal pages displayed by the system under
 test, or you can use the test driver to automatically click through the tests
 and collect the results. Since there are many test cases (>6000), you will
 generally use the test driver.

 A test portlet may define multiple test cases. The test case names must be
 unique in the system, and must be of a form that can be used as portlet names
 and as Java Properties file keys.

 Each portlet test module defines its own test case list and page file containing
 the page definitions expected by the tests contained in the module.

 The build process generates a consolidated test case list and consolidated page
 file containing all test cases and page definitions for all test modules. The
 consolidated test case list maps the test case to the name of the page on which
 the test case is located.

 The test driver works by reading the consolidated test case list. For each test
 case, the driver accesses the specified page. It logs in if necessary, clicks
 any available test case link, and collects the test case results from the
 browser window.

 The test driver accesses a TCK page by looking for and clicking a link whose
 link text is equal to the page name. If the test driver cannot find the link on
 current page, it will access the test server login URL configured in the
 portlet-tck_3.0/POM.xml file.

 The test driver checks whether a login is necessary by checking for the
 existence of an entry field that has the HTML element ID for the user name
 field. If the user name field is available, the driver enters the user name and
 password into the corresponding entry fields and submits the form. The username
 HTML element ID, password element ID, username and password are configurable
 in the portlet-tck_3.0/POM.xml file.

 The driver checks for links to be clicked and results to be collected by
 checking for specific HTML element IDs on the page. The HTML element IDs are
 based on the test case name. From the point of view of the test driver there can
 be several relevant element IDs on the page for each test case:

 "<testcasename>-setup"        - Test setup link
 "<testcasename>-clickme"      - Test execution link
 "<testcasename>-results"      - Test results field; "Succeeded" or "Failed"
 "<testcasename>-details"      - Test details field; provides failure reason

 After accessing the page on which the test case is to be found, the test driver
 first looks for a setup link. If found, it clicks the link and waits for the
 page to load.

 The driver then looks for a test execution link. If found,
 it clicks the link, waits for the new page to load, and then checks for the
 result elements.

 The driver reads the contents of the HTML elements that have the results &
 details element IDs and analyzes the results string to determine test case
 success or failure.

 Project Structure:
 ==================

 The project consists of the following modules:

 common      - Common code used by the test portlets and the test driver. This
               module also contains the XSL stylesheets used during the test
               portlet module build to generate the test case list and page
               configuration for the module.
 deploy      - The directory deploy/target/deploy-files contains the portlet war
               files and the page file to be deployed on the system under test.
 driver      - The maven-based test driver. When this module is built, it
               generates a conslidated test case list and page file for the test
               cases and pages defined by all test case portlet modules.
 TestModule1 - Defines several portlets, each of which is placed on its own page.
 TestModule2 - Defines several portlets, all of which are placed on the same
               portal page. Some tests produce links that are clicked by the test
               driver when the test is run. Some tests set public render
               parameters that are used by another test portlet.
 TestModule3 - Defines two portlets in separate portlet applications for public
               render parameter testing. Both portlets are placed on the same
               page. When the test driver clicks a test link in one portlet, the
               test results are displayed in the second portlet.
 V2*Tests    - Modules with this naming scheme contain test cases to demostrate
               backward compatibilty with the Portlet Specification version 2.0.
               Portlets whose class names contain the string "_SIG_" contain
               generated code for verifying the method signatures. These portlets
               should not be changed by hand. The remaining portlets contain
               additional V2 verification code.
 V3*Tests    - Modules with this naming scheme test v3.0 specific funtionality.

 The Page File:
 ==============

 The page file defines the page names for the TCK and the portlets expected to
 be on each page. This is an XML file in the form of a Pluto driver configuration
 file. The schema for this file is located in the Pluto project at the following
 location:

 ./pluto-portal-driver/src/main/resources/pluto-portal-driver-config.xsd

 After the TCK has been built, complete page file for the TCK pages is contained
 in the following file:

 ./pluto/portlet-tck_3.0/deploy/target/deploy-files/pluto-portal-driver-config.xml

 It is expected that vendors will extract the portlet to page mapping information
 from the page file in order to create the necessary configuration files for
 the portal under test.

 When creating pages for your system, note that the test driver expects links to
 all test case pages to be available. The test driver searches for the page links
 by link text, so it is important that the links have the names defined in the
 page file.

 Building the TCK:
 =================

 Building the TCK produces the test portlets, a list of test cases in XML format,
 an XML-based page file containing the page configuration expected by the test
 cases, and the test driver itself. Perform the build from the project base
 directory.

 To build portlet war files that do not contain any special configuration, use
 the command:

 mvn clean install

 To build portlet war files that are configured for deployment on Pluto, use the
 command:

 mvn clean install -Ppluto

 These commands can be executed either from the Pluto project base directory or
 from the ./portlet-tck_3.0 directory.

 Deploying TCK artifacts:
 ========================

 Deploy the portlet war files contained in the deploy/target/deploy-files
 directory on your system under test. Use the page file which has the default
 name of deploy/target/deploy-files/pluto-portal-driver-config.xml to create
 pages for your system. If you are testing with Apache Pluto, you can simply copy
 this file into the Pluto webapps/pluto/WEB-INF directory in order to set up the
 pages. The WAR files contain the test portlet code. Deploy these files on your
 system. To deploy them on Pluto, copy them into the Pluto webapps directory.

 Running the TCK:
 ================

 Configure the parent POM file to address your system under test. See properties
 section. You can set the host name and port for the system under test, and can
 also select the browser to be used by the Selenium WebDriver code. Note,
 however, that if you use a browser other than "firefox" or "HTMLUnit" you will
 have to download and install additional WebDriver software manually, and you may
 need to add additional configuration properties (See the test.browser.webDriver
 property in the parent POM).

 At the time of this writing, the HTMLUnit driver does not support JavaScript
 adequately for JSR 362 TCK purposes. You may need to try different drivers to
 find the appropriate WebDriver driver for your environment.

 All TCK tests were successfully executed through the test driver against Pluto
 using the ChromeDriver web driver. See the portlet-tck_3.0/POM.xml file for
 a configuration example.

 Run the TCK by switching to the driver subdirectory and execute the command:

 mvn test -Prun-tck

 If you want to run a subset of the tests, use the "-Dtest.module=" option. The
 driver will then only execute those tests whose test case name contains the
 specified string. For example, if you want to run the signature verification
 tests, use the command:

 mvn test -Prun-tck -Dtest.module=SIG

 You can also exclude specific tests by placing an exclamation point before the
 text to be matched. For example, if you want to exclude the signature verification
 tests, use the command:

 mvn test -Prun-tck -Dtest.module=!SIG

 The 'test.dryrun' flag will cause the test cases to be listed without executing them.
 For example, the following command will list all of the test cases whose test
 case name contains 'ResourceResponse', but will not execute them:

 mvn test -Prun-tck -Dtest.module=ResourceResponse -Dtest.dryrun=true

 The test driver will wait for a response for a default time of 3 seconds. If this
 value proves to be too short, you can change it in the portlet-tck_3.0/POM.xml
 file, or you can specify a timeout value using the 'test-timeout' command line
 option. The timeout value is specified in seconds, as follows:

 mvn test -Prun-tck -Dtest.module=ResourceResponse -Dtest.timeout=5

 The test driver has an ignore list containing test case names that are not
 required to pass in order for a portal product to be JSR 362 compliant. The
 ignore list is an XML file in properties file format located at:

 portlet-tck_3.0/driver/src/main/resources/xml-resources/ignoredTestCases.xml

 By default, the test driver will ignore and will not execute all test cases
 in the ignore list. If you wish to execute all test cases including those in
 the ignore list, set the command line flag 'test.ignore' to false:

 mvn test -Prun-tck -Dtest.ignore=false

 Under certain circumstances such as a very fast CPU or a large display size,
 Selenium throws a WebDriverException claiming that a button or link  is not
 clickable. If this happens, the window scrolling workarond can be enabled:

 mvn test -Prun-tck -Dtest.scroll=true

 Passing the TCK:
 ================

 TCK execution is self-policing. If you are a portal vendor and wish to claim
 JSR 362 compatibility for your product, you should be able to demonstrate that
 this TCK can be executed against your product with no errors.

 Contesting a Test Case:
 =======================

 When defining and implementing the TCK test cases, care was taken to assure
 proper implementation and to test only required functionality.

 However, mistakes may have occurred. If you feel that a test case is incorrectly
 implemented or is invalid as defined, please proceed as follows.

 In either case, please open an issue at the Pluto Jira:

 https://issues.apache.org/jira/browse/PLUTO/

 Prefix the summary with 'TCK:' in order to allow TCK issues to be readily
 identified. Please be sure to specify the exact name(s) of the test case(s) in
 question.

 If you feel that the test case was implemented incorrectly, you may correct the
 implementation and issue a pull request through github here:

 https://github.com/apache/portals-pluto/pulls

 If you feel that the test case is invalid as defined, you may update the ignore
 list and issue a pull request.

 If we agree that the test case is invalid or incorrectly implemented, we will
 fix it or integrate your fix into the Pluto master and version 3.0 branches.

	This project contains the JSR 362 Portlet Specification 3.0 TCK.

	Overview:
	=========

	The TCK contains portlets and a test driver that are used to perform the
	compatibility test. The portlets contain the code that actually exercises the
	portlet API. The driver uses Selenium 2 driven by Junit to access the system
	under test through a selenium webdriver browser.

	You can click through the tests on the portal pages displayed by the system under
	test, or you can use the test driver to automatically click through the tests
	and collect the results. Since there are many test cases (>6000), you will
	generally use the test driver.

	A test portlet may define multiple test cases. The test case names must be
	unique in the system, and must be of a form that can be used as portlet names
	and as Java Properties file keys.

	Each portlet test module defines its own test case list and page file containing
	the page definitions expected by the tests contained in the module.

	The build process generates a consolidated test case list and consolidated page
	file containing all test cases and page definitions for all test modules. The
	consolidated test case list maps the test case to the name of the page on which
	the test case is located.

	The test driver works by reading the consolidated test case list. For each test
	case, the driver accesses the specified page. It logs in if necessary, clicks
	any available test case link, and collects the test case results from the
	browser window.

	The test driver accesses a TCK page by looking for and clicking a link whose
	link text is equal to the page name. If the test driver cannot find the link on
	current page, it will access the test server login URL configured in the
	portlet-tck_3.0/POM.xml file.

	The test driver checks whether a login is necessary by checking for the
	existence of an entry field that has the HTML element ID for the user name
	field. If the user name field is available, the driver enters the user name and
	password into the corresponding entry fields and submits the form. The username
	HTML element ID, password element ID, username and password are configurable
	in the portlet-tck_3.0/POM.xml file.

	The driver checks for links to be clicked and results to be collected by
	checking for specific HTML element IDs on the page. The HTML element IDs are
	based on the test case name. From the point of view of the test driver there can
	be several relevant element IDs on the page for each test case:

	"<testcasename>-setup" - Test setup link
	"<testcasename>-clickme" - Test execution link
	"<testcasename>-results" - Test results field; "Succeeded" or "Failed"
	"<testcasename>-details" - Test details field; provides failure reason

	After accessing the page on which the test case is to be found, the test driver
	first looks for a setup link. If found, it clicks the link and waits for the
	page to load.

	The driver then looks for a test execution link. If found,
	it clicks the link, waits for the new page to load, and then checks for the
	result elements.

	The driver reads the contents of the HTML elements that have the results &
	details element IDs and analyzes the results string to determine test case
	success or failure.

	Project Structure:
	==================

	The project consists of the following modules:

	common - Common code used by the test portlets and the test driver. This
	module also contains the XSL stylesheets used during the test
	portlet module build to generate the test case list and page
	configuration for the module.
	deploy - The directory deploy/target/deploy-files contains the portlet war
	files and the page file to be deployed on the system under test.
	driver - The maven-based test driver. When this module is built, it
	generates a conslidated test case list and page file for the test
	cases and pages defined by all test case portlet modules.
	TestModule1 - Defines several portlets, each of which is placed on its own page.
	TestModule2 - Defines several portlets, all of which are placed on the same
	portal page. Some tests produce links that are clicked by the test
	driver when the test is run. Some tests set public render
	parameters that are used by another test portlet.
	TestModule3 - Defines two portlets in separate portlet applications for public
	render parameter testing. Both portlets are placed on the same
	page. When the test driver clicks a test link in one portlet, the
	test results are displayed in the second portlet.
	V2*Tests - Modules with this naming scheme contain test cases to demostrate
	backward compatibilty with the Portlet Specification version 2.0.
	Portlets whose class names contain the string "_SIG_" contain
	generated code for verifying the method signatures. These portlets
	should not be changed by hand. The remaining portlets contain
	additional V2 verification code.
	V3*Tests - Modules with this naming scheme test v3.0 specific funtionality.

	The Page File:
	==============

	The page file defines the page names for the TCK and the portlets expected to
	be on each page. This is an XML file in the form of a Pluto driver configuration
	file. The schema for this file is located in the Pluto project at the following
	location:

	./pluto-portal-driver/src/main/resources/pluto-portal-driver-config.xsd

	After the TCK has been built, complete page file for the TCK pages is contained
	in the following file:

	./pluto/portlet-tck_3.0/deploy/target/deploy-files/pluto-portal-driver-config.xml

	It is expected that vendors will extract the portlet to page mapping information
	from the page file in order to create the necessary configuration files for
	the portal under test.

	When creating pages for your system, note that the test driver expects links to
	all test case pages to be available. The test driver searches for the page links
	by link text, so it is important that the links have the names defined in the
	page file.

	Building the TCK:
	=================

	Building the TCK produces the test portlets, a list of test cases in XML format,
	an XML-based page file containing the page configuration expected by the test
	cases, and the test driver itself. Perform the build from the project base
	directory.

	To build portlet war files that do not contain any special configuration, use
	the command:

	mvn clean install

	To build portlet war files that are configured for deployment on Pluto, use the
	command:

	mvn clean install -Ppluto

	These commands can be executed either from the Pluto project base directory or
	from the ./portlet-tck_3.0 directory.

	Deploying TCK artifacts:
	========================

	Deploy the portlet war files contained in the deploy/target/deploy-files
	directory on your system under test. Use the page file which has the default
	name of deploy/target/deploy-files/pluto-portal-driver-config.xml to create
	pages for your system. If you are testing with Apache Pluto, you can simply copy
	this file into the Pluto webapps/pluto/WEB-INF directory in order to set up the
	pages. The WAR files contain the test portlet code. Deploy these files on your
	system. To deploy them on Pluto, copy them into the Pluto webapps directory.

	Running the TCK:
	================

	Configure the parent POM file to address your system under test. See properties
	section. You can set the host name and port for the system under test, and can
	also select the browser to be used by the Selenium WebDriver code. Note,
	however, that if you use a browser other than "firefox" or "HTMLUnit" you will
	have to download and install additional WebDriver software manually, and you may
	need to add additional configuration properties (See the test.browser.webDriver
	property in the parent POM).

	At the time of this writing, the HTMLUnit driver does not support JavaScript
	adequately for JSR 362 TCK purposes. You may need to try different drivers to
	find the appropriate WebDriver driver for your environment.

	All TCK tests were successfully executed through the test driver against Pluto
	using the ChromeDriver web driver. See the portlet-tck_3.0/POM.xml file for
	a configuration example.

	Run the TCK by switching to the driver subdirectory and execute the command:

	mvn test -Prun-tck

	If you want to run a subset of the tests, use the "-Dtest.module=" option. The
	driver will then only execute those tests whose test case name contains the
	specified string. For example, if you want to run the signature verification
	tests, use the command:

	mvn test -Prun-tck -Dtest.module=SIG

	You can also exclude specific tests by placing an exclamation point before the
	text to be matched. For example, if you want to exclude the signature verification
	tests, use the command:

	mvn test -Prun-tck -Dtest.module=!SIG

	The 'test.dryrun' flag will cause the test cases to be listed without executing them.
	For example, the following command will list all of the test cases whose test
	case name contains 'ResourceResponse', but will not execute them:

	mvn test -Prun-tck -Dtest.module=ResourceResponse -Dtest.dryrun=true

	The test driver will wait for a response for a default time of 3 seconds. If this
	value proves to be too short, you can change it in the portlet-tck_3.0/POM.xml
	file, or you can specify a timeout value using the 'test-timeout' command line
	option. The timeout value is specified in seconds, as follows:

	mvn test -Prun-tck -Dtest.module=ResourceResponse -Dtest.timeout=5

	The test driver has an ignore list containing test case names that are not
	required to pass in order for a portal product to be JSR 362 compliant. The
	ignore list is an XML file in properties file format located at:

	portlet-tck_3.0/driver/src/main/resources/xml-resources/ignoredTestCases.xml

	By default, the test driver will ignore and will not execute all test cases
	in the ignore list. If you wish to execute all test cases including those in
	the ignore list, set the command line flag 'test.ignore' to false:

	mvn test -Prun-tck -Dtest.ignore=false

	Under certain circumstances such as a very fast CPU or a large display size,
	Selenium throws a WebDriverException claiming that a button or link is not
	clickable. If this happens, the window scrolling workarond can be enabled:

	mvn test -Prun-tck -Dtest.scroll=true

	Passing the TCK:
	================

	TCK execution is self-policing. If you are a portal vendor and wish to claim
	JSR 362 compatibility for your product, you should be able to demonstrate that
	this TCK can be executed against your product with no errors.

	Contesting a Test Case:
	=======================

	When defining and implementing the TCK test cases, care was taken to assure
	proper implementation and to test only required functionality.

	However, mistakes may have occurred. If you feel that a test case is incorrectly
	implemented or is invalid as defined, please proceed as follows.

	In either case, please open an issue at the Pluto Jira:

	https://issues.apache.org/jira/browse/PLUTO/

	Prefix the summary with 'TCK:' in order to allow TCK issues to be readily
	identified. Please be sure to specify the exact name(s) of the test case(s) in
	question.

	If you feel that the test case was implemented incorrectly, you may correct the
	implementation and issue a pull request through github here:

	https://github.com/apache/portals-pluto/pulls

	If you feel that the test case is invalid as defined, you may update the ignore
	list and issue a pull request.

	If we agree that the test case is invalid or incorrectly implemented, we will
	fix it or integrate your fix into the Pluto master and version 3.0 branches.