| GETTING SETUP |
| |
| This document and the OpenEJB TCK setup can be checked out from here: |
| |
| svn co https://svn.apache.org/repos/tck/openejb-tck |
| |
| The Java EE 6 TCK and RI can be downloaded from here: |
| |
| svn export https://svn.apache.org/repos/tck/sun-tcks/javaee/6/javaeetck-6.0_29-Jun-2011.zip |
| svn export https://svn.apache.org/repos/tck/sun-tcks/javaee/6/javaee6_ri-3.0.1-b22.zip |
| |
| Both are required to run the TCK. The TCK is 813M, beware. |
| |
| Once unpacked, they can be "hooked" up via the maven settings.xml |
| file like so: |
| |
| <settings> |
| <profiles> |
| <profile> |
| <id>javaee-tck-environment</id> |
| |
| <activation> |
| <activeByDefault>true</activeByDefault> |
| </activation> |
| |
| <properties> |
| <javaee6.cts.home>/Users/dblevins/work/javaeetck</javaee6.cts.home> |
| <javaee6.ri.home>/Users/dblevins/work/javaeetck/glassfishv3</javaee6.ri.home> |
| </properties> |
| </profile> |
| </profiles> |
| </settings> |
| |
| The TCK will unzip into a directory called 'javaeetck/'. The RI will |
| unzip into a directory called 'glassfishv3'. As you can see above |
| we've unzipped the RI inside the javaeetck directory just to keep |
| things tidy. This is not required for the TCK to function properly. |
| |
| TEST RUN |
| |
| From inside the openejb-tck/trunk/ directory, a command like this |
| will get you a little taste of running the TCK: |
| |
| ./runtests --web tomee com.sun.ts.tests.ejb30.bb.localaccess.statelessclient |
| |
| We don't yet pass all of the TCK, but the above tests should be among |
| the passing sections and are a good way to validate all is setup |
| properly. |
| |
| TOMCAT |
| |
| By default the `runtests` command will execute the tests against an |
| OpenEJB standalone server. |
| |
| To run in Tomcat, a command like the following will work: |
| |
| ./runtests --tomcat-version 7.0.8 com.sun.ts.tests.ejb30.bb.localaccess.statelessclient |
| |
| Any Tomcat version can be specfified and it will be downloaded prior |
| to running the tck. The are stored in the $HOME/.m2/repository, but |
| with the group id org.apache.openejb.tck as to discourage accidental |
| dependence in other projects. |
| |
| TOMEE |
| |
| To run in Tomee, specify the webcontainer is "tomee". |
| |
| ./runtests --web tomee com.sun.ts.tests.ejb30.bb.localaccess.statelessclient |
| |
| MISC |
| |
| The target directory is not cleaned out at the beginning of a test |
| run. There are a few thousand tests and sometimes multiple |
| executions are required to get complete results. It's also nice to |
| be able to look back on older log files when tracking down and fixing |
| bugs that the tests uncover. |
| |
| Bottom line is you have to clear out the target directory manually. |
| On occasion some bad state will get into the server install in the |
| target/ directory. If you start getting weirld maven or groovy |
| errors, clean out the target dir and try again. |
| |
| TAB COMPLETION |
| |
| There is a nice little script in the root directory called |
| runtests.completer which, when sourced, can give be a great |
| time-saver when trying to navigate to run a specific test. |
| |
| In bash just source the file like so: |
| |
| source runtests.completer |
| |
| LOGS |
| |
| The TCK for the most part runs as a client in a separate vm. The |
| test results are sent to this vm and then logged here: |
| |
| target/logs/javatest.log |
| |
| When looking at exceptions in that log file often come from the |
| remote deployer tool -- the same tool we use on the command line for |
| deployment. Most of the deployment related exceptions were generated |
| on the server and sent to the client and that's why the show up in |
| that log. |
| |
| The server logs are in the usual place: |
| |
| target/openejb-4.0.0-SNAPSHOT/logs/openejb.log |
| target/apache-tomcat-7.0.8/logs/openejb.log |
| |
| SELECTING TESTS |
| |
| It is possible to select whole groups of tests or even individual |
| tests. The following are all valid ways to select which tests you'd |
| like to run. |
| |
| ./runtests -tv 7.0.8 -c com.sun.ts.tests.ejb30 com.sun.ts.tests.ejb |
| ./runtests -tv 7.0.8 -c com.sun.ts.tests.ejb30.lite.stateful.concurrency.accesstimeout |
| ./runtests -tv 7.0.8 -c com.sun.ts.tests.ejb30.lite.stateful.concurrency.accesstimeout.annotated |
| ./runtests -tv 7.0.8 -c com.sun.ts.tests.ejb30.lite.stateful.concurrency.accesstimeout.annotated.Client#beanClassLevel_from_ejbembed |
| |
| The first command runs of the ejb30 and ejb sections of the TCK |
| illustrating that it is possble to run many sections or tests at |
| once. The very last line shows the syntax for running just one |
| specific test. |
| |
| Note that the output of the tck shows which exact tests are being |
| run. For example: |
| |
| ...[tck output]... |
| com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel_from_ejbembed - FAILED |
| com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel_from_ejblitejsf - PASSED |
| com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel_from_ejblitejsp - PASSED |
| com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel_from_ejbliteservlet - PASSED |
| com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel_from_ejbliteservlet2 - PASSED |
| com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel2_from_ejbembed - FAILED |
| .... |
| |
| For the most part, you can copy and past that test name as-is and use |
| it to run a test that failed... with one slight adjustment. You need |
| to delete the "#java" part and then it will work. |
| |
| BAD |
| |
| ./runtests -tv 7.0.8 com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#java#beanClassLevel_from_ejbembed |
| |
| GOOD |
| |
| ./runtests -tv 7.0.8 com/sun/ts/tests/ejb30/lite/stateful/concurrency/accesstimeout/annotated/Client#beanClassLevel_from_ejbembed |
| |
| THE ROAD TO CERTIFICATION |
| |
| Getting from zero to passing is a long road. Failures and the |
| overall progress tends to go in three stages: |
| |
| 1. setup issues -- the right things are not where they need to be |
| |
| 2. missing features -- a key feature is missing causing failures |
| in unrelated tests. |
| |
| 3. compliance issues -- legitimate failures. |
| |
| During phase 1 there will be big jumps in numbers. It is best to |
| clear out as much of phase 1 as possible before moving on to any |
| issues of phase 2 or 3. |
| |
| During phase 2 it becomes apparent that many tests fail simply |
| because of an unrelated feature that many tests use, such as global |
| jndi support. As these features are added, the tests that still fail |
| are usually failing for more legitimate reasons -- actual compliance |
| issues -- this is phase 3. |
| |
| Phase 3 takes the longest and is often the hardest. Unlike phase 1 |
| or 2, the time spent debugging and fixing a test usually only results |
| in one or two more passing tests. It is also common that fixing a |
| specific test requires reworking part of the code. This inevitably |
| results in "two steps forward, one step backward" and other tests |
| might fail because of the change. This is normal. It is also the |
| reason why there should be no more phase 1 or 2 style issues, so that |
| it is possible to see the regressions. Working on phase 3 style |
| issues while there are still phase 1 and 2 style issues is a little |
| bit like working blind. You don't really know how many steps |
| backward you might be taking as a result of a change. It can be |
| done, but it is risky. |
| |
| WORKING TOGETHER |
| Communication:- |
| -Email:Send an email to tck-subscribe@openejb.apache.org to subscribe to the tck discussion list |
| -IRC: You can also jump on #tck channel at freenode |
| |
| We want to divide and conquer on each phase and clear it out as much |
| as possible before moving to the next one. We could possibly get up |
| to 80% passing before reaching phase 3. |
| |
| So the name of the game is "call your shot" or "name it and claim |
| it." Find an issue that affects as many tests as possible and post |
| that you are working on it so others know not to look into it as |
| well. |
| |
| If you get busy or stuck, no problem, just post again to let others |
| know the issue is up for grabs. This is also normal. Taking a quick |
| peak and then realizing that the issue involves someone else's area |
| of expertise is common. Even if you aren't able to fix something, |
| taking a look and reporting as much info as you can is incredibly |
| valuable. It's all part of the certification dance and will ideally |
| happen very often -- the right people working on the right things |
| gets you certified much faster. |
| |
| There are usually so many issues that finding the right one for you |
| is somewhat like sifting through a pile of legos looking for that |
| perfect piece. It doesn't always fit -- chuck it back and look for |
| another one. |
| |
| LEGAL CONSTRAINTS |
| |
| We are not allowed to publicly give specific information about the |
| TCK and our status. When referring publicly to the TCK we can only |
| give essentially "binary" information, do we pass or not. Publicly |
| saying which sections we pass and how much is generally not allowed. |
| |
| Talking publicly about the specific details of a compliance issue is |
| fine as long as the TCK and any TCK identifying information is not |
| mentioned. Publicly mentioning TCK test names, class names or |
| sharing stack traces is not allowed. That information should be |
| posted to the TCK list. All other information can be discussed |
| publicly. |
| |
| For example, saying "I noticed we have an issue with X part of the |
| specification under Y conditions" is OK. Saying "Test X fails" is |
| not. When in doubt, be a little generic. |
| |
| All that said, we are an open source project and we should strive to |
| be as public as possible and communicate as much as we can on the dev |
| list, in our JIRA and in our commit messages. It is an uncomfortable |
| reality that the driver of much development (the TCK) is private yet |
| the development itself is public. The more we share publicly, the |
| less the project seems driven by an invisible force. |
| |