CB-7459 Allow auto tests to be run for specific plugin(s)

* On the "Auto Tests" page for plugins, generate a list of checkboxes to select
  each of the detected plugins.  These are initially checked based on "enabled"
  status of each plugin.

* Subsequent runs (i.e. using the existing <Again> button) will reflect any user
  changes to the selected tests

* User selections are persisted to localStorage, to remember the settings for
  subsequent runs of the app.  The enabling of test modules is encapsulated in
  the framework, so both user and console/programmatic selection are retained the
  same way via localStorage

* When the page is initially loaded, keep the existing behaviour, where all
  enabled plugin tests are automatically run

* UI enhancements to show/hide list of tests, add Select/Unselect All buttons,
  and use shorter test names
3 files changed
tree: 53e1943b081d41d09e0e2bd410a19c02c343ba64
  1. www/
  2. .gitignore
  3. LICENSE
  4. plugin.xml
  5. README.md
README.md

Cordova Plugin Test Framework

The org.apache.cordova.test-framework plugin does two things:

  1. Defines the interface for cordova plugins to write tests
  2. Provides a test harness for actually running those tests

Tests run directly inside existing cordova projects, so you can rapidly switch between testing and development. You can also be sure that your test suite is testing the exact versions of plugins and platforms that your app is using.

TLDR; Try it

  1. Use your existing cordova app, or create a new one.

  2. To make this interesting, add some plugins which actually bundle tests. Here are a few examples:

     cordova plugin add http://git-wip-us.apache.org/repos/asf/cordova-plugin-device.git#cdvtest
     cordova plugin add http://git-wip-us.apache.org/repos/asf/cordova-plugin-device-motion.git#cdvtest
     cordova plugin add http://git-wip-us.apache.org/repos/asf/cordova-plugin-geolocation.git#cdvtest
    
  3. Follow the docs for Setting up the test harness.

Where do tests live?

Add a <js-module> named tests to your plugin.xml. E.g. org.apache.cordova.device plugin has this in its plugin.xml:

<js-module src="test/tests.js" name="tests">
</js-module>

The org.apache.cordova.test-framework plugin will automatically find all tests modules across all plugins.

Defining Auto Tests

Simply export a function named defineAutoTests, which (gasp!) defines your auto-tests when run. Use the jasmine-2.0 format. E.g.:

exports.defineAutoTests = function() {

  define('awesome tests', function() {
    it('do something sync', function() {
      expect(1).toBe(1);
      ...
    });

    it('do something async', function(done) {
      setTimeout(function() {
        expect(1).toBe(1);
        ...
        done();
      }, 100);
    });
  });

  define('more awesome tests', function() {
    ...
  });

};

Note: Your tests will automatically be labeled with your plugin id, so do not prefix your test descriptions.

Defining Manual Tests

Simply export a function named defineManualTests, which (gasp!) defines your manual-tests when run. Manual tests do not use jasmine-2.0, and success/failure results are not officially reported in any standard way. Instead, create buttons to run arbitraty javascript when clicked, and display output to user using console or by manipulating a provided DOM element. E.g.:

exports.defineManualTests = function(contentEl, createActionButton) {

  createActionButton('Simple Test', function() {
    console.log(JSON.stringify(foo, null, '\t'));
  });

  createActionButton('Complex Test', function() {
    contentEl.innerHTML = ...;
  });

};

Note: Your tests will automatically be labeled with your plugin id, so do not prefix your test descriptions.

See: org.apache.cordova.device's tests.

  1. Use your existing cordova app, or create a new one.

  2. Add this plugin:

     cordova plugin add http://git-wip-us.apache.org/repos/asf/cordova-plugin-test-framework.git
    
  3. Change the start page in config.xml with <content src="cdvtests/index.html" /> or navigate to cdvtests/index.html from within your app.

  4. Thats it!

FAQ

  • Q: Should I add org.apache.cordova.test-framework as a <dependancy> of my plugin?

    • A: No. The end-user should decide if they want to install the test framework, not your plugin (most users won't).
  • Q: What do I do if my plugin tests must have very large assets?

    • A: Don‘t bundle those assets with your plugin. If you can, have your tests fail gracefully if those assets don’t don't exist (perhaps log a warning, perhaps fail a single asset-checking test, and skip the rest). Then, ideally download those assets automatically into local storage the first time tests run. Or create a manual test step to download and install assets. As a final alternative, split those test assets into a separate plugin, and instruct users to install that plugin to run your full test suite.
  • Q: Should I ship my app with the test framework plugin installed?

    • A: Not likely. If you want, you can. Then your app could even embed a link to the test page (cdvtests/index.html) from a help section of your app, to give end users a way to run your test suite out in the feild. That may help diagnose causes of issues within your app. Maybe.