We are currently working on introducing a CI (Continuous Integration) build process for the PRs submitted to the cordova plugin repositories. This would help the community to catch issues before checking in the code. I am currently working on a prototype. I would like to share the idea and get early feedback on it.
Currently, Cordova plugins have continuous intergration builds running:
This process, though helpful, have the following disadvantages:
This proposal recommends to use Jenkins as our new CI system instead of buildbot. With Jenkins and its huge list of plugins, we could achieve a lot more functionality such as GitHub Pull requests, updating PRs, generating test reports and a much better UI.
Initially, we considered using the Jenkins Master machine provided by ASF. But, after further discussions through emails and infra ticket, we realized that it is not possible to hook up our slave machines with the ASF master. So, we need a new Jenkins master machine that could be hooked to our slave machines and accessible publicly. We have decided to use a Jenkins master machine that is deployed on Azure cloud. You could see the prototype here - http://cordova-ci.cloudapp.net:8080/
This master machine will be connected to two slaves which are actual physical machines. One of them will be a Windows 10 machine and the other will be a Mac machine.
The Jenkins master uses the ghprb (Github Pull Request Builder Plugin) plugin to poll github, every five minutes, to check for new PRs and changes to existing PRs. If a new PR (or a commit to an existing PR) has been detected, then the change is picked up and a new CI build is triggered on the following platforms:
This build uses the cordova-paramedic package to test the plugin. This package will help to test the specific plugin that is changed. Paramedic also provides a simple and easy way for the PR submitter to reproduce the error locally and test the changes.
Once the build is complete, the PR will be updated with the a comment indicating the result of the CI build and a link to the build job. You could see a sample PR created as a prototype here - https://github.com/sarangan12/cordova-plugin-console/pull/18. The build link will have detailed console logs and also device logs.
A new github user (such as “CordovaQA”) will be created to update the comments in the PR. This approach has been used by Apache Spark project also.
In addition to new changes submitted to PR, a CI build could be triggered by fixed phrases in the PR comments (such as “CordovaQA - Test these changes.”). In addition to the CI builds per PR, there will be periodic builds per plugin on the master branch.The Jenkins master will have Github OAuth integrated in it. Everybody (even without a GitHub authentication) will have view access. Edit/Configure access will be provided based on per request basis.
Currently, the prototype is a work in progress. It should be completed soon. I will send out the updates to the dev list once the prototype is complete.
In future we might consider using services such as saucelabs to run our CI builds (instead of physical slaves).