wiki-files/BuildBot/BuildBot.md.html - ofbiz-tools - Git at Google

 <!DOCTYPE html>
 <html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
 <head>
   <meta charset="utf-8" />
   <meta name="generator" content="pandoc" />
   <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
   <title>BuildBot</title>
   <style type="text/css">
       code{white-space: pre-wrap;}
       span.smallcaps{font-variant: small-caps;}
       span.underline{text-decoration: underline;}
       div.column{display: inline-block; vertical-align: top; width: 50%;}
   </style>
   <!--[if lt IE 9]>
     <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
   <![endif]-->
 </head>
 <body>
 <!--
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 -->
 <h2 id="what-is-builbbot-and-why-using-it">What is BuilbBot and why using it?</h2>
 <p>OFBiz uses <a href="https://en.wikipedia.org/wiki/Buildbot">BuilbBot</a> as its Continuous Integration (CI) tool. There are other CI tools <a href="https://ci.apache.org/">used at the ASF</a>. There is also <a href="https://github.com/apache/infrastructure-p6/blob/production/modules/subversion_server/files/hooks/buildbot_project_paths">a file in Git for project paths</a>. You can’t clone, but you can make PRs inside it AIUI, via the github UI, just make a new branch and PR it. More information about creating a new branch below.</p>
 <h2 id="ofbiz-and-buildbot">OFBiz and BuildBot</h2>
 <p><a href="https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf">Here is the OFBiz BuildBot script</a>. If you are interested on modifying it, please understand its content and read the BuildBot documentation. Note that the Infra team is always there to help you.</p>
 <h3 id="schedulers-and-builders">Schedulers and Builders</h3>
 <p>BuildBot uses schedulers and builders. Schedulers decide which builders to trigger on Git commits.</p>
 <p>You can see the OFBiz builders and their states in the <a href="https://ci.apache.org/waterfall">common Buildbot waterfall view</a> (look for “ofb”).</p>
 <p>The <a href="https://ci.apache.org/builders/ofbizTrunkFramework">ofbizTrunkFramework</a> builder is triggered on commits in the trunk framework branch. Then only the framework integration tests are run. This also triggers the <a href="https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins">ofbizTrunkFrameworkPlugins</a> if the build succeed. Then the whole integration tests are run. To check that a commit in the framework puts no regressions in plugins and especially because the dependencies of ofbiz-framework without ofbiz-plugins is different from the dependencies of ofbiz-framework + ofbiz-plugins (plugins components are Gradle sub-projects). So testing needs to happen in both scenarios because you might face library version bugs.</p>
 <p>Of course a commit into the trunk plugins also triggers the ofbizTrunkFrameworkPlugins.</p>
 <p>The same apply to the <a href="https://ci.apache.org/builders/ofbizBranch17Framework">ofbizBranch17Framework</a> and the <a href="https://ci.apache.org/builders/ofbizBranch17FrameworkPlugins">ofbizBranch17FrameworkPlugins</a> and next releases branches, they follows the same structure than the trunk.</p>
 <p>In an OFBiz builder page you can see <a href="https://ci.apache.org/builders/ofbizTrunkFramework?numbuilds=100">an history of the builds up to 100+</a>. OFBiz randomly uses 3 servers (except for RAT which always only uses asf945_ubuntu) to build. You may want to explore more to get acquainted with the tool…</p>
 <p>There is also the <a href="https://ci.apache.org/builders/ofbizTrunkFrameworkRat">RAT builder</a>. It checks the licenses in OFBiz. You can find the result <a href="https://ci.apache.org/projects/ofbiz/site/trunk/rat-output.html">here</a>. There are also RAT builders for the current <a href="https://ci.apache.org/projects/ofbiz/site/next/rat-output.html">next</a> and <a href="https://ci.apache.org/projects/ofbiz/site/stable/rat-output.html">stable</a> release branches.</p>
 <h3 id="tests-results">tests results</h3>
 <p>Here are the <a href="https://ci.apache.org/projects/ofbiz/logs/">test results</a>. The folders structure reflects the current Git repo structure.</p>
 <h3 id="technical-information">Technical information</h3>
 <p>When you create a new branch you need to let know BuildBot about it. This because BuildBot uses Git hooks to triggers builds on commits. <a href="https://github.com/apache/infrastructure-p6/blob/production/modules/subversion_server/files/hooks/buildbot_project_paths">The file to change is here</a>. You can’t clone, but you can make PRs inside it AIUI, via the github UI, just make a new branch and PR it.</p>
 <h2 id="handling-issues">Handling issues</h2>
 <p>Sometimes (rarely) you can get transient tests errors in BuildBot. This mean tests don’t all pass in BuildBot, though they pass in your local instance. In such case, it’s most certainly an issue with servers. Those are hard workers and make errors from time to time, which shows that not only human make errors.</p>
 <p>Before doing anything it’s best to check which BuildBot step is impacted, and if it exists have a look at the logfile (stdio)</p>
 <p>Some other errors may happen, like * Git not updating * upload not working * you name it…</p>
 <p>In such case you can trigger a build from IRC to see if the problem resolves by itself. Most of the time tests and G issues are resolved this way. If it does not then the best is to ask Infra help, either on <a href="https://the-asf.slack.com/archives/CBX4TSBQ8">Infra Slack channel</a> or by creating an <a href="https://issues.apache.org/jira/projects/INFRA/summary">Infra Jira issue</a>.</p>
 <p>+When something like that happens, I get to IRC (I use https://kiwiirc.com/nextclient/irc.freenode.net/#ofbiz, of course you can pick your own). There using a recognisable username (I use jleroux) I get to the ofbiz channel (#ofbiz, you are normally on it using the previous link). I wait for ofbiz-bot to appear in participant panel, click on it, click on query and then I can make a request in the chat line to restart a build.</p>
 <p>A request is of the form</p>
 <pre><code>force build ofbizTrunkFramework</code></pre>
 <p>you can put the text you want to appear in builder, after this expression, something like</p>
 <pre><code>forces manual build after weird error</code></pre>
 <p>So the whole request is of the form</p>
 <pre><code>force build ofbizTrunkFramework forces manual build after weird error</code></pre>
 <p>Note that with our last config (see <a href="https://issues.apache.org/jira/browse/INFRA-15394">INFRA-15394</a>) the plugins builders are dependent and automatically launched by the framework builders but only on commits. So if you use an IRC command like <code>force build ofbizTrunkFramework</code> only this builder will be launched not the dependent ofbizTrunkFrameworkPlugins. We can’t call a scheduler from IRC. It needs a Git commit.</p>
 <h3 id="random-conflicts-on-port-8080-during-tests">Random conflicts on port 8080 during tests</h3>
 <p>One case which comes back from time to time is a conflit on port 8080 due to the automatic startup of tomcat. It’s due to security patches being applied on one of 3 the servers BuildBot uses for OFBiz, hence the random aspect. In such case we need to ask infra to manually disable Tomcat on this server. This happened 4th times already, last case was <a href="https://issues.apache.org/jira/browse/INFRA-15829">INFRA-15829</a> where things are best explained.</p>
 <h3 id="creating-a-new-branch">Creating a new branch</h3>
 <ol type="1">
 <li>Copy and adapt an existing branch scheduler and builder in <a href="https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf">the OFBiz BuildBot script</a>.</li>
 <li>Ask Infra for a new directories structure for the tests (eg https://issues.apache.org/jira/browse/INFRA-17513)<br />
 </li>
 <li>Adds the new branches <a href="https://github.com/apache/infrastructure-p6/blob/production/modules/subversion_server/files/hooks/buildbot_project_paths">in the Git file for project paths</a>. You can’t clone, but you can make PRs inside it AIUI, via the github UI, just make a new branch and PR it.</li>
 <li>Remove old branches if no longer supported.</li>
 </ol>
 </body>
 </html>
	<!DOCTYPE html>
	<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
	<head>
	<meta charset="utf-8" />
	<meta name="generator" content="pandoc" />
	<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
	<title>BuildBot</title>
	<style type="text/css">
	code{white-space: pre-wrap;}
	span.smallcaps{font-variant: small-caps;}
	span.underline{text-decoration: underline;}
	div.column{display: inline-block; vertical-align: top; width: 50%;}
	</style>
	<!--[if lt IE 9]>
	<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
	<![endif]-->
	</head>
	<body>
	<!--
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->
	<h2 id="what-is-builbbot-and-why-using-it">What is BuilbBot and why using it?</h2>
	<p>OFBiz uses <a href="https://en.wikipedia.org/wiki/Buildbot">BuilbBot</a> as its Continuous Integration (CI) tool. There are other CI tools <a href="https://ci.apache.org/">used at the ASF</a>. There is also <a href="https://github.com/apache/infrastructure-p6/blob/production/modules/subversion_server/files/hooks/buildbot_project_paths">a file in Git for project paths</a>. You can’t clone, but you can make PRs inside it AIUI, via the github UI, just make a new branch and PR it. More information about creating a new branch below.</p>
	<h2 id="ofbiz-and-buildbot">OFBiz and BuildBot</h2>
	<p><a href="https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf">Here is the OFBiz BuildBot script</a>. If you are interested on modifying it, please understand its content and read the BuildBot documentation. Note that the Infra team is always there to help you.</p>
	<h3 id="schedulers-and-builders">Schedulers and Builders</h3>
	<p>BuildBot uses schedulers and builders. Schedulers decide which builders to trigger on Git commits.</p>
	<p>You can see the OFBiz builders and their states in the <a href="https://ci.apache.org/waterfall">common Buildbot waterfall view</a> (look for “ofb”).</p>
	<p>The <a href="https://ci.apache.org/builders/ofbizTrunkFramework">ofbizTrunkFramework</a> builder is triggered on commits in the trunk framework branch. Then only the framework integration tests are run. This also triggers the <a href="https://ci.apache.org/builders/ofbizTrunkFrameworkPlugins">ofbizTrunkFrameworkPlugins</a> if the build succeed. Then the whole integration tests are run. To check that a commit in the framework puts no regressions in plugins and especially because the dependencies of ofbiz-framework without ofbiz-plugins is different from the dependencies of ofbiz-framework + ofbiz-plugins (plugins components are Gradle sub-projects). So testing needs to happen in both scenarios because you might face library version bugs.</p>
	<p>Of course a commit into the trunk plugins also triggers the ofbizTrunkFrameworkPlugins.</p>
	<p>The same apply to the <a href="https://ci.apache.org/builders/ofbizBranch17Framework">ofbizBranch17Framework</a> and the <a href="https://ci.apache.org/builders/ofbizBranch17FrameworkPlugins">ofbizBranch17FrameworkPlugins</a> and next releases branches, they follows the same structure than the trunk.</p>
	<p>In an OFBiz builder page you can see <a href="https://ci.apache.org/builders/ofbizTrunkFramework?numbuilds=100">an history of the builds up to 100+</a>. OFBiz randomly uses 3 servers (except for RAT which always only uses asf945_ubuntu) to build. You may want to explore more to get acquainted with the tool…</p>
	<p>There is also the <a href="https://ci.apache.org/builders/ofbizTrunkFrameworkRat">RAT builder</a>. It checks the licenses in OFBiz. You can find the result <a href="https://ci.apache.org/projects/ofbiz/site/trunk/rat-output.html">here</a>. There are also RAT builders for the current <a href="https://ci.apache.org/projects/ofbiz/site/next/rat-output.html">next</a> and <a href="https://ci.apache.org/projects/ofbiz/site/stable/rat-output.html">stable</a> release branches.</p>
	<h3 id="tests-results">tests results</h3>
	<p>Here are the <a href="https://ci.apache.org/projects/ofbiz/logs/">test results</a>. The folders structure reflects the current Git repo structure.</p>
	<h3 id="technical-information">Technical information</h3>
	<p>When you create a new branch you need to let know BuildBot about it. This because BuildBot uses Git hooks to triggers builds on commits. <a href="https://github.com/apache/infrastructure-p6/blob/production/modules/subversion_server/files/hooks/buildbot_project_paths">The file to change is here</a>. You can’t clone, but you can make PRs inside it AIUI, via the github UI, just make a new branch and PR it.</p>
	<h2 id="handling-issues">Handling issues</h2>
	<p>Sometimes (rarely) you can get transient tests errors in BuildBot. This mean tests don’t all pass in BuildBot, though they pass in your local instance. In such case, it’s most certainly an issue with servers. Those are hard workers and make errors from time to time, which shows that not only human make errors.</p>
	<p>Before doing anything it’s best to check which BuildBot step is impacted, and if it exists have a look at the logfile (stdio)</p>
	<p>Some other errors may happen, like * Git not updating * upload not working * you name it…</p>
	<p>In such case you can trigger a build from IRC to see if the problem resolves by itself. Most of the time tests and G issues are resolved this way. If it does not then the best is to ask Infra help, either on <a href="https://the-asf.slack.com/archives/CBX4TSBQ8">Infra Slack channel</a> or by creating an <a href="https://issues.apache.org/jira/projects/INFRA/summary">Infra Jira issue</a>.</p>
	<p>+When something like that happens, I get to IRC (I use https://kiwiirc.com/nextclient/irc.freenode.net/#ofbiz, of course you can pick your own). There using a recognisable username (I use jleroux) I get to the ofbiz channel (#ofbiz, you are normally on it using the previous link). I wait for ofbiz-bot to appear in participant panel, click on it, click on query and then I can make a request in the chat line to restart a build.</p>
	<p>A request is of the form</p>
	<pre><code>force build ofbizTrunkFramework</code></pre>
	<p>you can put the text you want to appear in builder, after this expression, something like</p>
	<pre><code>forces manual build after weird error</code></pre>
	<p>So the whole request is of the form</p>
	<pre><code>force build ofbizTrunkFramework forces manual build after weird error</code></pre>
	<p>Note that with our last config (see <a href="https://issues.apache.org/jira/browse/INFRA-15394">INFRA-15394</a>) the plugins builders are dependent and automatically launched by the framework builders but only on commits. So if you use an IRC command like <code>force build ofbizTrunkFramework</code> only this builder will be launched not the dependent ofbizTrunkFrameworkPlugins. We can’t call a scheduler from IRC. It needs a Git commit.</p>
	<h3 id="random-conflicts-on-port-8080-during-tests">Random conflicts on port 8080 during tests</h3>
	<p>One case which comes back from time to time is a conflit on port 8080 due to the automatic startup of tomcat. It’s due to security patches being applied on one of 3 the servers BuildBot uses for OFBiz, hence the random aspect. In such case we need to ask infra to manually disable Tomcat on this server. This happened 4th times already, last case was <a href="https://issues.apache.org/jira/browse/INFRA-15829">INFRA-15829</a> where things are best explained.</p>
	<h3 id="creating-a-new-branch">Creating a new branch</h3>
	<ol type="1">
	<li>Copy and adapt an existing branch scheduler and builder in <a href="https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/ofbiz.conf">the OFBiz BuildBot script</a>.</li>
	<li>Ask Infra for a new directories structure for the tests (eg https://issues.apache.org/jira/browse/INFRA-17513)<br />
	</li>
	<li>Adds the new branches <a href="https://github.com/apache/infrastructure-p6/blob/production/modules/subversion_server/files/hooks/buildbot_project_paths">in the Git file for project paths</a>. You can’t clone, but you can make PRs inside it AIUI, via the github UI, just make a new branch and PR it.</li>
	<li>Remove old branches if no longer supported.</li>
	</ol>
	</body>
	</html>