documentation/0.16.0-SNAPSHOT/precommit/architecture/index.html - yetus - Git at Google

 <!---
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.
 -->

 <html>
     <head>
         <meta charset="utf-8">
         <title>Apache Yetus</title>
         <meta name="viewport" content="width=device-width, initial-scale=1.0">
         <meta name="description" content="">
         <meta name="author" content="">

         <link href="../../../../assets/css/bootstrap.css" rel="stylesheet">
         <link href="../../../../assets/css/bootstrap-theme.css" rel="stylesheet">
         <link href="../../../../assets/css/font-awesome.css" rel="stylesheet">

         <!-- JS -->
         <script type="text/javascript" src="../../../../assets/js/jquery-2.1.4.min.js"></script>
         <script type="text/javascript" src="../../../../assets/js/bootstrap.js"></script>
   </head>
     <body>

 <div class="navbar navbar-inverse navbar-static-top" role="navigation">
     <div class="container">
         <div class="navbar-header">
             <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
                 <span class="sr-only">Toggle navigation</span>
                 <span class="icon-bar"></span>
                 <span class="icon-bar"></span>
                 <span class="icon-bar"></span>
             </button>
             <a class="img-responsive pull-left" href="/">
                 <img style="max-height: 40px; margin-top: 5px; margin-bottom: 5px;" src="../../../../assets/img/yetus_logo.png" alt="Apache Yetus logo" />
             </a>
         </div>
         <div class="navbar-collapse collapse">
             <ul class="nav navbar-nav">
                 <li><a href="/downloads/">Downloads</a>
                 <li class="dropdown">
                     <a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation <span class="caret"></span></a>
                     <ul class="dropdown-menu" role="menu">
                       <li><a href="/documentation/0.13.0/">Docs for v0.13.0</a></li>
                       <li><a href="/documentation/0.14.1/">Docs for v0.14.1</a></li>
                       <li><a href="/documentation/0.15.0/">Docs for v0.15.0</a></li>
                       <li><a href="/documentation/in-progress/">In Progress Docs for Contributors</a>
                       </li>
                       <li><a href="/documentation/history/">History of the Project</a>
                       </li>
                     </ul>
                 </li>
                 <li class="dropdown">
                     <a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a>
                     <ul class="dropdown-menu" role="menu" aria-labelledby="drop1">
                         <li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a>
                         </li>
                         <li role="presentation"><a role="menuitem" tabindex="-1" href="https://issues.apache.org/jira/browse/YETUS"><i class="fa fa-bug"></i> JIRA (Bugs)</a>
                         </li>
                         <li role="presentation"><a role="menuitem" tabindex="-1" href="https://gitbox.apache.org/repos/asf/yetus.git"><i class="fa fa-code"></i> Source (Apache)</a>
                         </li>
                         <li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus"><i class="fa fa-github-alt"></i> Source (GitHub)</a>
                         </li>
                         <li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute/"><i class="fa fa-code-fork"></i> Contributing</a>
                         </li>
                         <li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus"><i class="fa fa-twitter"></i> @ApacheYetus</a>
                         </li>
                     </ul>
                 </li>
                 <li>
                     <li class="dropdown">
                         <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
                         <ul class="dropdown-menu" role="menu">
                             <li><a href="https://www.apache.org">Apache Homepage</a>
                             </li>
                             <li><a href="https://www.apache.org/licenses/">Apache License</a>
                             </li>
                             <li><a href="https://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
                             </li>
                             <li><a href="https://www.apache.org/foundation/thanks.html">Thanks</a>
                             </li>
                             <li><a href="https://www.apache.org/security/">Security</a>
                             </li>
                         </ul>
                     </li>
                 </li>
             </ul>
         </div>
         <!--/.nav-collapse -->
     </div>
 </div>

       <div class="container">
         <!---
   Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.
 -->

 <h1 id="phases">Phases</h1>

 <!-- MarkdownTOC levels="1,2" autolink="true" indent="  " bullets="*" bracket="round" -->

 <ul>
   <li><a href="#initialize">Initialize</a></li>
   <li><a href="#precheck">Precheck</a></li>
   <li><a href="#patch-file-tests">Patch File Tests</a></li>
   <li><a href="#compile-cycle-branch">Compile Cycle (Branch)</a></li>
   <li><a href="#distribution-clean">Distribution Clean</a></li>
   <li><a href="#patch-application">Patch Application</a></li>
   <li><a href="#compile-cycle-patch">Compile Cycle (Patch)</a></li>
   <li><a href="#unit-tests">Unit Tests</a></li>
   <li><a href="#reporting">Reporting</a></li>
   <li><a href="#test-flow">Test Flow</a></li>
 </ul>

 <!-- /MarkdownTOC -->

 <p>test-patch works effectively under several different phases:</p>

 <h2 id="initialize">Initialize</h2>

 <p>This is where test-patch configures and validates the environment.<br />
 Some things done in this phase:</p>

 <ul>
   <li>Defaults</li>
   <li>Parameter handling</li>
   <li>Importing plug-ins and personalities</li>
   <li>Docker container launching</li>
   <li>Re-exec support</li>
   <li>Patch file downloading</li>
   <li>git repository management (fresh pull, branch switching, etc)</li>
 </ul>

 <h2 id="precheck">Precheck</h2>

 <p>Checks done here are <em>fatal</em>.</p>

 <p>This acts as a verification of all of the setup parts and is the final<br />
 place to short-cut the full test cycle.  The most significant built-in<br />
 check done here is verifying the patch file is a valid.</p>

 <h2 id="patch-file-tests">Patch File Tests</h2>

 <p>Tests that only require the patch file are run.  Note that the repository<br />
 is still from the initial checkout!</p>

 <h2 id="compile-cycle-branch">Compile Cycle (Branch)</h2>

 <p>When compilation must be done, we follow these five steps:</p>

 <ul>
   <li>The list of modules that require analysis is built.</li>
   <li>A precompile step to set things up for the actual compile</li>
   <li>The actual compile</li>
   <li>A postcompile to do analysis on the output of that compile phase</li>
   <li>A rebuild phase to run tests that require recompiles</li>
 </ul>

 <p>The first time this is done is with the pristine checkout.  This is called<br />
 the "branch compile".  For this pass, this is where the 'before' work is<br />
 handled.  Some things that typically get checked in this phase:</p>

 <ul>
   <li>The first pass of files and modules that will get patched</li>
   <li>Validation and information gathering of the source tree pre-patch</li>
   <li>javadoc, scaladoc, etc</li>
 </ul>

 <h2 id="distribution-clean">Distribution Clean</h2>

 <p>This step is to wipe the repository clean back to a pristine state such that<br />
 the previous cycle will not impact the next cycle.</p>

 <h2 id="patch-application">Patch Application</h2>

 <p>The patch gets applied.</p>

 <h2 id="compile-cycle-patch">Compile Cycle (Patch)</h2>

 <p>Now that the patch has been applied the steps to compile we outlined in the<br />
 compilation (branch) phase are repeated but with the patch applied. This is<br />
 where a lot of 'after' checks are performed.</p>

 <h2 id="unit-tests">Unit Tests</h2>

 <p>Since unit tests are generally the slowest part of the precommit process, they<br />
 are run last.  At this point, all the prerequisites to running them should be<br />
 in place and ready to go.</p>

 <h2 id="reporting">Reporting</h2>

 <p>Finally, the results are reported to the screen and, optionally, to JIRA<br />
 and/or whatever bug system has been configured.</p>

 <h1 id="test-flow">Test Flow</h1>

 <p>The basic workflow for many of the sub-items in individual phases are:</p>

 <ol>
   <li>print a header, so the end user knows that something is happening</li>
   <li>verify if the test is needed.  If so, continue on.  Otherwise,<br />
 return success and let the next part of the phase execute.</li>
   <li>Ask the personality about what modules and what flags should get used</li>
   <li>Execute maven (or some other build tool) in the given modules with the<br />
 given flags. Log the output and record the time and result code.</li>
   <li>Do any extra work as appropriate (diffs, counts, etc) and either accept<br />
 the status and message given by the maven run or change the vote,<br />
 message, log file, etc, based upon this extra work.</li>
   <li>Add the outcome(s) to the report generator</li>
 </ol>

 <p>As one can see, the modules list is one of the key inputs into what actually<br />
 gets executed.  As a result, projects must full flexibility in either adding,<br />
 modifying, or even removing modules from the test list.  If a personality<br />
 removes the entire list of modules, then that test should just be ignored.</p>

     </div>

 <div class="container">
     <hr>
     <footer class="footer">
         <div class="row-fluid">
             <div class="span12 text-left">
               <div class="span12">
                 Copyright 2008-2023 <a href="https://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="https://www.apache.org/licenses/">Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation.
               </div>
             </div>

         </div>

     </footer>
 </div>

   </body>
 </html>
	<!---
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->

	<html>
	<head>
	<meta charset="utf-8">
	<title>Apache Yetus</title>
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<meta name="description" content="">
	<meta name="author" content="">

	<link href="../../../../assets/css/bootstrap.css" rel="stylesheet">
	<link href="../../../../assets/css/bootstrap-theme.css" rel="stylesheet">
	<link href="../../../../assets/css/font-awesome.css" rel="stylesheet">

	<!-- JS -->
	<script type="text/javascript" src="../../../../assets/js/jquery-2.1.4.min.js"></script>
	<script type="text/javascript" src="../../../../assets/js/bootstrap.js"></script>
	</head>
	<body>

	<div class="navbar navbar-inverse navbar-static-top" role="navigation">
	<div class="container">
	<div class="navbar-header">
	<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-collapse">
	<span class="sr-only">Toggle navigation</span>
	<span class="icon-bar"></span>
	<span class="icon-bar"></span>
	<span class="icon-bar"></span>
	</button>
	<a class="img-responsive pull-left" href="/">
	<img style="max-height: 40px; margin-top: 5px; margin-bottom: 5px;" src="../../../../assets/img/yetus_logo.png" alt="Apache Yetus logo" />
	</a>
	</div>
	<div class="navbar-collapse collapse">
	<ul class="nav navbar-nav">
	<li><a href="/downloads/">Downloads</a>
	<li class="dropdown">
	<a class="dropdown-toggle" data-toggle="dropdown" href="#">Documentation <span class="caret"></span></a>
	<ul class="dropdown-menu" role="menu">
	<li><a href="/documentation/0.13.0/">Docs for v0.13.0</a></li>
	<li><a href="/documentation/0.14.1/">Docs for v0.14.1</a></li>
	<li><a href="/documentation/0.15.0/">Docs for v0.15.0</a></li>
	<li><a href="/documentation/in-progress/">In Progress Docs for Contributors</a>
	</li>
	<li><a href="/documentation/history/">History of the Project</a>
	</li>
	</ul>
	</li>
	<li class="dropdown">
	<a class="dropdown-toggle" data-toggle="dropdown" href="#">Get Involved <span class="caret"></span></a>
	<ul class="dropdown-menu" role="menu" aria-labelledby="drop1">
	<li role="presentation"><a role="menuitem" tabindex="-1" href="/mailinglists"><i class="fa fa-commenting"></i> Mailing Lists</a>
	</li>
	<li role="presentation"><a role="menuitem" tabindex="-1" href="https://issues.apache.org/jira/browse/YETUS"><i class="fa fa-bug"></i> JIRA (Bugs)</a>
	</li>
	<li role="presentation"><a role="menuitem" tabindex="-1" href="https://gitbox.apache.org/repos/asf/yetus.git"><i class="fa fa-code"></i> Source (Apache)</a>
	</li>
	<li role="presentation"><a role="menuitem" tabindex="-1" href="https://github.com/apache/yetus"><i class="fa fa-github-alt"></i> Source (GitHub)</a>
	</li>
	<li role="presentation"><a role="menuitem" tabindex="-1" href="/contribute/"><i class="fa fa-code-fork"></i> Contributing</a>
	</li>
	<li role="presentation"><a role="menuitem" tabindex="-1" href="https://twitter.com/ApacheYetus"><i class="fa fa-twitter"></i> @ApacheYetus</a>
	</li>
	</ul>
	</li>
	<li>
	<li class="dropdown">
	<a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
	<ul class="dropdown-menu" role="menu">
	<li><a href="https://www.apache.org">Apache Homepage</a>
	</li>
	<li><a href="https://www.apache.org/licenses/">Apache License</a>
	</li>
	<li><a href="https://www.apache.org/foundation/sponsorship.html">Sponsorship</a>
	</li>
	<li><a href="https://www.apache.org/foundation/thanks.html">Thanks</a>
	</li>
	<li><a href="https://www.apache.org/security/">Security</a>
	</li>
	</ul>
	</li>
	</li>
	</ul>
	</div>
	<!--/.nav-collapse -->
	</div>
	</div>

	<div class="container">
	<!---
	Licensed to the Apache Software Foundation (ASF) under one
	or more contributor license agreements. See the NOTICE file
	distributed with this work for additional information
	regarding copyright ownership. The ASF licenses this file
	to you under the Apache License, Version 2.0 (the
	"License"); you may not use this file except in compliance
	with the License. You may obtain a copy of the License at

	http://www.apache.org/licenses/LICENSE-2.0

	Unless required by applicable law or agreed to in writing,
	software distributed under the License is distributed on an
	"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
	KIND, either express or implied. See the License for the
	specific language governing permissions and limitations
	under the License.
	-->

	<h1 id="phases">Phases</h1>

	<!-- MarkdownTOC levels="1,2" autolink="true" indent=" " bullets="*" bracket="round" -->

	<ul>
	<li><a href="#initialize">Initialize</a></li>
	<li><a href="#precheck">Precheck</a></li>
	<li><a href="#patch-file-tests">Patch File Tests</a></li>
	<li><a href="#compile-cycle-branch">Compile Cycle (Branch)</a></li>
	<li><a href="#distribution-clean">Distribution Clean</a></li>
	<li><a href="#patch-application">Patch Application</a></li>
	<li><a href="#compile-cycle-patch">Compile Cycle (Patch)</a></li>
	<li><a href="#unit-tests">Unit Tests</a></li>
	<li><a href="#reporting">Reporting</a></li>
	<li><a href="#test-flow">Test Flow</a></li>
	</ul>

	<!-- /MarkdownTOC -->

	<p>test-patch works effectively under several different phases:</p>

	<h2 id="initialize">Initialize</h2>

	<p>This is where test-patch configures and validates the environment.<br />
	Some things done in this phase:</p>

	<ul>
	<li>Defaults</li>
	<li>Parameter handling</li>
	<li>Importing plug-ins and personalities</li>
	<li>Docker container launching</li>
	<li>Re-exec support</li>
	<li>Patch file downloading</li>
	<li>git repository management (fresh pull, branch switching, etc)</li>
	</ul>

	<h2 id="precheck">Precheck</h2>

	<p>Checks done here are <em>fatal</em>.</p>

	<p>This acts as a verification of all of the setup parts and is the final<br />
	place to short-cut the full test cycle. The most significant built-in<br />
	check done here is verifying the patch file is a valid.</p>

	<h2 id="patch-file-tests">Patch File Tests</h2>

	<p>Tests that only require the patch file are run. Note that the repository<br />
	is still from the initial checkout!</p>

	<h2 id="compile-cycle-branch">Compile Cycle (Branch)</h2>

	<p>When compilation must be done, we follow these five steps:</p>

	<ul>
	<li>The list of modules that require analysis is built.</li>
	<li>A precompile step to set things up for the actual compile</li>
	<li>The actual compile</li>
	<li>A postcompile to do analysis on the output of that compile phase</li>
	<li>A rebuild phase to run tests that require recompiles</li>
	</ul>

	<p>The first time this is done is with the pristine checkout. This is called<br />
	the "branch compile". For this pass, this is where the 'before' work is<br />
	handled. Some things that typically get checked in this phase:</p>

	<ul>
	<li>The first pass of files and modules that will get patched</li>
	<li>Validation and information gathering of the source tree pre-patch</li>
	<li>javadoc, scaladoc, etc</li>
	</ul>

	<h2 id="distribution-clean">Distribution Clean</h2>

	<p>This step is to wipe the repository clean back to a pristine state such that<br />
	the previous cycle will not impact the next cycle.</p>

	<h2 id="patch-application">Patch Application</h2>

	<p>The patch gets applied.</p>

	<h2 id="compile-cycle-patch">Compile Cycle (Patch)</h2>

	<p>Now that the patch has been applied the steps to compile we outlined in the<br />
	compilation (branch) phase are repeated but with the patch applied. This is<br />
	where a lot of 'after' checks are performed.</p>

	<h2 id="unit-tests">Unit Tests</h2>

	<p>Since unit tests are generally the slowest part of the precommit process, they<br />
	are run last. At this point, all the prerequisites to running them should be<br />
	in place and ready to go.</p>

	<h2 id="reporting">Reporting</h2>

	<p>Finally, the results are reported to the screen and, optionally, to JIRA<br />
	and/or whatever bug system has been configured.</p>

	<h1 id="test-flow">Test Flow</h1>

	<p>The basic workflow for many of the sub-items in individual phases are:</p>

	<ol>
	<li>print a header, so the end user knows that something is happening</li>
	<li>verify if the test is needed. If so, continue on. Otherwise,<br />
	return success and let the next part of the phase execute.</li>
	<li>Ask the personality about what modules and what flags should get used</li>
	<li>Execute maven (or some other build tool) in the given modules with the<br />
	given flags. Log the output and record the time and result code.</li>
	<li>Do any extra work as appropriate (diffs, counts, etc) and either accept<br />
	the status and message given by the maven run or change the vote,<br />
	message, log file, etc, based upon this extra work.</li>
	<li>Add the outcome(s) to the report generator</li>
	</ol>

	<p>As one can see, the modules list is one of the key inputs into what actually<br />
	gets executed. As a result, projects must full flexibility in either adding,<br />
	modifying, or even removing modules from the test list. If a personality<br />
	removes the entire list of modules, then that test should just be ignored.</p>

	</div>

	<div class="container">
	<hr>
	<footer class="footer">
	<div class="row-fluid">
	<div class="span12 text-left">
	<div class="span12">
	Copyright 2008-2023 <a href="https://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="https://www.apache.org/licenses/">Apache License v2.0</a>. Apache Yetus and the Apache feather logo are trademarks of The Apache Software Foundation.
	</div>
	</div>

	</div>

	</footer>
	</div>

	</body>
	</html>