src/main/webapp/contribute-post-commit-test-policies.html

---
title: Contribute | Post-commit test policies
layout: default
---

      <!-- Marketing messaging and featurettes
      ================================================== -->
      <!-- Wrap the rest of the page in another container to center all the content. -->

      <div class="container">
        <!-- CONTRIBUTE -->
        <div class="row mb-5 mt-5 pt-3">
          <div class="col-2 sidenav-menu pt-3">
            <div class="pb-3 text-uppercase"><strong>Contribute</strong></div>
            <ul class="list-unstyled mb-3">
              <li><a href="contribute.html">Get started contributing</a></li>
              <li class="sidenav-menu-sub-title">Technical Docs</li>
              <li>
                <ul class="list-unstyled sidenav-menu-sub-list">
                  <li><a href="contribute-testing.html">Testing guide</a></li>
                  <li><a href="contribute-dependencies-guide.html">Dependencies guide</a></li>
                </ul>
              </li>
              <li class="sidenav-menu-sub-title">Policies</li>
              <li>
                <ul class="list-unstyled sidenav-menu-sub-list">
                  <li><a href="contribute-pre-commit-test-policies.html">Pre-commit test policies</a></li>
                  <li><a href="contribute-post-commit-test-policies.html">Post-commit test policies</a></li>
                </ul>
              </li>
              <li class="sidenav-menu-sub-title">Committers</li>
              <li>
                <ul class="list-unstyled sidenav-menu-sub-list">
                  <li><a href="contribute-become-a-committer.html">Become a committer</a></li>
                  <li><a href="contribute-committer-guide.html">Committer guide</a></li>
                  <li><a href="contribute-release-guide.html">Release guide</a></li>
                </ul>
              </li>
            </ul>
          </div>
          <div class="col-8 pt-3 offset-4">
            <h2>Post-commit tests policies</h2>

            <p>Post-commit tests validate that Unomi works correctly in a live environment. The tests also catch errors that are hard to predict in the design and implementation stages</p>

            <p>Even though post-commit tests run after the code is merged into the repository,  it is important that the tests pass reliably. Jenkins executes post-commit tests against the HEAD of the master branch. If post-commit tests fail, there is a problem with the HEAD build. In addition, post-commit tests are time consuming to run, and it is often hard to triage test failures.</p>

            <h3>Policies</h3>

            <p>To ensure that Unomi's post-commit tests are reliable and healthy, the Unomi community follows these post-commit test policies:</p>

            <ul>
              <li><a href="contribute-post-commit-policies-details.html#rollback_first">Rollback first</a></li>
              <li><a href="contribute-post-commit-policies-details.html#failing_test_is_critical_bug">A failing test is a critical bug</a></li>
              <li><a href="contribute-post-commit-policies-details.html#flake_is_failing">A flaky test is a critical bug</a></li>
              <li><a href="contribute-post-commit-policies-details.html#remove_flake">Flaky tests must either be fixed or removed</a></li>
              <li><a href="contribute-post-commit-policies-details.html#precommit_for_postcommit">Fixes for post-commit failures should include a corresponding new pre-commit test</a></li>
            </ul>

            <h3>Post-commit test failure scenarios</h3>

            <p>When a post-commit test fails, follow the provided steps for your situation.</p>

            <h4>I found a test failure</h4>

            <ol>
              <li>Create a <a target="_blank" href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20UNOMI%20AND%20component%20%3D%20test-failures">JIRA issue</a> and assign it to yourself.</li>
              <li>Do high level triage of the failure.</li>
              <li><a href="contribute-post-commit-guides.html#find_specialist">Assign the JIRA issue to a relevant person</a>.</li>
            </ol>

            <h4>I was assigned a JIRA issue for a test failure</h4>

            <ol>
              <li><a href="contribute-post-commit-guides.html#rollback">Rollback the culprit change</a>.</li>
              <li>If you determine that rollback will take longer than 8 hours, <a href="contribute-post-commit-guides.html#disabling">disable the test temporarily</a> while you rollback or create a fix.</li>
            </ol>

            <blockquote>
              <p>Note: Rollback is always the first course of action. If a fix is trivial, open a pull request with the proposed fix while doing rollback.</p>
            </blockquote>

            <h4>My change was rolled back due to a test failure</h4>

            <ol>
              <li>Look at the JIRA issue to find the reason for the rollback.</li>
              <li>Fix your code and re-run the post-commit tests.</li>
              <li>Implement new pre-commit tests that will catch similar bugs before future code is merged into the repository.</li>
              <li>Open a new PR that contains your fix and the new pre-commit tests.</li>
            </ol>

            <h3>Useful links</h3>

            <ul>
              <li><a href="contribute-testing.html#best_practices">Best practices for writing tests</a></li>
            </ul>
          </div>
        </div>
      </div>