content/verification.html - apex-site - Git at Google

 <html lang="en"><head>

     <meta charset="utf-8">
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=1">
     <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
     <meta name="description" content="Apex is an enterprise grade native YARN big data-in-motion platform that unifies stream processing as well as batch processing.">
     <meta name="author" content="Apache Software Foundation">
     <link rel="icon" href="favicon.ico">

     <title>Apache Apex</title>

     <!-- Main Stylesheet -->
     <link href="css/main.css" rel="stylesheet">

     <script>
       (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
         (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
         m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
       })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

       ga('create', 'UA-85540278-1', 'auto');
       ga('send', 'pageview');
     </script>

   </head>

   <body>
     <nav class="navbar navbar-default navbar-static-top" id="main-nav">
       <div class="container">

       <div class="navbar-header">
         <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false">
           <span class="sr-only">Toggle navigation</span>
           <span class="icon-bar"></span>
           <span class="icon-bar"></span>
           <span class="icon-bar"></span>
         </button>
         <a class="navbar-brand" href="/">
           <img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo">
           Apache Apex<span class="trademark">&trade;</span>
         </a>
       </div>

       <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
         <ul class="nav navbar-right navbar-nav">
           <li class="nav-item">
             <a class="nav-link " href="/">Home</a>
           </li>
           <li class="nav-item">
             <a class="nav-link " href="/docs.html">Documentation</a>
           </li>
           <li class="nav-item">
             <a class="nav-link " href="/powered-by-apex.html">Powered By Apex</a>
           </li>
           <li class="nav-item">
             <a class="nav-link " href="/roadmap.html">Roadmap</a>
           </li>

             <li class="nav-item nav-mouseover">
               <ul class="dropdown-menu">
                 <li class=""><a href="/community.html#mailing-lists">Mailing Lists</a></li>
                 <li class=""><a href="/community.html#issue-tracking">Issue Tracking</a></li>
                 <li class=""><a href="http://stackoverflow.com/questions/tagged/apache-apex">Stack Overflow</a></li>
                 <li class=""><a href="/community.html#events">Events</a></li>
                 <li class=""><a href="/community.html#contributing">Contributing</a></li>
                 <li class=""><a href="http://www.apache.org/foundation/how-it-works.html">Apache Foundation</a></li>
               </ul>
               <a href="/community.html" class="nav-link">Community</a>
             </li>


             <li class="nav-item nav-mouseover">
               <ul class="dropdown-menu">
                 <li class=""><a href="https://github.com/apache/apex-core">Apex Core</a></li>
                 <li class=""><a href="https://github.com/apache/apex-malhar">Apex Malhar</a></li>
                 <li class=""><a href="https://github.com/apache/apex-site">Apex Site</a></li>
               </ul>
               <a href="https://github.com/apache/apex-core" class="nav-link">Github</a>
             </li>

           <li class="nav-item">
             <a class="nav-link btn btn-success btn-download" href="/downloads.html">Download</a>
           </li>
         </ul>

       </div>
     </nav>

 <div class="container">

   <h1 id="how-to-verify-apache-apex-release-candidate-builds">How to verify Apache Apex release candidate builds</h1>
 <h2 id="prerequisites">Prerequisites</h2>
 <p>Download the KEYS file and import it.  Verify that the key just imported is correct.</p>
 <pre><code class="lang-bash">wget https://dist.apache.org/repos/dist/release/apex/KEYS
 gpg --import KEYS
 gpg --fingerprint &lt;key-id&gt;
 </code></pre>
 <p>The &lt;key-id&gt; of the release manager can be found out in the <em>PGP key</em> link in the mail. It would be alphanumric number like D920A98Cbb. The above step can be skipped if you have already imported key of the current release manager.</p>
 <p>More about key verification at <a href="http://www.apache.org/info/verification.html">http://www.apache.org/info/verification.html</a></p>
 <p>You can also create your own key which is required if you would like to sign the build. This step is optional.</p>
 <pre><code class="lang-bash">gpg --gen-key
 </code></pre>
 <p>Please provide 4096 as keysize while generating the key.</p>
 <h2 id="file-integrity-check">File integrity check</h2>
 <p>Download all files present in staging directory of the RC build. Staging directory link is shared in VOTE thread of the release candidate. It is recommended that these files are downloaded in a clean directory.</p>
 <pre><code class="lang-bash">wget -r -np -nd &lt;staging-area-link&gt;/
 </code></pre>
 <p>Note the link should end with &quot;/&quot;.</p>
 <p>Define the apex release candidate variable. We will use <em>apache-apex-core-3.6.0</em> as an example.</p>
 <pre><code class="lang-bash">APEX_RELEASE_CANDIDATE=apache-apex-core-3.6.0
 </code></pre>
 <p>Verify integrity of tar.gz file:</p>
 <pre><code class="lang-bash">gpg --verify $APEX_RELEASE_CANDIDATE-source-release.tar.gz.asc
 md5sum --check $APEX_RELEASE_CANDIDATE-source-release.tar.gz.md5
 shasum -a 512 --check $APEX_RELEASE_CANDIDATE-source-release.tar.gz.sha512
 </code></pre>
 <p>Verify integrity of .zip file:</p>
 <pre><code class="lang-bash">gpg --verify $APEX_RELEASE_CANDIDATE-source-release.zip.asc
 md5sum --check $APEX_RELEASE_CANDIDATE-source-release.zip.md5
 shasum -a 512  --check $APEX_RELEASE_CANDIDATE-source-release.zip.sha512
 </code></pre>
 <h2 id="source-code-verification">Source code verification</h2>
 <p>You can extract source either using .tar.gz file or .zip file.</p>
 <h3 id="using-tar-gz-source">Using .tar.gz source</h3>
 <p>Extract source using .tar.gz:</p>
 <pre><code class="lang-bash">tar -zxvf $APEX_RELEASE_CANDIDATE-source-release.tar.gz
 </code></pre>
 <h3 id="using-zip-source">Using .zip source</h3>
 <pre><code class="lang-bash">unzip $APEX_RELEASE_CANDIDATE-source-release.zip
 </code></pre>
 <p>Any of the two commands above will create a directory named after <em>apex release candidate</em>.</p>
 <p>Change directory:</p>
 <pre><code class="lang-bash">cd $APEX_RELEASE_CANDIDATE
 </code></pre>
 <h3 id="miscellaneous-checks-part-i-pre-compilation-">Miscellaneous checks: Part I (Pre Compilation)</h3>
 <p>Existence of LICENSE, NOTICE, README.md and CHANGELOG.md files.
 Please make sure these files are present in the parent folder after extracting the zip or tar. The CHANGELOG.md file should contain change log for the current release.</p>
 <p>No unexpected binary files in the sources.
 The extracted directory should not contain any binary. Need to test this before compiling the source code.</p>
 <pre><code class="lang-bash">find . -type f -name &#39;*.*&#39; | sed &#39;s|.*\.||&#39; | sort -u
 </code></pre>
 <p>Please make sure no binary extension is listed here. Likely candidates are .jar files.  Additionally, for Malhar, demo package files, i.e. .apa files.</p>
 <h3 id="check-for-compilation-license-headers-etc-">Check for compilation, license headers, etc.</h3>
 <p>For Apex Core:</p>
 <pre><code class="lang-bash">mvn clean apache-rat:check verify -Dlicense.skip=false install
 </code></pre>
 <p>For Apex Malhar:</p>
 <pre><code class="lang-bash">mvn clean apache-rat:check verify -Dlicense.skip=false -Pall-modules install
 </code></pre>
 <p>Jars should be installed in your maven repository, typically in <em>~/.m2/repository/org/apache/apex/</em> directory</p>
 <p>Following step is optional and needs the key generated by you as in prerequisite section given above.</p>
 <pre><code class="lang-bash">mvn verify -Papache-release -DskipTests
 </code></pre>
 <h2 id="launch-demos">Launch demos</h2>
 <p>For verification of Apex build, check backward compatibility with respect to Malhar. Locally update <em>apex.core.version</em> in Malhar to point to <em>to-be released</em> version of apex-core and recompile Malhar.</p>
 <p>Launch a few demos to make sure everything is working fine using <em>apex</em> utility. The apex script is present at <apex-core-folder>/engine/src/main/scripts/apex . If apex-core is being verified, then simply use engine/src/main/scripts/apex to launch the script. Otherwise, prefix it with apex-core folder path.</p>
 <pre><code class="lang-bash">engine/src/main/scripts/apex
 </code></pre>
 <p>You will get apex prompt, where demos can be launched.</p>
 <pre><code class="lang-bash">apex&gt; launch &lt;demo-apa-file&gt;
 </code></pre>
 <p>Demo apa files are typically in apex-malhar/demos directory.</p>
 <h2 id="miscellaneous-checks-part-ii-post-compilation-">Miscellaneous checks: Part II (Post compilation)</h2>
 <p>Correct artifact names:
 All the apex artifacts generated in local maven build directory should contain classs jar, sources jars, javadocs jar. All the artifacts are under <em>~/.m2/repository/org/apache/apex/</em> directory on your local machine.</p>

 </div>

   <hr>
   <div class="container">
     <footer id="main-footer">
       <p>
         Copyright &copy; <span id="copyright-year">2015</span> <a href="http://apache.org">The Apache Software Foundation</a>,
         Licensed under the Apache License, Version 2.0<br>
         Apache and the Apache feather logo are trademarks of The Apache Software Foundation. | <a href="/privacy.html">Privacy Policy</a><br>
         <a class="footer-link-img" href="http://apache.org"><img src="/images/asf_logo.svg" alt="The Apache Software Foundation"></a>
       </p>
     </footer>
   </div> <!-- /container -->

   <!-- Placed at the end of the document so the pages load faster -->
   <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
   <script src="/js/bootstrap.min.js"></script>
   <script src="js/docs.js"></script>

   <script>
     $('#copyright-year').text((new Date()).getFullYear());

     $('.btn-download').click(function(e){
       ga('send', {
         hitType: 'event',
         eventCategory: 'Download Button',
         eventAction: 'Button Click',
         eventLabel: 'Page Header'
       });
     });

   </script>

 </body>
 </html>
	<html lang="en"><head>

	<meta charset="utf-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<!-- The above 3 meta tags must come first in the head; any other head content must come after these tags -->
	<meta name="description" content="Apex is an enterprise grade native YARN big data-in-motion platform that unifies stream processing as well as batch processing.">
	<meta name="author" content="Apache Software Foundation">
	<link rel="icon" href="favicon.ico">

	<title>Apache Apex</title>

	<!-- Main Stylesheet -->
	<link href="css/main.css" rel="stylesheet">

	<script>
	(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]\|\|function(){
	(i[r].q=i[r].q\|\|[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
	m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
	})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

	ga('create', 'UA-85540278-1', 'auto');
	ga('send', 'pageview');
	</script>

	</head>

	<body>
	<nav class="navbar navbar-default navbar-static-top" id="main-nav">
	<div class="container">

	<div class="navbar-header">
	<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false">
	<span class="sr-only">Toggle navigation</span>
	<span class="icon-bar"></span>
	<span class="icon-bar"></span>
	<span class="icon-bar"></span>
	</button>
	<a class="navbar-brand" href="/">
	<img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo">
	Apache Apex<span class="trademark">™</span>
	</a>
	</div>

	<div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
	<ul class="nav navbar-right navbar-nav">
	<li class="nav-item">
	<a class="nav-link " href="/">Home</a>
	</li>
	<li class="nav-item">
	<a class="nav-link " href="/docs.html">Documentation</a>
	</li>
	<li class="nav-item">
	<a class="nav-link " href="/powered-by-apex.html">Powered By Apex</a>
	</li>
	<li class="nav-item">
	<a class="nav-link " href="/roadmap.html">Roadmap</a>
	</li>

	<li class="nav-item nav-mouseover">
	<ul class="dropdown-menu">
	<li class=""><a href="/community.html#mailing-lists">Mailing Lists</a></li>
	<li class=""><a href="/community.html#issue-tracking">Issue Tracking</a></li>
	<li class=""><a href="http://stackoverflow.com/questions/tagged/apache-apex">Stack Overflow</a></li>
	<li class=""><a href="/community.html#events">Events</a></li>
	<li class=""><a href="/community.html#contributing">Contributing</a></li>
	<li class=""><a href="http://www.apache.org/foundation/how-it-works.html">Apache Foundation</a></li>
	</ul>
	<a href="/community.html" class="nav-link">Community</a>
	</li>


	<li class="nav-item nav-mouseover">
	<ul class="dropdown-menu">
	<li class=""><a href="https://github.com/apache/apex-core">Apex Core</a></li>
	<li class=""><a href="https://github.com/apache/apex-malhar">Apex Malhar</a></li>
	<li class=""><a href="https://github.com/apache/apex-site">Apex Site</a></li>
	</ul>
	<a href="https://github.com/apache/apex-core" class="nav-link">Github</a>
	</li>

	<li class="nav-item">
	<a class="nav-link btn btn-success btn-download" href="/downloads.html">Download</a>
	</li>
	</ul>

	</div>
	</nav>

	<div class="container">

	<h1 id="how-to-verify-apache-apex-release-candidate-builds">How to verify Apache Apex release candidate builds</h1>
	<h2 id="prerequisites">Prerequisites</h2>
	<p>Download the KEYS file and import it. Verify that the key just imported is correct.</p>
	<pre><code class="lang-bash">wget https://dist.apache.org/repos/dist/release/apex/KEYS
	gpg --import KEYS
	gpg --fingerprint <key-id>
	</code></pre>
	<p>The <key-id> of the release manager can be found out in the <em>PGP key</em> link in the mail. It would be alphanumric number like D920A98Cbb. The above step can be skipped if you have already imported key of the current release manager.</p>
	<p>More about key verification at <a href="http://www.apache.org/info/verification.html">http://www.apache.org/info/verification.html</a></p>
	<p>You can also create your own key which is required if you would like to sign the build. This step is optional.</p>
	<pre><code class="lang-bash">gpg --gen-key
	</code></pre>
	<p>Please provide 4096 as keysize while generating the key.</p>
	<h2 id="file-integrity-check">File integrity check</h2>
	<p>Download all files present in staging directory of the RC build. Staging directory link is shared in VOTE thread of the release candidate. It is recommended that these files are downloaded in a clean directory.</p>
	<pre><code class="lang-bash">wget -r -np -nd <staging-area-link>/
	</code></pre>
	<p>Note the link should end with "/".</p>
	<p>Define the apex release candidate variable. We will use <em>apache-apex-core-3.6.0</em> as an example.</p>
	<pre><code class="lang-bash">APEX_RELEASE_CANDIDATE=apache-apex-core-3.6.0
	</code></pre>
	<p>Verify integrity of tar.gz file:</p>
	<pre><code class="lang-bash">gpg --verify $APEX_RELEASE_CANDIDATE-source-release.tar.gz.asc
	md5sum --check $APEX_RELEASE_CANDIDATE-source-release.tar.gz.md5
	shasum -a 512 --check $APEX_RELEASE_CANDIDATE-source-release.tar.gz.sha512
	</code></pre>
	<p>Verify integrity of .zip file:</p>
	<pre><code class="lang-bash">gpg --verify $APEX_RELEASE_CANDIDATE-source-release.zip.asc
	md5sum --check $APEX_RELEASE_CANDIDATE-source-release.zip.md5
	shasum -a 512 --check $APEX_RELEASE_CANDIDATE-source-release.zip.sha512
	</code></pre>
	<h2 id="source-code-verification">Source code verification</h2>
	<p>You can extract source either using .tar.gz file or .zip file.</p>
	<h3 id="using-tar-gz-source">Using .tar.gz source</h3>
	<p>Extract source using .tar.gz:</p>
	<pre><code class="lang-bash">tar -zxvf $APEX_RELEASE_CANDIDATE-source-release.tar.gz
	</code></pre>
	<h3 id="using-zip-source">Using .zip source</h3>
	<pre><code class="lang-bash">unzip $APEX_RELEASE_CANDIDATE-source-release.zip
	</code></pre>
	<p>Any of the two commands above will create a directory named after <em>apex release candidate</em>.</p>
	<p>Change directory:</p>
	<pre><code class="lang-bash">cd $APEX_RELEASE_CANDIDATE
	</code></pre>
	<h3 id="miscellaneous-checks-part-i-pre-compilation-">Miscellaneous checks: Part I (Pre Compilation)</h3>
	<p>Existence of LICENSE, NOTICE, README.md and CHANGELOG.md files.
	Please make sure these files are present in the parent folder after extracting the zip or tar. The CHANGELOG.md file should contain change log for the current release.</p>
	<p>No unexpected binary files in the sources.
	The extracted directory should not contain any binary. Need to test this before compiling the source code.</p>
	<pre><code class="lang-bash">find . -type f -name '.' \| sed 's\|.*\.\|\|' \| sort -u
	</code></pre>
	<p>Please make sure no binary extension is listed here. Likely candidates are .jar files. Additionally, for Malhar, demo package files, i.e. .apa files.</p>
	<h3 id="check-for-compilation-license-headers-etc-">Check for compilation, license headers, etc.</h3>
	<p>For Apex Core:</p>
	<pre><code class="lang-bash">mvn clean apache-rat:check verify -Dlicense.skip=false install
	</code></pre>
	<p>For Apex Malhar:</p>
	<pre><code class="lang-bash">mvn clean apache-rat:check verify -Dlicense.skip=false -Pall-modules install
	</code></pre>
	<p>Jars should be installed in your maven repository, typically in <em>~/.m2/repository/org/apache/apex/</em> directory</p>
	<p>Following step is optional and needs the key generated by you as in prerequisite section given above.</p>
	<pre><code class="lang-bash">mvn verify -Papache-release -DskipTests
	</code></pre>
	<h2 id="launch-demos">Launch demos</h2>
	<p>For verification of Apex build, check backward compatibility with respect to Malhar. Locally update <em>apex.core.version</em> in Malhar to point to <em>to-be released</em> version of apex-core and recompile Malhar.</p>
	<p>Launch a few demos to make sure everything is working fine using <em>apex</em> utility. The apex script is present at <apex-core-folder>/engine/src/main/scripts/apex . If apex-core is being verified, then simply use engine/src/main/scripts/apex to launch the script. Otherwise, prefix it with apex-core folder path.</p>
	<pre><code class="lang-bash">engine/src/main/scripts/apex
	</code></pre>
	<p>You will get apex prompt, where demos can be launched.</p>
	<pre><code class="lang-bash">apex> launch <demo-apa-file>
	</code></pre>
	<p>Demo apa files are typically in apex-malhar/demos directory.</p>
	<h2 id="miscellaneous-checks-part-ii-post-compilation-">Miscellaneous checks: Part II (Post compilation)</h2>
	<p>Correct artifact names:
	All the apex artifacts generated in local maven build directory should contain classs jar, sources jars, javadocs jar. All the artifacts are under <em>~/.m2/repository/org/apache/apex/</em> directory on your local machine.</p>

	</div>

	<hr>
	<div class="container">
	<footer id="main-footer">
	<p>
	Copyright © <span id="copyright-year">2015</span> <a href="http://apache.org">The Apache Software Foundation</a>,
	Licensed under the Apache License, Version 2.0<br>
	Apache and the Apache feather logo are trademarks of The Apache Software Foundation. \| <a href="/privacy.html">Privacy Policy</a><br>
	<a class="footer-link-img" href="http://apache.org"><img src="/images/asf_logo.svg" alt="The Apache Software Foundation"></a>
	</p>
	</footer>
	</div> <!-- /container -->

	<!-- Placed at the end of the document so the pages load faster -->
	<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
	<script src="/js/bootstrap.min.js"></script>
	<script src="js/docs.js"></script>

	<script>
	$('#copyright-year').text((new Date()).getFullYear());

	$('.btn-download').click(function(e){
	ga('send', {
	hitType: 'event',
	eventCategory: 'Download Button',
	eventAction: 'Button Click',
	eventLabel: 'Page Header'
	});
	});

	</script>

	</body>
	</html>