| <!DOCTYPE html> |
| <!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> |
| <!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> |
| <!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> |
| <!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1"/> |
| <title>Apache Gearpump (Incubating): Gearpump Performance Report</title> |
| |
| <meta name="description" content="Gearpump Performance Report"> |
| |
| |
| |
| |
| |
| |
| <link rel="stylesheet" href="./css/bootstrap-3.3.5.min.css"> |
| <style> |
| body { |
| padding-top: 60px; |
| padding-bottom: 40px; |
| } |
| </style> |
| <link rel="stylesheet" href="./css/main.css"> |
| <link rel="stylesheet" href="css/pygments-default.css"> |
| <script src="./js/vendor/modernizr-2.6.1-respond-1.1.0.min.js"></script> |
| </head> |
| <body> |
| <!--[if lt IE 7]> |
| <p class="chromeframe">You are using an outdated browser. <a href="http://browsehappy.com/">Upgrade your browser today</a> or <a href="http://www.google.com/chromeframe/?redirect=true">install Google Chrome Frame</a> to better experience this site.</p>l |
| <![endif]--> |
| |
| <div class="navbar navbar-inverse navbar-fixed-top" id="topbar"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="/">Apache Gearpump (incubating)</a> |
| </div> |
| <div id="navbar" class="collapse navbar-collapse"> |
| <ul class="nav navbar-nav"> |
| <li><a href="overview.html">Overview</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Docs<b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="releases/latest/index.html">Latest Release (0.8.1)</a></li> |
| <li><a href="usecases.html">Use Cases</a></li> |
| <li class="divider"></li> |
| <li><a href="publications.html">Publications</a></li> |
| </ul> |
| </li> |
| <li><a href="downloads.html">Downloads</a></li> |
| <li><a href="faq.html">FAQ</a></li> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community<b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li class="dropdown-header">Community</li> |
| <li><a href="community.html#mailing-lists">Mailing Lists</a></li> |
| <li><a href="community.html#issue-tracker">Issue Tracker</a></li> |
| <li><a href="community.html#source-code-repositories">Source Code Repositories</a></li> |
| <li><a href="community.html#who-we-are">Who We Are</a></li> |
| <li><a href="https://twitter.com/ApacheGearpump" target="_blank">Follow us on Twitter</a></li> |
| <li class="divider"></li> |
| <li class="dropdown-header">Contribute</li> |
| <li><a href="how-to-contribute.html">How to Contribute</a></li> |
| <li><a href="coding-style.html">Coding Style</a></li> |
| </ul> |
| </li> |
| </ul> |
| |
| <ul class="nav navbar-nav"> |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF<b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="license.html">License</a></li> |
| <li><a href="//apache.org/foundation/how-it-works.html" target="_blank">How Apache Works</a></li> |
| <li><a href="//apache.org/foundation" target="_blank">Foundation</a></li> |
| <li><a href="//www.apache.org/foundation/sponsorship.html" target="_blank">Sponsoring Apache</a></li> |
| </ul> |
| </li> |
| </ul> |
| |
| <a class="ribbon hidden-xs" href="//github.com/apache/incubator-gearpump"><img src="img/forkme_right_red_aa0000.png" alt="Fork me on GitHub"/></a> |
| |
| </div> |
| </div> |
| </div> |
| |
| <div class="container" id="content" style="margin-bottom: 50px"> |
| |
| <h1 class="title">Gearpump Performance Report</h1> |
| |
| |
| <h1 id="performance-evaluation">Performance Evaluation</h1> |
| |
| <p>To illustrate the performance of Gearpump, we mainly focused on two aspects, throughput and latency, using a micro benchmark called SOL (an example in the Gearpump package) whose topology is quite simple. SOLStreamProducer delivers messages to SOLStreamProcessor constantly and SOLStreamProcessor does nothing. We set up a 4-nodes cluster with 10GbE network and each node’s hardware is briefly shown as follows:</p> |
| |
| <p>Processor: 32 core Intel(R) Xeon(R) CPU E5-2690 2.90GHz |
| Memory: 64GB</p> |
| |
| <h2 id="throughput">Throughput</h2> |
| |
| <p>We tried to explore the upper bound of the throughput, after launching 48 SOLStreamProducer and 48 SOLStreamProcessor the Figure below shows that the whole throughput of the cluster can reach about 18 million messages/second(100 bytes per message)</p> |
| |
| <h2 id="latency">Latency</h2> |
| |
| <p>When we transfer message at the max throughput above, the average latency between two tasks is 8ms.</p> |
| |
| <h2 id="fault-recovery-time">Fault Recovery time</h2> |
| |
| <p>When the corruption is detected, for example the Executor is down, Gearpump will reallocate the resource and restart the application. It takes about 10 seconds to recover the application.</p> |
| |
| <p><img src="img/dashboard.png" alt="Dashboard" /></p> |
| |
| <h2 id="how-to-setup-the-benchmark-environment">How to setup the benchmark environment?</h2> |
| |
| <h3 id="prepare-the-env">Prepare the env</h3> |
| |
| <p>1). Set up a 4-nodes Gearpump cluster with 10GbE network which have 4 Workers on each node. In our test environment, each node has 64GB memory and Intel(R) Xeon(R) 32-core processor E5-2690 2.90GHz. Make sure the metrics is enabled in Gearpump.</p> |
| |
| <p>2). Submit a SOL application with 48 StreamProducers and 48 StreamProcessors:</p> |
| |
| <p><code>bash |
| bin/gear app -jar ./examples/sol-$VERSION-assembly.jar -streamProducer 48 -streamProcessor 48 |
| </code></p> |
| |
| <p>3). Launch Gearpump’s dashboard and browser http://$HOST:8090/, switch to the Applications tab and you can see the detail information of your application. The HOST should be the node runs dashboard.</p> |
| |
| </div> <!-- /container --> |
| |
| <footer class="navbar navbar-default navbar-fixed-bottom"> |
| <div class="container text-center" style="padding: 14px 0 10px 0; line-height: 150%"> |
| <p class="text-muted"> |
| Copyright © 2016 <a href="//apache.org" target="_blank">The Apache Software Foundation</a>. |
| All Rights Reserved.<br/> |
| Apache and the Apache feather logo are trademarks of The Apache Software Foundation. |
| </p> |
| </div> |
| </footer> |
| |
| <script src="js/vendor/jquery-2.1.4.min.js"></script> |
| <script src="js/vendor/bootstrap-3.3.5.min.js"></script> |
| <script src="js/vendor/anchor-1.1.1.min.js"></script> |
| <script src="js/main.js"></script> |
| |
| <!-- MathJax Section --> |
| <script type="text/x-mathjax-config"> |
| MathJax.Hub.Config({ |
| TeX: { equationNumbers: { autoNumber: "AMS" } } |
| }); |
| </script> |
| <script> |
| // Note that we load MathJax this way to work with local file (file://), HTTP and HTTPS. |
| // We could use "//cdn.mathjax...", but that won't support "file://". |
| (function(d, script) { |
| script = d.createElement('script'); |
| script.type = 'text/javascript'; |
| script.async = true; |
| script.onload = function(){ |
| MathJax.Hub.Config({ |
| tex2jax: { |
| inlineMath: [ ["$", "$"], ["\\\\(","\\\\)"] ], |
| displayMath: [ ["$$","$$"], ["\\[", "\\]"] ], |
| processEscapes: true, |
| skipTags: ['script', 'noscript', 'style', 'textarea', 'pre'] |
| } |
| }); |
| }; |
| script.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + |
| 'cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'; |
| d.getElementsByTagName('head')[0].appendChild(script); |
| }(document)); |
| </script> |
| </body> |
| </html> |
| |
