| <!DOCTYPE html> |
| <!--[if lt IE 7]> |
| <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> |
| <!--[if IE 7]> |
| <html class="no-js lt-ie9 lt-ie8"> <![endif]--> |
| <!--[if IE 8]> |
| <html class="no-js lt-ie9"> <![endif]--> |
| <!--[if gt IE 8]><!--> |
| <html class="no-js"> <!--<![endif]--> |
| <head> |
| <meta charset="utf-8"> |
| <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> |
| <meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1"/> |
| <title>faq - Gearpump 0.8.1 Documentation</title> |
| |
| <meta name="description" |
| content="Gearpump Frequently Asked Questions"> |
| |
| |
| |
| |
| <link rel="stylesheet" href="css/bootstrap-3.3.5.min.css"> |
| <style> |
| body { |
| padding-top: 60px; |
| padding-bottom: 40px; |
| } |
| </style> |
| <link rel="stylesheet" href="css/main.css"> |
| <link rel="stylesheet" href="css/pygments-default.css"> |
| <script src="js/vendor/modernizr-2.6.1-respond-1.1.0.min.js"></script> |
| </head> |
| <body> |
| <!--[if lt IE 7]> |
| <p class="chromeframe">You are using an outdated browser. <a href="http://browsehappy.com/">Upgrade |
| your browser today</a> or <a href="http://www.google.com/chromeframe/?redirect=true">install |
| Google Chrome Frame</a> to better experience this site.</p> |
| <![endif]--> |
| |
| <div class="navbar navbar-inverse navbar-fixed-top" id="topbar"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" |
| data-target="#navbar" aria-expanded="false" aria-controls="navbar"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="http://gearpump.apache.org">Gearpump |
| <span class="label label-primary" style="font-size: .6em">0.8.1</span> |
| </a> |
| </div> |
| <div id="navbar" class="collapse navbar-collapse"> |
| <ul class="nav navbar-nav"> |
| <li><a href="index.html">Overview</a></li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Introduction<b |
| class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="submit-your-1st-application.html">Submit Your 1st Application</a></li> |
| <li><a href="commandline.html">Client Command Line</a></li> |
| <li class="divider"></li> |
| <li><a href="basic-concepts.html">Basic Concepts</a></li> |
| <li><a href="features.html">Technical Highlights</a></li> |
| <li><a href="message-delivery.html">Reliable Message Delivery</a></li> |
| <li><a href="performance-report.html">Performance</a></li> |
| <li><a href="gearpump-internals.html">Gearpump Internals</a></li> |
| </ul> |
| </li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Deploying<b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li class="dropdown-header">Deployment</li> |
| <li><a href="deployment-local.html">Local Mode</a> |
| <li> |
| <li><a href="deployment-standalone.html">Standalone Mode</a></li> |
| <li><a href="deployment-yarn.html">YARN Mode</a></li> |
| <li><a href="deployment-docker.html">Docker Mode</a> |
| <li> |
| <li class="divider"></li> |
| <li><a href="deployment-ui-authentication.html">UI Authentication</a></li> |
| <li><a href="deployment-ha.html">High Availability</a></li> |
| <li><a href="deployment-msg-delivery.html">Reliable Message Delivery</a></li> |
| <li><a href="deployment-configuration.html">Configuration</a></li> |
| <li><a href="deployment-resource-isolation.html">Resource Isolation</a></li> |
| <li class="divider"></li> |
| <li><a href="deployment-security.html">YARN Security Guide</a></li> |
| </ul> |
| </li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">Programming Guide<b |
| class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="dev-write-1st-app.html">Write Your 1st App</a></li> |
| <li><a href="dev-custom-serializer.html">Customized Message Passing</a></li> |
| <li class="divider"></li> |
| <li><a href="api/scala/index.html">Scala API</a></li> |
| <li><a href="api/java/index.html">Java API</a></li> |
| <li><a href="dev-rest-api.html">RESTful API</a></li> |
| <li class="divider"></li> |
| <li><a href="dev-connectors.html">Gearpump Connectors</a></li> |
| <li class="divider"></li> |
| <li><a href="dev-storm.html">Storm Compatibility</a></li> |
| <!-- |
| <li><a href="dev-samoa.html">Samoa Compatibility</a></li> |
| <li class="divider"></li> |
| <li><a href="dev-iot.html">Gearpump with IoT</a></li> |
| --> |
| </ul> |
| </li> |
| |
| <li class="dropdown"> |
| <a href="#" class="dropdown-toggle" data-toggle="dropdown">More<b class="caret"></b></a> |
| <ul class="dropdown-menu"> |
| <li><a href="faq.html">FAQ</a></li> |
| </ul> |
| </li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| |
| <div class="container" id="content"> |
| |
| <h1 class="title">Frequently Asked Questions</h1> |
| |
| |
| <h5 id="whats-relation-between-gearpump-and-yarn">What’s Relation between Gearpump and YARN?</h5> |
| <p>Gearpump can run on top of YARN as a YARN application. Gearpump’s ApplicationMaster provides the application management, deployment and scheduling of DAGs after arbitrating and receiving container resources from YARN</p> |
| |
| <h5 id="relation-with-storm-and-spark-streaming">Relation with Storm and Spark Streaming</h5> |
| <p>Storm and spark streaming are proven platforms, and there are many production deployments. Compared with them, Gearpump is less proven and there is no production deployment yet. However, there is no single platform that can cover every use case; Gearpump has its own +1 points in some special fields. As an instance, for IoT use cases, Gearpump may be considered convenient because the topology can be deployed to edge device with feature of location transparency. For another example, when users want to upgrade the application online without service interruption, Gearpump may be suitable as it can dynamically modify the computation DAG on the fly.</p> |
| |
| <h5 id="what-does-gearpump-mean">What does Gearpump mean?</h5> |
| <p>The name Gearpump is a reference to the engineering term “Gear Pump”, which is a super simple pump that consists of only two gears, but is very powerful at streaming water from left to right.</p> |
| |
| <h5 id="why-not-using-akka-persistence-to-store-the-checkpoint-file">Why not using akka persistence to store the checkpoint file?</h5> |
| <ol> |
| <li>We only checkpoint file to disk when necessary.(not record level)</li> |
| <li>We have custom checkpoint file format</li> |
| </ol> |
| |
| <h5 id="have-you-considered-the-akka-stream-api-for-the-high-level-dsl">Have you considered the akka stream API for the high level DSL?</h5> |
| <p>We are looking into a hands of candidates for what a good DSL should be. Akka stream API is one of the candidates.</p> |
| |
| <h5 id="why-wrapping-the-task-instead-of-using-the-actor-interface-directly">Why wrapping the Task, instead of using the Actor interface directly?</h5> |
| |
| <ol> |
| <li>It is easier to conduct Unit test</li> |
| <li>We have custom logic and messages to ensure the data consistency, like flow control and message loss detection.</li> |
| <li>As the Gearpump interface evolves rapidly, for now, we want to be conservative in exposing more powerful functions so that we don’t tie our hands for future refactoring. It lets us feel safe.</li> |
| </ol> |
| |
| <h5 id="why-does-my-task-has-extremely-high-message-latency-eg-10-seconds-">Why does my task has extremely high message latency (e.g. 10 seconds) ?</h5> |
| |
| <p>Please check whether you are doing blocking jobs (e.g. sleep, IO) in your task. By default, all tasks in an executor share a thread pool. The blocking tasks could use up all the threads while other tasks don’t get a chance to run. In that case, you can set <code>gearpump.task-dispatcher</code> to <code>"gearpump.single-thread-dispatcher"</code> in <code>gear.conf</code> such that a unique thread is dedicated to each task.</p> |
| |
| <p>Generally, we recommend use the default <code>share-thread-pool-dispatcher</code> which has better performance and only turn to the <code>single-thread-dispatcher</code> when you have to.</p> |
| |
| <h5 id="why-cant-i-open-dashboard-even-if-the-services-process-has-been-launched-successfully-">Why can’t I open Dashboard even if the Services process has been launched successfully ?</h5> |
| |
| <p>By default, our Services process binds to a local <strong>IPv6 port</strong>. It’s possible that another process on your system has already taken up the same <strong>IPv4 port</strong>. You may check by <code>lsof -i -P | grep -i "Listen"</code> if your system is Unix/Linux.</p> |
| |
| |
| |
| </div> |
| <!-- /container --> |
| |
| <script src="js/vendor/jquery-2.1.4.min.js"></script> |
| <script src="js/vendor/bootstrap-3.3.5.min.js"></script> |
| <script src="js/vendor/anchor-1.1.1.min.js"></script> |
| <script src="js/main.js"></script> |
| |
| <!-- MathJax Section --> |
| <script type="text/x-mathjax-config"> |
| MathJax.Hub.Config({ |
| TeX: { equationNumbers: { autoNumber: "AMS" } } |
| }); |
| |
| </script> |
| <script> |
| // Note that we load MathJax this way to work with local file (file://), HTTP and HTTPS. |
| // We could use "//cdn.mathjax...", but that won't support "file://". |
| (function (d, script) { |
| script = d.createElement('script'); |
| script.type = 'text/javascript'; |
| script.async = true; |
| script.onload = function () { |
| MathJax.Hub.Config({ |
| tex2jax: { |
| inlineMath: [["$", "$"], ["\\\\(", "\\\\)"]], |
| displayMath: [["$$", "$$"], ["\\[", "\\]"]], |
| processEscapes: true, |
| skipTags: ['script', 'noscript', 'style', 'textarea', 'pre'] |
| } |
| }); |
| }; |
| script.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + |
| 'cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'; |
| d.getElementsByTagName('head')[0].appendChild(script); |
| }(document)); |
| </script> |
| </body> |
| </html> |
