| <!DOCTYPE html> |
| <html> |
| <head> |
| <meta charset="utf-8"> |
| <title></title> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet"> |
| <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml"> |
| |
| <link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" /> |
| |
| |
| |
| <!-- Google Analytics Magic --> |
| <script type="text/javascript"> |
| var _gaq = _gaq || []; |
| _gaq.push(['_setAccount', 'UA-20226872-1']); |
| _gaq.push(['_setDomainName', 'apache.org']); |
| _gaq.push(['_trackPageview']); |
| |
| (function() { |
| var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; |
| ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; |
| var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); |
| })(); |
| </script> |
| </head> |
| <body> |
| <!-- magical breadcrumbs --> |
| <div class="topnav"> |
| <ul class="breadcrumb"> |
| <li> |
| <div class="dropdown"> |
| <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a> |
| <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel"> |
| <li><a href="http://www.apache.org">Apache Homepage</a></li> |
| <li><a href="http://www.apache.org/licenses/">License</a></li> |
| <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li> |
| <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| <li><a href="http://www.apache.org/security/">Security</a></li> |
| </ul> |
| </div> |
| </li> |
| <li><a href="http://mesos.apache.org">Apache Mesos</a></li> |
| |
| |
| <li><a href="/documentation |
| /">Documentation |
| </a></li> |
| |
| |
| </ul><!-- /breadcrumb --> |
| </div> |
| |
| <!-- navbar excitement --> |
| <div class="navbar navbar-static-top" role="navigation"> |
| <div class="navbar-inner"> |
| <div class="container"> |
| <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a> |
| <div class="nav-collapse"> |
| <ul class="nav nav-pills navbar-right"> |
| <li><a href="/gettingstarted/">Getting Started</a></li> |
| <li><a href="/documentation/latest/">Documentation</a></li> |
| <li><a href="/downloads/">Downloads</a></li> |
| <li><a href="/community/">Community</a></li> |
| </ul> |
| </div> |
| </div> |
| </div> |
| </div><!-- /.navbar --> |
| |
| <div class="container"> |
| |
| <div class="row-fluid"> |
| <div class="col-md-4"> |
| <h4>If you're new to Mesos</h4> |
| <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p> |
| |
| <h4>If you'd like to get involved or you're looking for support</h4> |
| <p>See our <a href="/community/">community</a> page for more details.</p> |
| </div> |
| <div class="col-md-8"> |
| <h1>Mesos Architecture</h1> |
| |
| <p><img src="http://mesos.apache.org/assets/img/documentation/architecture3.jpg" alt="Mesos Architecture" /></p> |
| |
| <p>The above figure shows the main components of Mesos. Mesos consists of a <em>master</em> daemon that manages <em>slave</em> daemons running on each cluster node, and <em>mesos applications</em> (also called <em>frameworks</em>) that run <em>tasks</em> on these slaves.</p> |
| |
| <p>The master enables fine-grained sharing of resources (cpu, ram, …) across applications by making them <em>resource offers</em>. Each resource offer contains a list of <slave ID, resource1: amount1, resource2, amount2, ...>. The master decides <em>how many</em> resources to offer to each framework according to a given organizational policy, such as fair sharing, or strict priority. To support a diverse set of policies, the master employs a modular architecture that makes it easy to add new allocation modules via a plugin mechanism.</p> |
| |
| <p>A framework running on top of Mesos consists of two components: a <em>scheduler</em> that registers with the master to be offered resources, and an <em>executor</em> process that is launched on slave nodes to run the framework’s tasks (/documentation/latest/see the <a href="app-framework-development-guide/">App/Framework development guide</a> for more details about application schedulers and executors). While the master determines <strong>how many</strong> resources are offered to each framework, the frameworks' schedulers select <strong>which</strong> of the offered resources to use. When a frameworks accepts offered resources, it passes to Mesos a description of the tasks it wants to run on them. In turn, Mesos launches the tasks on the corresponding slaves.</p> |
| |
| <h2>Example of resource offer</h2> |
| |
| <p>The figure below shows an example of how a framework gets scheduled to run a task.</p> |
| |
| <p><img src="http://mesos.apache.org/assets/img/documentation/architecture-example.jpg" alt="Mesos Architecture" /></p> |
| |
| <p>Let’s walk through the events in the figure.</p> |
| |
| <ol> |
| <li>Slave 1 reports to the master that it has 4 CPUs and 4 GB of memory free. The master then invokes the allocation policy module, which tells it that framework 1 should be offered all available resources.</li> |
| <li>The master sends a resource offer describing what is available on slave 1 to framework 1.</li> |
| <li>The framework’s scheduler replies to the master with information about two tasks to run on the slave, using <2 CPUs, 1 GB RAM> for the first task, and <1 CPUs, 2 GB RAM> for the second task.</li> |
| <li>Finally, the master sends the tasks to the slave, which allocates appropriate resources to the framework’s executor, which in turn launches the two tasks (depicted with dotted-line borders in the figure). Because 1 CPU and 1 GB of RAM are still unallocated, the allocation module may now offer them to framework 2.</li> |
| </ol> |
| |
| |
| <p>In addition, this resource offer process repeats when tasks finish and new resources become free.</p> |
| |
| <p>While the thin interface provided by Mesos allows it to scale and allows the frameworks to evolve independently, one question remains: how can the constraints of a framework be satisfied without Mesos knowing about these constraints? For example, how can a framework achieve data locality without Mesos knowing which nodes store the data required by the framework? Mesos answers these questions by simply giving frameworks the ability to <strong>reject</strong> offers. A framework will reject the offers that do not satisfy its constraints and accept the ones that do. In particular, we have found that a simple policy called delay scheduling, in which frameworks wait for a limited time to acquire nodes storing the input data, yields nearly optimal data locality.</p> |
| |
| <p>You can also read much more about the Mesos architecture in this <a href="http://mesos.berkeley.edu/mesos_tech_report.pdf">technical paper</a>.</p> |
| |
| </div> |
| </div> |
| |
| |
| <hr> |
| |
| <!-- footer --> |
| <div class="footer"> |
| <p>© 2012-2014 <a href="http://apache.org">The Apache Software Foundation</a>. |
| Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p> |
| </div><!-- /footer --> |
| |
| </div> <!-- /container --> |
| |
| <!-- JS --> |
| <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script> |
| <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script> |
| </body> |
| </html> |
