Google Git
Sign in
apache / mesos-site / e7fb58271cdc54f9a9222da662d2fa6169f7c5c3 / . / publish / documentation / latest / attributes-resources / index.html
blob: 878d41a84ef3a26870de33f78d4d79f08ff95b6a [file] [log] [blame]
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Apache Mesos - Attributes and Resources</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta property="og:locale" content="en_US"/>
<meta property="og:type" content="website"/>
<meta property="og:title" content="Apache Mesos"/>
<meta property="og:site_name" content="Apache Mesos"/>
<meta property="og:url" content="http://mesos.apache.org/"/>
<meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
<meta property="og:description"
content="Apache Mesos abstracts resources away from machines,
enabling fault-tolerant and elastic distributed systems
to easily be built and run effectively."/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:site" content="@ApacheMesos"/>
<meta name="twitter:title" content="Apache Mesos"/>
<meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
<meta name="twitter:description"
content="Apache Mesos abstracts resources away from machines,
enabling fault-tolerant and elastic distributed systems
to easily be built and run effectively."/>
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
<link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
<!-- Google Analytics Magic -->
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-20226872-1']);
_gaq.push(['_setDomainName', 'apache.org']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<!-- magical breadcrumbs -->
<div class="topnav">
<div class="container">
<ul class="breadcrumb">
<li>
<div class="dropdown">
<a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
<ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
<li><a href="http://www.apache.org">Apache Homepage</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
</ul>
</div>
</li>
<li><a href="http://mesos.apache.org">Apache Mesos</a></li>
<li><a href="/documentation
/">Documentation
</a></li>
</ul><!-- /.breadcrumb -->
</div><!-- /.container -->
</div><!-- /.topnav -->
<!-- navbar excitement -->
<div class="navbar navbar-default navbar-static-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a>
</div><!-- /.navbar-header -->
<div class="navbar-collapse collapse" id="mesos-menu">
<ul class="nav navbar-nav navbar-right">
<li><a href="/gettingstarted/">Getting Started</a></li>
<li><a href="/blog/">Blog</a></li>
<li><a href="/documentation/latest/">Documentation</a></li>
<li><a href="/downloads/">Downloads</a></li>
<li><a href="/community/">Community</a></li>
</ul>
</div><!-- /#mesos-menu -->
</div><!-- /.container -->
</div><!-- /.navbar -->
<div class="content">
<div class="container">
<div class="row-fluid">
<div class="col-md-4">
<h4>If you're new to Mesos</h4>
<p>See the <a href="/gettingstarted/">getting started</a> page for more
information about downloading, building, and deploying Mesos.</p>
<h4>If you'd like to get involved or you're looking for support</h4>
<p>See our <a href="/community/">community</a> page for more details.</p>
</div>
<div class="col-md-8">
<h1>Mesos Attributes &amp; Resources</h1>
<p>Mesos has two basic methods to describe the agents that comprise a cluster. One of these is managed by the Mesos master, the other is simply passed onwards to the frameworks using the cluster.</p>
<h2>Types</h2>
<p>The types of values that are supported by Attributes and Resources in Mesos are scalar, ranges, sets and text.</p>
<p>The following are the definitions of these types:</p>
<pre><code>scalar : floatValue
floatValue : ( intValue ( "." intValue )? ) | ...
intValue : [0-9]+
range : "[" rangeValue ( "," rangeValue )* "]"
rangeValue : scalar "-" scalar
set : "{" text ( "," text )* "}"
text : [a-zA-Z0-9_/.-]
</code></pre>
<h2>Attributes</h2>
<p>Attributes are key-value pairs (where value is optional) that Mesos passes along when it sends offers to frameworks. An attribute value supports three different <em>types</em>: scalar, range or text.</p>
<pre><code>attributes : attribute ( ";" attribute )*
attribute : text ":" ( scalar | range | text )
</code></pre>
<h2>Resources</h2>
<p>Mesos can manage three different <em>types</em> of resources: scalars, ranges, and sets. These are used to represent the different resources that a Mesos agent has to offer. For example, a scalar resource type could be used to represent the amount of memory on an agent. Scalar resources are represented using floating point numbers to allow fractional values to be specified (e.g., &ldquo;1.5 CPUs&rdquo;). Mesos only supports three decimal digits of precision for scalar resources (e.g., reserving &ldquo;1.5123 CPUs&rdquo; is considered equivalent to reserving &ldquo;1.512 CPUs&rdquo;). For GPUs, Mesos only supports whole number values.</p>
<p>Resources can be specified either with a JSON array or a semicolon-delimited string of key-value pairs. If, after examining the examples below, you have questions about the format of the JSON, inspect the <code>Resource</code> protobuf message definition in <code>include/mesos/mesos.proto</code>.</p>
<p>As JSON:</p>
<pre><code>[
{
"name": "&lt;resource_name&gt;",
"type": "SCALAR",
"scalar": {
"value": &lt;resource_value&gt;
}
},
{
"name": "&lt;resource_name&gt;",
"type": "RANGES",
"ranges": {
"range": [
{
"begin": &lt;range_beginning&gt;,
"end": &lt;range_ending&gt;
},
...
]
}
},
{
"name": "&lt;resource_name&gt;",
"type": "SET",
"set": {
"item": [
"&lt;first_item&gt;",
...
]
},
"role": "&lt;role_name&gt;"
},
...
]
</code></pre>
<p>As a list of key-value pairs:</p>
<pre><code>resources : resource ( ";" resource )*
resource : key ":" ( scalar | range | set )
key : text ( "(" resourceRole ")" )?
resourceRole : text | "*"
</code></pre>
<p>Note that <code>resourceRole</code> must be a valid role name; see the <a href="/documentation/latest/./roles/">roles</a> documentation for details.</p>
<h2>Predefined Uses &amp; Conventions</h2>
<p>There are several kinds of resources that have predefined behavior:</p>
<ul>
<li><code>cpus</code></li>
<li><code>gpus</code></li>
<li><code>disk</code></li>
<li><code>mem</code></li>
<li><code>ports</code></li>
</ul>
<p>Note that <code>disk</code> and <code>mem</code> resources are specified in megabytes. The master&rsquo;s user interface will convert resource values into a more human-readable format: for example, the value <code>15000</code> will be displayed as <code>14.65GB</code>.</p>
<p>An agent without <code>cpus</code> and <code>mem</code> resources will not have its resources advertised to any frameworks.</p>
<h2>Examples</h2>
<p>By default, Mesos will try to autodetect the resources available at the local machine when <code>mesos-agent</code> starts up. Alternatively, you can explicitly configure which resources an agent should make available.</p>
<p>Here are some examples of how to configure the resources at a Mesos agent:</p>
<pre><code>--resources='cpus:24;gpus:2;mem:24576;disk:409600;ports:[21000-24000,30000-34000];bugs(debug_role):{a,b,c}'
--resources='[{"name":"cpus","type":"SCALAR","scalar":{"value":24}},{"name":"gpus","type":"SCALAR","scalar":{"value":2}},{"name":"mem","type":"SCALAR","scalar":{"value":24576}},{"name":"disk","type":"SCALAR","scalar":{"value":409600}},{"name":"ports","type":"RANGES","ranges":{"range":[{"begin":21000,"end":24000},{"begin":30000,"end":34000}]}},{"name":"bugs","type":"SET","set":{"item":["a","b","c"]},"role":"debug_role"}]'
</code></pre>
<p>Or given a file <code>resources.txt</code> containing the following:</p>
<pre><code>[
{
"name": "cpus",
"type": "SCALAR",
"scalar": {
"value": 24
}
},
{
"name": "gpus",
"type": "SCALAR",
"scalar": {
"value": 2
}
},
{
"name": "mem",
"type": "SCALAR",
"scalar": {
"value": 24576
}
},
{
"name": "disk",
"type": "SCALAR",
"scalar": {
"value": 409600
}
},
{
"name": "ports",
"type": "RANGES",
"ranges": {
"range": [
{
"begin": 21000,
"end": 24000
},
{
"begin": 30000,
"end": 34000
}
]
}
},
{
"name": "bugs",
"type": "SET",
"set": {
"item": [
"a",
"b",
"c"
]
},
"role": "debug_role"
}
]
</code></pre>
<p>You can do:</p>
<pre><code>$ path/to/mesos-agent --resources=file:///path/to/resources.txt ...
</code></pre>
<p>In this case, we have five resources of three different types: scalars, a range, and a set. There are scalars called <code>cpus</code>, <code>gpus</code>, <code>mem</code> and <code>disk</code>, a range called <code>ports</code>, and a set called <code>bugs</code>. <code>bugs</code> is assigned to the role <code>debug_role</code>, while the other resources do not specify a role and are thus assigned to the default role.</p>
<p>Note: the &ldquo;default role&rdquo; can be set by the <code>--default_role</code> flag.</p>
<ul>
<li>scalar called <code>cpus</code>, with the value <code>24</code></li>
<li>scalar called <code>gpus</code>, with the value <code>2</code></li>
<li>scalar called <code>mem</code>, with the value <code>24576</code></li>
<li>scalar called <code>disk</code>, with the value <code>409600</code></li>
<li>range called <code>ports</code>, with values <code>21000</code> through <code>24000</code> and <code>30000</code> through <code>34000</code> (inclusive)</li>
<li>set called <code>bugs</code>, with the values <code>a</code>, <code>b</code> and <code>c</code>, assigned to the role <code>debug_role</code></li>
</ul>
<p>To configure the attributes of a Mesos agent, you can use the <code>--attributes</code> command-line flag of <code>mesos-agent</code>:</p>
<pre><code>--attributes='rack:abc;zone:west;os:centos5;level:10;keys:[1000-1500]'
</code></pre>
<p>That will result in configuring the following five attributes:</p>
<ul>
<li><code>rack</code> with text value <code>abc</code></li>
<li><code>zone</code> with text value <code>west</code></li>
<li><code>os</code> with text value <code>centos5</code></li>
<li><code>level</code> with scalar value 10</li>
<li><code>keys</code> with range value <code>1000</code> through <code>1500</code> (inclusive)</li>
</ul>
</div>
</div>
</div><!-- /.container -->
</div><!-- /.content -->
<hr>
<!-- footer -->
<div class="footer">
<div class="container">
<div class="col-md-4 social-blk">
<span class="social">
<a href="https://twitter.com/ApacheMesos"
class="twitter-follow-button"
data-show-count="false" data-size="large">Follow @ApacheMesos</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
<a href="https://twitter.com/intent/tweet?button_hashtag=mesos"
class="twitter-hashtag-button"
data-size="large"
data-related="ApacheMesos">Tweet #mesos</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
</span>
</div>
<div class="col-md-8 trademark">
<p>&copy; 2012-2017 <a href="http://apache.org">The Apache Software Foundation</a>.
Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.
<p>
</div>
</div><!-- /.container -->
</div><!-- /.footer -->
<!-- JS -->
<script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
<script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
</body>
</html>