blob: ad2232422b3ffdc07e724dc44eb7b35e3541321a [file] [log] [blame]
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Apache Mesos - Multiple Disks</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta property="og:locale" content="en_US"/>
<meta property="og:type" content="website"/>
<meta property="og:title" content="Apache Mesos"/>
<meta property="og:site_name" content="Apache Mesos"/>
<meta property="og:url" content="http://mesos.apache.org/"/>
<meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
<meta property="og:description"
content="Apache Mesos abstracts resources away from machines,
enabling fault-tolerant and elastic distributed systems
to easily be built and run effectively."/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:site" content="@ApacheMesos"/>
<meta name="twitter:title" content="Apache Mesos"/>
<meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
<meta name="twitter:description"
content="Apache Mesos abstracts resources away from machines,
enabling fault-tolerant and elastic distributed systems
to easily be built and run effectively."/>
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
<link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
<!-- Google Analytics Magic -->
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-20226872-1']);
_gaq.push(['_setDomainName', 'apache.org']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<!-- magical breadcrumbs -->
<div class="topnav">
<div class="container">
<ul class="breadcrumb">
<li>
<div class="dropdown">
<a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
<ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
<li><a href="http://www.apache.org">Apache Homepage</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
</ul>
</div>
</li>
<li><a href="http://mesos.apache.org">Apache Mesos</a></li>
<li><a href="/documentation
/">Documentation
</a></li>
</ul><!-- /.breadcrumb -->
</div><!-- /.container -->
</div><!-- /.topnav -->
<!-- navbar excitement -->
<div class="navbar navbar-default navbar-static-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a>
</div><!-- /.navbar-header -->
<div class="navbar-collapse collapse" id="mesos-menu">
<ul class="nav navbar-nav navbar-right">
<li><a href="/gettingstarted/">Getting Started</a></li>
<li><a href="/blog/">Blog</a></li>
<li><a href="/documentation/latest/">Documentation</a></li>
<li><a href="/downloads/">Downloads</a></li>
<li><a href="/community/">Community</a></li>
</ul>
</div><!-- /#mesos-menu -->
</div><!-- /.container -->
</div><!-- /.navbar -->
<div class="content">
<div class="container">
<div class="row-fluid">
<div class="col-md-4">
<h4>If you're new to Mesos</h4>
<p>See the <a href="/gettingstarted/">getting started</a> page for more
information about downloading, building, and deploying Mesos.</p>
<h4>If you'd like to get involved or you're looking for support</h4>
<p>See our <a href="/community/">community</a> page for more details.</p>
</div>
<div class="col-md-8">
<h1>Multiple Disks</h1>
<p>Mesos provides a mechanism for operators to expose multiple disk resources. When
creating <a href="/documentation/latest/./persistent-volume/">persistent volumes</a> frameworks can decide
whether to use specific disks by examining the <code>source</code> field on the disk
resources offered.</p>
<h2>Types of Disk Resources</h2>
<p><code>Disk</code> resources come in three forms:</p>
<ul>
<li>A <code>Root</code> disk is presented by not having the <code>source</code> set in <code>DiskInfo</code>.</li>
<li>A <code>Path</code> disk is presented by having the <code>PATH</code> enum set for <code>source</code> in
<code>DiskInfo</code>. It also has a <code>root</code> which the operator uses to specify the
directory to be used to store data.</li>
<li>A <code>Mount</code> disk is presented by having the <code>MOUNT</code> enum set for <code>source</code> in
<code>DiskInfo</code>. It also has a <code>root</code> which the operator uses to specify the
mount point used to store data.</li>
</ul>
<p>Operators can use the JSON-formated <code>--resources</code> option on the agent to provide
these different kind of disk resources on agent start-up. Example resource
values in JSON format can be found below. By default (if <code>--resources</code> is not
specified), the Mesos agent will only make the root disk available to the
cluster.</p>
<p><strong>NOTE:</strong> Once you specify any <code>Disk</code> resource manually (i.e., via the
<code>--resources</code> flag), Mesos will stop auto-detecting the <code>Root</code> disk resource.
Hence if you want to use the <code>Root</code> disk you will need to manually specify it
using the format described below.</p>
<h3><code>Root</code> disk</h3>
<p>A <code>Root</code> disk is the basic disk resource in Mesos. It usually maps to the
storage on the main operating system drive that the operator has presented to
the agent. Data is mapped into the <code>work_dir</code> of the agent.</p>
<p>An example resources value for a root disk is shown below. Note that the
operator could optionally specify a <code>role</code> for the disk, which would result in
<a href="/documentation/latest/./reservation/">statically reserving</a> the disk for a single <a href="/documentation/latest/./roles/">role</a>.</p>
<pre><code> {
"resources" : [
{
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 }
}
]
}
</code></pre>
<h3><code>Path</code> disks</h3>
<p>A <code>Path</code> disk is an auxiliary disk resource provided by the operator. This
can be carved up into smaller chunks by creating persistent volumes that use
less than the total available space on the disk. Common uses for this kind of
disk are extra logging space, file archives or caches, or other non
performance-critical applications. Operators can present extra disks on their
agents as <code>Path</code> disks simply by creating a directory and making that the <code>root</code>
of the <code>Path</code> in <code>DiskInfo</code>&rsquo;s <code>source</code>.</p>
<p><code>Path</code> disks are also useful for mocking up a multiple disk environment by
creating some directories on the operating system drive. This should only be
done in a testing or staging environment. Note that creating multiple <code>Path</code>
disks on the same filesystem requires statically partitioning the available disk
space. For example, suppose a 10GB storage device is mounted to <code>/foo</code> and the
Mesos agent is configured with two <code>Path</code> disks at <code>/foo/disk1</code> and
<code>/foo/disk2</code>. To avoid the risk of running out of space on the device, <code>disk1</code>
and <code>disk2</code> should be configured (when the Mesos agent is started) to use at
most 10GB of disk space in total.</p>
<p>An example resources value for a <code>Path</code> disk is shown below. Note that the
operator could optionally specify a <code>role</code> for the disk, which would result in
<a href="/documentation/latest/./reservation/">statically reserving</a> the disk for a single <a href="/documentation/latest/./roles/">role</a>.</p>
<pre><code> {
"resources" : [
{
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
"disk" : {
"source" : {
"type" : "PATH",
"path" : { "root" : "/mnt/data" }
}
}
}
]
}
</code></pre>
<h3><code>Mount</code> disks</h3>
<p>A <code>Mount</code> disk is an auxiliary disk resource provided by the operator. This
<strong>cannot</strong> be carved up into smaller chunks by frameworks. This lack of
flexibility allows operators to provide assurances to frameworks that they will
have exclusive access to the disk device. Common uses for this kind of disk
include database storage, write-ahead logs, or other performance-critical
applications.</p>
<p>On Linux, <code>Mount</code> disks must map to a <code>mount</code> point in the <code>/proc/mounts</code>
table. Operators should mount a physical disk with their preferred file system
and provide the mount point as the <code>root</code> of the <code>Mount</code> in <code>DiskInfo</code>&rsquo;s
<code>source</code>.</p>
<p>Aside from the performance advantages of <code>Mount</code> disks, applications running on
them should be able to rely on disk errors when they attempt to exceed the
capacity of the volume. This holds true as long as the file system in use
correctly propagates these errors. Due to this expectation, the <code>disk/du</code>
isolation is disabled for <code>Mount</code> disks.</p>
<p>An example resources value for a <code>Mount</code> disk is shown below. Note that the
operator could optionally specify a <code>role</code> for the disk, which would result in
<a href="/documentation/latest/./reservation/">statically reserving</a> the disk for a single <a href="/documentation/latest/./roles/">role</a>.</p>
<pre><code> {
"resources" : [
{
"name" : "disk",
"type" : "SCALAR",
"scalar" : { "value" : 2048 },
"disk" : {
"source" : {
"type" : "MOUNT",
"mount" : { "root" : "/mnt/data" }
}
}
}
]
}
</code></pre>
<h4><code>Block</code> disks</h4>
<p>Mesos currently does not allow operators to expose raw block devices. It may do
so in the future, but there are security and flexibility concerns that need to
be addressed in a design document first.</p>
<h3>Implementation</h3>
<p>A <code>Path</code> disk will have sub-directories created within the <code>root</code> which will be
used to differentiate the different volumes that are created on it. When a
persistent volume on a <code>Path</code> disk is destroyed, Mesos will remove all the files
and directories stored in the volume, as well as the sub-directory within <code>root</code>
that was created by Mesos for the volume.</p>
<p>A <code>Mount</code> disk will <strong>not</strong> have sub-directories created, allowing applications
to use the full file system mounted on the device. This construct allows Mesos
tasks to access volumes that contain pre-existing directory structures. This can
be useful to simplify ingesting data such as a pre-existing Postgres database or
HDFS data directory. Note that when a persistent volume on a <code>Mount</code> disk is
destroyed, Mesos will remove all the files and directories stored in the volume,
but will <strong>not</strong> remove the root directory (i.e., the mount point).</p>
<p>Operators should be aware of these distinctions when inspecting or cleaning up
remnant data.</p>
</div>
</div>
</div><!-- /.container -->
</div><!-- /.content -->
<hr>
<!-- footer -->
<div class="footer">
<div class="container">
<div class="col-md-4 social-blk">
<span class="social">
<a href="https://twitter.com/ApacheMesos"
class="twitter-follow-button"
data-show-count="false" data-size="large">Follow @ApacheMesos</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
<a href="https://twitter.com/intent/tweet?button_hashtag=mesos"
class="twitter-hashtag-button"
data-size="large"
data-related="ApacheMesos">Tweet #mesos</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
</span>
</div>
<div class="col-md-8 trademark">
<p>&copy; 2012-2017 <a href="http://apache.org">The Apache Software Foundation</a>.
Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.
<p>
</div>
</div><!-- /.container -->
</div><!-- /.footer -->
<!-- JS -->
<script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
<script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
</body>
</html>