| <!DOCTYPE html> |
| <html> |
| <head> |
| <meta charset="utf-8"> |
| <title>Apache Mesos - Sandbox</title> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| |
| <meta property="og:locale" content="en_US"/> |
| <meta property="og:type" content="website"/> |
| <meta property="og:title" content="Apache Mesos"/> |
| <meta property="og:site_name" content="Apache Mesos"/> |
| <meta property="og:url" content="http://mesos.apache.org/"/> |
| <meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/> |
| <meta property="og:description" |
| content="Apache Mesos abstracts resources away from machines, |
| enabling fault-tolerant and elastic distributed systems |
| to easily be built and run effectively."/> |
| |
| <meta name="twitter:card" content="summary"/> |
| <meta name="twitter:site" content="@ApacheMesos"/> |
| <meta name="twitter:title" content="Apache Mesos"/> |
| <meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/> |
| <meta name="twitter:description" |
| content="Apache Mesos abstracts resources away from machines, |
| enabling fault-tolerant and elastic distributed systems |
| to easily be built and run effectively."/> |
| |
| <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet"> |
| <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml"> |
| <link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" /> |
| |
| |
| |
| <!-- Google Analytics Magic --> |
| <script type="text/javascript"> |
| var _gaq = _gaq || []; |
| _gaq.push(['_setAccount', 'UA-20226872-1']); |
| _gaq.push(['_setDomainName', 'apache.org']); |
| _gaq.push(['_trackPageview']); |
| |
| (function() { |
| var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; |
| ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; |
| var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); |
| })(); |
| </script> |
| |
| </head> |
| <body> |
| <!-- magical breadcrumbs --> |
| <div class="topnav"> |
| <div class="container"> |
| <ul class="breadcrumb"> |
| <li> |
| <div class="dropdown"> |
| <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a> |
| <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel"> |
| <li><a href="http://www.apache.org">Apache Homepage</a></li> |
| <li><a href="http://www.apache.org/licenses/">License</a></li> |
| <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li> |
| <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> |
| <li><a href="http://www.apache.org/security/">Security</a></li> |
| </ul> |
| </div> |
| </li> |
| |
| <li><a href="http://mesos.apache.org">Apache Mesos</a></li> |
| |
| |
| <li><a href="/documentation |
| /">Documentation |
| </a></li> |
| |
| |
| </ul><!-- /.breadcrumb --> |
| </div><!-- /.container --> |
| </div><!-- /.topnav --> |
| |
| <!-- navbar excitement --> |
| <div class="navbar navbar-default navbar-static-top" role="navigation"> |
| <div class="container"> |
| <div class="navbar-header"> |
| <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false"> |
| <span class="sr-only">Toggle navigation</span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| <span class="icon-bar"></span> |
| </button> |
| <a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a> |
| </div><!-- /.navbar-header --> |
| |
| <div class="navbar-collapse collapse" id="mesos-menu"> |
| <ul class="nav navbar-nav navbar-right"> |
| <li><a href="/gettingstarted/">Getting Started</a></li> |
| <li><a href="/blog/">Blog</a></li> |
| <li><a href="/documentation/latest/">Documentation</a></li> |
| <li><a href="/downloads/">Downloads</a></li> |
| <li><a href="/community/">Community</a></li> |
| </ul> |
| </div><!-- /#mesos-menu --> |
| </div><!-- /.container --> |
| </div><!-- /.navbar --> |
| |
| <div class="content"> |
| <div class="container"> |
| <div class="row-fluid"> |
| <div class="col-md-4"> |
| <h4>If you're new to Mesos</h4> |
| <p>See the <a href="/gettingstarted/">getting started</a> page for more |
| information about downloading, building, and deploying Mesos.</p> |
| |
| <h4>If you'd like to get involved or you're looking for support</h4> |
| <p>See our <a href="/community/">community</a> page for more details.</p> |
| </div> |
| <div class="col-md-8"> |
| <h1>Mesos “Sandbox”</h1> |
| |
| <p>Mesos refers to the “sandbox” as a temporary directory that holds files specific |
| to a single executor. Each time an executor is run, the executor is given its |
| own sandbox and the executor’s working directory is set to the sandbox.</p> |
| |
| <h2>Sandbox files</h2> |
| |
| <p>The sandbox holds:</p> |
| |
| <ul> |
| <li>Files <a href="/documentation/latest/./fetcher/">fetched by Mesos</a>, prior to starting |
| the executor’s tasks.</li> |
| <li>The output of the executor and tasks (as files “stdout” and “stderr”).</li> |
| <li>Files created by the executor and tasks, with some exceptions.</li> |
| </ul> |
| |
| |
| <p><strong>NOTE:</strong> With the introduction of <a href="/documentation/latest/./persistent-volume/">persistent volumes</a>, |
| executors and tasks should never create files outside of the sandbox. However, |
| some containerizers do not enforce this sandboxing.</p> |
| |
| <h2><a name="where-is-it"></a>Where is the sandbox?</h2> |
| |
| <p>The sandbox is located within the agent’s working directory (which is specified |
| via the <code>--work_dir</code> flag). To find a particular executor’s sandbox, you must |
| know the agent’s ID, the executor’s framework’s ID, and the executor’s ID. |
| Each run of the executor will have a corresponding sandbox, denoted by a |
| container ID.</p> |
| |
| <p>The sandbox is located on the agent, inside a directory tree like the following:</p> |
| |
| <pre><code>root ('--work_dir') |
| |-- slaves |
| | |-- latest (symlink) |
| | |-- <agent ID> |
| | |-- frameworks |
| | |-- <framework ID> |
| | |-- executors |
| | |-- <executor ID> |
| | |-- runs |
| | |-- latest (symlink) |
| | |-- <container ID> (Sandbox!) |
| </code></pre> |
| |
| <h2>Using the sandbox</h2> |
| |
| <p><strong>NOTE:</strong> For anything other than Mesos, the executor, or the task(s), the |
| sandbox should be considered a read-only directory. This is not enforced via |
| permissions, but the executor/tasks may malfunction if the sandbox is mutated |
| unexpectedly.</p> |
| |
| <h3>Via a file browser</h3> |
| |
| <p>If you have access to the machine running the agent, you can <a href="#where-is-it">navigate to the |
| sandbox directory directly</a>.</p> |
| |
| <h3>Via the Mesos web UI</h3> |
| |
| <p>Sandboxes can be browsed and downloaded via the Mesos web UI. Tasks and |
| executors will be shown with a “Sandbox” link. Any files that live in the |
| sandbox will appear in the web UI.</p> |
| |
| <h3>Via the <code>/files</code> endpoint</h3> |
| |
| <p>Underneath the web UI, the files are fetched from the agent via the <code>/files</code> |
| endpoint running on the agent.</p> |
| |
| <table class="table table-striped"> |
| <thead> |
| <tr> |
| <th width="30%"> |
| Endpoint |
| </th> |
| <th> |
| Description |
| </th> |
| </tr> |
| </thead> |
| |
| <tr> |
| <td> |
| <code>/files/browse?path=...</code> |
| </td> |
| <td> |
| Returns a JSON list of files and directories contained in the path. |
| Each list is a JSON object containing all the fields normally found in |
| <code>ls -l</code>. |
| </td> |
| </tr> |
| <tr> |
| <td> |
| <code>/files/debug</code> |
| </td> |
| <td> |
| Returns a JSON object holding the internal mapping of files managed by |
| this endpoint. This endpoint can be used to quickly fetch the paths |
| of all files exposed on the agent. |
| </td> |
| </tr> |
| <tr> |
| <td> |
| <code>/files/download?path=...</code> |
| </td> |
| <td> |
| Returns the raw contents of the file located at the given path. |
| Where the file extension is understood, the <code>Content-Type</code> |
| header will be set appropriately. |
| </td> |
| </tr> |
| <tr> |
| <td> |
| <code>/files/read?path=...</code> |
| </td> |
| <td> |
| Reads a chunk of the file located at the given path and returns a JSON |
| object containing the read <code>"data"</code> and the |
| <code>"offset"</code> in bytes. |
| <blockquote> |
| <p> |
| <strong>NOTE:</strong> This endpoint is not designed to read |
| arbitrary binary files. Binary files may be returned as |
| invalid/un-parseable JSON. |
| Use <code>/files/download</code> instead. |
| </p> |
| </blockquote> |
| Optional query parameters: |
| <ul> |
| <li><code>offset</code> - can be used to page through the file.</li> |
| <li><code>length</code> - maximum size of the chunk to read.</li> |
| </ul> |
| </td> |
| </tr> |
| </table> |
| |
| |
| <h2>Sandbox size</h2> |
| |
| <p>The maximum size of the sandbox is dependent on the containerization of the |
| executor and isolators:</p> |
| |
| <ul> |
| <li>Mesos containerizer - For backwards compatibility, the Mesos containerizer |
| does not enforce a container’s disk quota by default. However, if the |
| <code>--enforce_container_disk_quota</code> flag is enabled on the agent, and |
| <code>disk/du</code> is specified in the <code>--isolation</code> flag, the executor |
| will be killed if the sandbox size exceeds the executor’s <code>disk</code> resource.</li> |
| <li>Docker containerizer - As of Docker <code>1.9.1</code>, the Docker containerizer |
| does not enforce nor support a disk quota. See the |
| <a href="https://github.com/docker/docker/issues/3804">Docker issue</a>.</li> |
| </ul> |
| |
| |
| <h2>Sandbox lifecycle</h2> |
| |
| <p>Sandbox files are scheduled for garbage collection when:</p> |
| |
| <ul> |
| <li>An executor is removed or terminated.</li> |
| <li>A framework is removed.</li> |
| <li>An executor is recovered unsuccessfully during agent recovery.</li> |
| </ul> |
| |
| |
| <p><strong>NOTE:</strong> During agent recovery, all of the executor’s runs, except for the |
| latest run, are scheduled for garbage collection as well.</p> |
| |
| <p>Garbage collection is scheduled based on the <code>--gc_delay</code> agent flag. By |
| default, this is one week since the sandbox was last modified. |
| After the delay, the files are deleted.</p> |
| |
| <p>Additionally, according to the <code>--disk_watch_interval</code> agent flag, files |
| scheduled for garbage collection are pruned based on the available disk and |
| the <code>--gc_disk_headroom</code> agent flag. |
| See <a href="/documentation/latest/./configuration/#gc_disk_headroom">the formula here</a>.</p> |
| |
| </div> |
| </div> |
| |
| </div><!-- /.container --> |
| </div><!-- /.content --> |
| |
| <hr> |
| |
| |
| |
| <!-- footer --> |
| <div class="footer"> |
| <div class="container"> |
| <div class="col-md-4 social-blk"> |
| <span class="social"> |
| <a href="https://twitter.com/ApacheMesos" |
| class="twitter-follow-button" |
| data-show-count="false" data-size="large">Follow @ApacheMesos</a> |
| <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> |
| <a href="https://twitter.com/intent/tweet?button_hashtag=mesos" |
| class="twitter-hashtag-button" |
| data-size="large" |
| data-related="ApacheMesos">Tweet #mesos</a> |
| <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> |
| </span> |
| </div> |
| |
| <div class="col-md-8 trademark"> |
| <p>© 2012-2017 <a href="http://apache.org">The Apache Software Foundation</a>. |
| Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation. |
| <p> |
| </div> |
| </div><!-- /.container --> |
| </div><!-- /.footer --> |
| |
| <!-- JS --> |
| <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script> |
| <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script> |
| </body> |
| </html> |
