Google Git
Sign in
apache / mesos-site / e7fb58271cdc54f9a9222da662d2fa6169f7c5c3 / . / publish / documentation / latest / modules / index.html
blob: 133eca8ef0afb4e8348a51489fb1bd27871d442f [file] [log] [blame]
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Apache Mesos - Modules</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta property="og:locale" content="en_US"/>
<meta property="og:type" content="website"/>
<meta property="og:title" content="Apache Mesos"/>
<meta property="og:site_name" content="Apache Mesos"/>
<meta property="og:url" content="http://mesos.apache.org/"/>
<meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
<meta property="og:description"
content="Apache Mesos abstracts resources away from machines,
enabling fault-tolerant and elastic distributed systems
to easily be built and run effectively."/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:site" content="@ApacheMesos"/>
<meta name="twitter:title" content="Apache Mesos"/>
<meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/>
<meta name="twitter:description"
content="Apache Mesos abstracts resources away from machines,
enabling fault-tolerant and elastic distributed systems
to easily be built and run effectively."/>
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
<link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
<!-- Google Analytics Magic -->
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-20226872-1']);
_gaq.push(['_setDomainName', 'apache.org']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<!-- magical breadcrumbs -->
<div class="topnav">
<div class="container">
<ul class="breadcrumb">
<li>
<div class="dropdown">
<a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
<ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
<li><a href="http://www.apache.org">Apache Homepage</a></li>
<li><a href="http://www.apache.org/licenses/">License</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
<li><a href="http://www.apache.org/security/">Security</a></li>
</ul>
</div>
</li>
<li><a href="http://mesos.apache.org">Apache Mesos</a></li>
<li><a href="/documentation
/">Documentation
</a></li>
</ul><!-- /.breadcrumb -->
</div><!-- /.container -->
</div><!-- /.topnav -->
<!-- navbar excitement -->
<div class="navbar navbar-default navbar-static-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false">
<span class="sr-only">Toggle navigation</span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a>
</div><!-- /.navbar-header -->
<div class="navbar-collapse collapse" id="mesos-menu">
<ul class="nav navbar-nav navbar-right">
<li><a href="/gettingstarted/">Getting Started</a></li>
<li><a href="/blog/">Blog</a></li>
<li><a href="/documentation/latest/">Documentation</a></li>
<li><a href="/downloads/">Downloads</a></li>
<li><a href="/community/">Community</a></li>
</ul>
</div><!-- /#mesos-menu -->
</div><!-- /.container -->
</div><!-- /.navbar -->
<div class="content">
<div class="container">
<div class="row-fluid">
<div class="col-md-4">
<h4>If you're new to Mesos</h4>
<p>See the <a href="/gettingstarted/">getting started</a> page for more
information about downloading, building, and deploying Mesos.</p>
<h4>If you'd like to get involved or you're looking for support</h4>
<p>See our <a href="/community/">community</a> page for more details.</p>
</div>
<div class="col-md-8">
<h1>Mesos Modules</h1>
<p>Experimental support for Mesos modules was introduced in Mesos 0.21.0.</p>
<h3>Disclaimer</h3>
<ul>
<li><p>Use and development of Mesos modules is at own risk!</p></li>
<li><p>Please direct all questions to the relevant module writer and/or write to
modules@mesos.apache.org. Questions related to modules sent to user and dev list
will be redirected to the modules list.</p></li>
</ul>
<h2>What are Mesos modules?</h2>
<p>Mesos modules provide a way to easily extend inner workings of Mesos by creating
and using shared libraries that are loaded on demand. Modules can be used to
customize Mesos without having to recompiling/relinking for each specific use
case. Modules can isolate external dependencies into separate libraries, thus
resulting into a smaller Mesos core.
Modules also make it easy to experiment with new features.
For example, imagine loadable allocators that contain a VM (Lua, Python, &hellip;)
which makes it possible to try out new allocator algorithms written in
scripting languages without forcing those dependencies into the project.
Finally, modules provide an easy way for third parties to easily extend Mesos
without having to know all the internal details.</p>
<h2>Where can Mesos modules be used?</h2>
<p>Modules can be specified for master, agent and tests. Modules can also be used
with schedulers that link against libmesos. Currently, modules cannot be
used with executors.</p>
<h2><a name="Invoking"></a>Invoking Mesos modules</h2>
<p>There are two ways to specify a list of modules to be loaded and be available to
the internal subsystems.</p>
<ul>
<li><code>--modules=filepath</code> to specify the list of modules via a manifest file
containing a JSON-formatted string. &lsquo;filepath&rsquo; can be of the form
&lsquo;file:///path/to/file&rsquo; or &lsquo;/path/to/file&rsquo;.</li>
</ul>
<p>Use <code>--modules="{...}"</code> to specify the list of modules inline.</p>
<ul>
<li><code>--modules_dir=dirpath</code>to specify a directory name that contains several
module manifest files. Each file must contain a valid JSON-formatted
string containing module description. The module manifest files are processed
in alphabetical order.</li>
</ul>
<p>NOTE: Only one of <code>--modules</code> or <code>--modules_dir</code> flag can be used at time.</p>
<h3>Example JSON strings:</h3>
<ol>
<li><p>Load a library <code>libfoo.so</code> with two modules <code>org_apache_mesos_bar</code> and
<code>org_apache_mesos_baz</code>.</p>
<pre><code> {
"libraries": [
{
"file": "/path/to/libfoo.so",
"modules": [
{
"name": "org_apache_mesos_bar",
},
{
"name": "org_apache_mesos_baz"
}
]
}
]
}
</code></pre></li>
<li><p>Load the module <code>org_apache_mesos_bar</code> from the library <code>foo</code> and pass
the command-line argument <code>X</code> with value <code>Y</code> (module <code>org_apache_mesos_baz</code>
is loaded without any command-line parameters):</p>
<pre><code> {
"libraries": [
{
"name": "foo",
"modules": [
{
"name": "org_apache_mesos_bar"
"parameters": [
{
"key": "X",
"value": "Y",
}
]
},
{
"name": "org_apache_mesos_baz"
}
]
}
]
}
</code></pre></li>
<li><p>Specifying modules inline:</p>
<pre><code> --modules='{"libraries":[{"file":"/path/to/libfoo.so", "modules":[{"name":"org_apache_mesos_bar"}]}]}'
</code></pre></li>
</ol>
<h3>Library names</h3>
<p>For each library, at least one of the &ldquo;file&rdquo; or &ldquo;name&rdquo; parameter must be
specified. The &ldquo;file&rdquo; parameter may refer to a filename (e.g., &ldquo;libfoo.so&rdquo;), a
relative path (e.g., &ldquo;myLibs/libfoo.so&rdquo;), or an absolute path (e.g.,
&ldquo;/home/mesos/lib/libfoo.so&rdquo;). The &ldquo;name&rdquo; parameter refers to a library name
(e.g., &ldquo;foo&rdquo;). If &ldquo;name&rdquo; is specified, it is automatically expanded to a proper
library name for the current platform (e.g., &ldquo;foo&rdquo; gets expanded to &ldquo;libfoo.so&rdquo;
on Linux, and &ldquo;libfoo.dylib&rdquo; on OS X).</p>
<p>If a library path is not specified in the &ldquo;file&rdquo; parameter, the library is
searched in the standard library paths or directories pointed to by the
<code>LD_LIBRARY_PATH</code> (<code>DYLD_LIBRARY_PATH</code> on OS X) environment variable.</p>
<p>If both &ldquo;file&rdquo; and &ldquo;name&rdquo; are specified, &ldquo;name&rdquo; is ignored.</p>
<h2>What kinds of modules are supported?</h2>
<p>Here are the various module kinds currently available.</p>
<h3>Allocator</h3>
<p>The Mesos master&rsquo;s <em>allocator</em> periodically determines which framework(s) should be offered the cluster&rsquo;s available resources. Allocator modules enable experimenting with specialized resource allocation algorithms. An example of these could be an allocator that provides a feature currently not supported by the built-in Hierarchical Dominant Resource Fairness allocator, like oversubscription with preemption.</p>
<p>To load a custom allocator into Mesos master, you need to:</p>
<ul>
<li><p>Introduce it to the Mesos master by listing it in the <code>--modules</code> configuration,</p></li>
<li><p>Select it as the allocator via the <code>--allocator</code> flag.</p></li>
</ul>
<p>For example, the following command will run the Mesos master with <code>ExternalAllocatorModule</code> (see <a href="#Example-JSON-strings">this section</a> for JSON format):</p>
<pre><code>./bin/mesos-master.sh --work_dir=m/work --modules="file://&lt;modules-including-allocator&gt;.json" --allocator=ExternalAllocatorModule
</code></pre>
<h3>Anonymous</h3>
<p>Anonymous modules do <strong>not</strong> receive any callbacks but simply
coexists with their parent process.</p>
<p>Unlike other named modules, an anonymous module does not directly
replace or provide essential Mesos functionality (such as an
Isolator module does). Unlike a decorator module it does not
directly add or inject data into Mesos core either.</p>
<p>Anonymous modules do not require any specific selectors (flags) as
they are immediately instantiated when getting loaded via the
<code>--modules</code> flag by the Mesos master or agent.</p>
<h3>Authentication</h3>
<p>Authenticatee and Authenticator modules allow for third parties to quickly
develop and plug-in new authentication methods. An example for such modules
could be to support PAM (LDAP, MySQL, NIS, UNIX) backed authentication.</p>
<h3>Hook</h3>
<p>Similar to Apache Webserver Modules, hooks allows module writers to tie into
internal components which may not be suitable to be abstracted entirely behind
modules but rather lets them define actions on so-called hooks.</p>
<p>The available hooks API is defined in mesos/hook.hpp and for each hook defines
the insertion point and available context. An example of this context is the
task information which is passed to masterLaunchTaskHook.</p>
<p>Some hooks take in an object (e.g. TaskInfo) and return all or part of that
object (e.g. task labels), so that the hook can modify or replace the contents
in-flight. These hooks are referred to as <em>decorators</em>.</p>
<p>In order to enable decorator modules to remove metadata (environment variables
or labels), the effect of the return value for decorator hooks changed in
Mesos 0.23.0.</p>
<p>The Result<T> return values before and after Mesos 0.23.0 means:</p>
<table class="table table-striped">
<tr>
<th>State</th>
<th>Before (0.22.x)</th>
<th>After (0.23.0+)</th>
</tr>
<tr>
<td>Error</td>
<td>Error is propagated to the call-site</td>
<td>No change</td>
</tr>
<tr>
<td>None</td>
<td>The result of the decorator is not applied</td>
<td>No change</td>
</tr>
<tr>
<td>Some</td>
<td>The result of the decorator is appended</td>
<td>The result of the decorator overwrites the final labels/environment
object</td>
</tr>
</table>
<p>To load a hook into Mesos, you need to:</p>
<ul>
<li><p>introduce it to Mesos by listing it in the <code>--modules</code> configuration,</p></li>
<li><p>select it as a hook module via the <code>--hooks</code> flag.</p></li>
</ul>
<p>For example, the following command will run the Mesos agent with the
<code>TestTaskHook</code> hook:</p>
<pre><code>./bin/mesos-agent.sh --master=&lt;IP&gt;:&lt;PORT&gt; --modules="file://&lt;path-to-modules-config&gt;.json" --hooks=TestTaskHook
</code></pre>
<h3>Isolator</h3>
<p>Isolator modules enable experimenting with specialized isolation and monitoring
capabilities. Examples of these could be 3rdparty resource isolation mechanisms
for GPGPU hardware, networking, etc.</p>
<p>If a custom isolator module needs to checkpoint state to disk, it should do this
on its own; Mesos does not provide public helpers to checkpoint module state.
The standard location for such stored state is:</p>
<pre><code>&lt;runtime_dir&gt;/isolators/&lt;name_of_isolator&gt;/
</code></pre>
<h3>Master Contender and Detector</h3>
<p>Contender and Detector modules enable developers to implement custom leader
election and master detection mechanisms, other than relying on Zookeeper by
default.</p>
<p>An example for such modules could be to use distributed Key-Value storage such
as <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://www.consul.io/">consul</a>.</p>
<p>To load custom contender and detector module, you need to:</p>
<ul>
<li><p>Supply <code>--modules</code> when running Mesos master,</p></li>
<li><p>Specify selected contender and detector modules with <code>--master_contender</code> and
<code>--master_detector</code> flags on Mesos Master and <code>--master_detector</code> on Mesos Slave.</p></li>
</ul>
<p>For example, the following command runs Mesos master with
<code>org_apache_mesos_TestMasterContender</code> and <code>org_apache_mesos_TestMasterDetector</code>:</p>
<p><code>./bin/mesos-master.sh --modules="file://&lt;path-to-modules-config&gt;.json" --master_contender=org_apache_mesos_TestMasterContender --master_detector=org_apache_mesos_TestMasterDetector</code></p>
<p>And this one runs Mesos slave with <code>org_apache_mesos_TestMasterDetector</code>:</p>
<p><code>./bin/mesos-slave.sh --modules="file://&lt;path-to-modules-config&gt;.json" --master_detector=org_apache_mesos_TestMasterDetector</code></p>
<h2>Writing Mesos modules</h2>
<h3>A HelloWorld module</h3>
<p>The following snippet describes the implementation of a module named
&ldquo;org_apache_mesos_bar&rdquo; of &ldquo;TestModule&rdquo; kind:</p>
<pre><code class="{.cpp}">#include &lt;iostream&gt;
#include "test_module.hpp"
class TestModuleImpl : public TestModule
{
public:
TestModuleImpl()
{
std::cout &lt;&lt; "HelloWorld!" &lt;&lt; std::endl;
}
virtual int foo(char a, long b)
{
return a + b;
}
virtual int bar(float a, double b)
{
return a * b;
}
};
static TestModule* create()
{
return new TestModule();
}
static bool compatible()
{
return true;
}
// Declares a module named 'org_apache_mesos_TestModule' of
// 'TestModule' kind.
// Mesos core binds the module instance pointer as needed.
// The compatible() hook is provided by the module for compatibility checks.
// The create() hook returns an object of type 'TestModule'.
mesos::modules::Module&lt;TestModule&gt; org_apache_mesos_TestModule(
MESOS_MODULE_API_VERSION,
MESOS_VERSION,
"Apache Mesos",
"modules@mesos.apache.org",
"This is a test module.",
compatible,
create);
</code></pre>
<h3>Building a module</h3>
<p> The following assumes that Mesos is installed in the standard location, i.e.
the Mesos dynamic library and header files are available.</p>
<pre><code>g++ -lmesos -fpic -o test_module.o test_module.cpp
$ gcc -shared -o libtest_module.so test_module.o
</code></pre>
<h3>Testing a module</h3>
<p>Apart from testing the module by hand with explicit use of &ndash;modules flag, one
can run the entire mesos test suite with the given module. For example, the
following command will run the mesos test suite with the
<code>org_apache_mesos_TestCpuIsolator</code> module selected for isolation:</p>
<pre><code>./bin/mesos-tests.sh --modules="/home/kapil/mesos/isolator-module/modules.json" --isolation="org_apache_mesos_TestCpuIsolator"
</code></pre>
<h3>Module naming convention</h3>
<p>Each module name should be unique. Having duplicate module names in the Json
string will cause the process to abort.</p>
<p>Therefore, we encourage module writers to name their modules according to Java
package naming scheme
(http://docs.oracle.com/javase/tutorial/java/package/namingpkgs.html).</p>
<p>For example:</p>
<table class="table table-striped">
<tr>
<th> Module Name </th> <th> Module Domain name </th> <th> Module Symbol Name </th>
</tr>
<tr>
<td> foobar </td> <td> mesos.apache.org </td> <td> org_apache_mesos_foobar </td>
</tr>
<tr>
<td> barBaz </td> <td> example.com </td> <td> com_example_barBaz </td>
</table>
<p>In short:
- Keep case of module name.
- Lower case and reverse domain.
- Separate with underscore.
- Do not simply use the kind name as module name.
- Different modules from the same organization still need different names.</p>
<h2>Module Versioning and backwards compatibility</h2>
<p>Before loading the above module, a dynamic library that contains the module
needs to be loaded into Mesos. This happens early in Mesos startup code. The
Mesos developer does not need to touch that code when introducing new module
kinds. However, the developer is responsible for registering what versions of
any given module are expected to remain compatible with Mesos as it progresses.
This information is maintained in a table in <code>src/module/manager.cpp</code>. It
contains an entry for every possible module kind that Mesos recognizes, each
with a corresponding Mesos release version number. This number needs to be
adjusted by the Mesos developer to reflect the current Mesos version whenever
compatibility between Mesos and modules that get compiled against it gets
broken. Given that module implementation for older Mesos versions can still be
written in the future, this may be impossible to tell and so in doubt it is best
to just bump the required module version to the current Mesos version. But if
one can be reasonably sure, assuming cooperative module developers, that a
certain kind of module will continue to function across several Mesos versions,
the table provides an easy way to specify this.</p>
<p>For successfully loading the module, the following relationship
must exist between the various versions:</p>
<p><code> kind version &lt;= Library version &lt;= Mesos version</code></p>
<table class="table table-striped">
<tr>
<td>Mesos </td> <td>kind version </td> <td> Library </td> <td>Is module loadable </td> <td>Reason </td>
</tr>
<tr>
<td>0.18.0 </td> <td> 0.18.0 </td> <td> 0.18.0 </td> <td> yes </td> <td> </td>
</tr>
<tr>
<td>1.0.0 </td> <td> 0.18.0 </td> <td> 0.18.0 </td> <td> yes </td> <td> </td>
</tr>
<tr>
<td>1.0.0 </td> <td> 0.18.0 </td> <td> 0.21.0 </td> <td> yes </td> <td> </td>
</tr>
<tr>
<td>0.18.0 </td> <td> 0.18.0 </td> <td> 1.0.0 </td> <td> NO </td> <td> Library compiled against a newer Mesos release. </td>
</tr>
<tr>
<td>1.0.0 </td> <td> 0.21.0 </td> <td> 0.18.0 </td> <td> NO </td> <td> Module/Library older than the kind version supported by Mesos. </td>
</tr>
<tr>
<td>1.0.0 </td> <td> 1.0.0 </td> <td> 0.18.0 </td> <td> NO </td> <td> Module/Library older than the kind version supported by Mesos. </td>
</tr>
</table>
<h2>Mesos module API changes</h2>
<p>Record of incompatible changes to the modules API.</p>
<h3>Version 2</h3>
<ul>
<li>Added support for module-specific command-line parameters.</li>
<li>Changed function signature for create().</li>
</ul>
<h3>Version 1</h3>
<p>Initial version of the modules API.</p>
<h2>Appendix:</h2>
<h3>JSON Schema:</h3>
<pre><code>{
"type":"object",
"required":false,
"properties":{
"libraries":{
"type":"array",
"required":false,
"items":{
"type":"object",
"required":false,
"properties":{
"file":{
"type":"string",
"required":false
},
"name":{
"type":"string",
"required":false
},
"modules":{
"type":"array",
"required":false,
"items":{
"type":"object",
"required":false,
"properties":{
"name":{
"type":"string",
"required":true
},
"parameters":{
"type":"array",
"required":false,
"items":{
"type":"object",
"required":false,
"properties":{
"key":{
"type":"string",
"required":true
},
"value":{
"type":"string",
"required":true
}
}
}
}
}
}
}
}
}
}
}
}
</code></pre>
</div>
</div>
</div><!-- /.container -->
</div><!-- /.content -->
<hr>
<!-- footer -->
<div class="footer">
<div class="container">
<div class="col-md-4 social-blk">
<span class="social">
<a href="https://twitter.com/ApacheMesos"
class="twitter-follow-button"
data-show-count="false" data-size="large">Follow @ApacheMesos</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
<a href="https://twitter.com/intent/tweet?button_hashtag=mesos"
class="twitter-hashtag-button"
data-size="large"
data-related="ApacheMesos">Tweet #mesos</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
</span>
</div>
<div class="col-md-8 trademark">
<p>&copy; 2012-2017 <a href="http://apache.org">The Apache Software Foundation</a>.
Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.
<p>
</div>
</div><!-- /.container -->
</div><!-- /.footer -->
<!-- JS -->
<script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
<script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
</body>
</html>