The Mesos development roadmap can roughly be separated into 3 main areas:

  1. Building a community and promoting adoption
  2. Core Mesos development
  3. Mesos Application development

1. Growing a development community and promoting adoption

  1. Migrate into Apache Incubator (see the incubator proposal)
    1. Migrate issues from github issues into JIRA
  2. Documentation management and organization

2. Core Mesos (e.g. scheduling, resource management, etc.)

  1. More advanced allocation modules that implement the following functionality
    1. Resource revocation
    2. Resource inheritance, hierarchical scheduling
    3. A Mesos Service Level Objective
    4. Scheduling based on history
  2. Migrate to ProtocolBuffers as the serialization format (under development at Twitter)
  3. User authentication
  4. Mesos application debugging support
    1. More advanced User Interface and Event History (i.e. logging) - See issue #143 for more details.
  5. Testing infrastructure, and more tests!

3. Mesos applications

This category of future work is probably the most important! Technically speaking, a mesos application is defined as a mesos scheduler plus a mesos executor. Practically speaking, applications can serve many purposes. These can be broken down into a few categories.

3.1 Applications providing cluster OS functionality (e.g. storage, synchronization, naming...)

The core of Mesos has been designed using the same philosophy behind traditional Microkernel operating systems. This means that the core of Mesos (the kernel, if you will) provides a minimal set of low-level abstractions for cluster resources (see Mesos Architecture for an introduction to the resource offer abstraction). This, in turn means that higher level abstractions (analogous to the filesystem, memory sharing, etc. in traditional operating systems, as well as some abstractions that have no analog in traditional single node operating systems such as DNS).

those which are primarily intended to be used by other applications (as opposed to being used by human users). They can be seen as operating system modules that implement functionality in the form of services.

  1. Meta Framework Development (under development at Berkeley, and also at Twitter)
    1. Allow users to submit a job (specifying their resource constraints) and have the job wait in a queue until the resources are acquired (the Application Scheduler translates those constraints into accepting or rejecting resource offers)
  2. Slave data storage/caching services (a generalization of MapReduce's map output server)
  3. Distributed file systems, like HDFS
  4. Further Hypertable integration
  5. Mesos package management application (i.e. the “apt-get” of the cluster... apt-get install hadoop-0.20.0)
  6. A machine metadata database

3.2 Applications providing user facing services (e.g. web apps, PL abstractions...)

This category of applications is intended to interface with users. Due to nature of distributed applications (i.e. vs. what can be solved by simply using single computer applications) these apps tend to either (a) serving thousands to millions of users at a time (e.g. web applications), (b) large parallel computations (like MPI style jobs), (c) data intensive (e.g. enabling data analytics at large scale), or some combination of the above.

  1. Spark
  2. A graph processing framework?
  3. A streaming database framework?
  4. Web applications (that server users)
  5. High performance computing, MPI style jobs...

Research Roadmap

In addition to a popular, (increasingly) stable, and (soon) production system, Mesos is also a research project that is pushing the borders of distributed system research! Check out the “About Mesos” page for links to Mesos research papers, experiments, and other information about Mesos research.