Mesos provides a mechanism to create a persistent volume from disk resources. When launching a task, you can create a volume that exists outside the task‘s sandbox and will persist on the node even after the task dies or completes. When the task exits, its resources -- including the persistent volume -- can be offered back to the framework, so that the framework can launch the same task again, launch a recovery task, or launch a new task that consumes the previous task’s output as its input. Persistent volumes enable stateful services such as HDFS and Cassandra to store their data within Mesos rather than having to resort to workarounds (e.g., writing task state to a distributed filesystem that is mounted at a well-known location outside the task's sandbox).
Persistent volumes can only be created from reserved disk resources, whether it be statically reserved or dynamically reserved. A dynamically reserved persistent volume also cannot be unreserved without having explicitly destroyed the volume. These rules exist to limit the accidental mistakes such as: a persistent volume containing sensitive data being offered to other frameworks in the cluster.
Please refer to the Reservation documentation for details regarding reservation mechanisms available in Mesos.
Persistent volumes can be created by operators and authorized frameworks. We require a principal
from the operator or framework in order to authenticate/authorize the operations. Authorization is specified via the existing ACL mechanism. (Coming Soon)
Offer::Operation::Create
and Offer::Operation::Destroy
messages are available for frameworks to send back via the acceptOffers
API as a response to a resource offer./create
and /destroy
HTTP endpoints are available for operators to manage persistent volumes through the master. (Coming Soon).In the following sections, we will walk through examples of each of the interfaces described above.
Offer::Operation::Create
A framework is able to create volumes through the resource offer cycle. Suppose we receive a resource offer with 2048 MB of dynamically reserved disk.
{ "id" : <offer_id>, "framework_id" : <framework_id>, "slave_id" : <slave_id>, "hostname" : <hostname>, "resources" : [ { "name" : "disk", "type" : "SCALAR", "scalar" : { "value" : 2048 }, "role" : <framework_role>, "reservation" : { "principal" : <framework_principal> } } ] }
We can create a persistent volume from the 2048 MB of disk resources by sending the following Offer::Operation
message via the acceptOffers
API. Offer::Operation::Create
has a volumes
field which we specify with the persistent volume information. We need to specify the following:
"RW"
is the only possible value.{ "type" : Offer::Operation::CREATE, "create": { "volumes" : [ { "name" : "disk", "type" : "SCALAR", "scalar" : { "value" : 2048 }, "role" : <framework_role>, "reservation" : { "principal" : <framework_principal> }, "disk": { "persistence": { "id" : <persistent_volume_id> }, "volume" : { "container_path" : <container_path>, "mode" : <mode> } } } ] } }
The subsequent resource offer will contain the following persistent volume:
{ "id" : <offer_id>, "framework_id" : <framework_id>, "slave_id" : <slave_id>, "hostname" : <hostname>, "resources" : [ { "name" : "disk", "type" : "SCALAR", "scalar" : { "value" : 2048 }, "role" : <framework_role>, "reservation" : { "principal" : <framework_principal> }, "disk": { "persistence": { "id" : <persistent_volume_id> }, "volume" : { "container_path" : <container_path>, "mode" : <mode> } } } ] }
Offer::Operation::Destroy
A framework is able to destroy persistent volumes through the resource offer cycle. In Offer::Operation::Create, we created a persistent volume from 2048 MB of disk resources. Mesos will not garbage-collect this volume until we explicitly destroy it. Suppose we would like to destroy the volume we created. First, we receive a resource offer (copy/pasted from above):
{ "id" : <offer_id>, "framework_id" : <framework_id>, "slave_id" : <slave_id>, "hostname" : <hostname>, "resources" : [ { "name" : "disk", "type" : "SCALAR", "scalar" : { "value" : 2048 }, "role" : <framework_role>, "reservation" : { "principal" : <framework_principal> }, "disk": { "persistence": { "id" : <persistent_volume_id> }, "volume" : { "container_path" : <container_path>, "mode" : <mode> } } } ] }
We destroy the persistent volume by sending the Offer::Operation
message via the acceptOffers
API. Offer::Operation::Destroy
has a volumes
field which we specify the persistent volumes to be destroyed.
{ "type" : Offer::Operation::DESTROY, "destroy" : { "volumes" : [ { "name" : "disk", "type" : "SCALAR", "scalar" : { "value" : 2048 }, "role" : <framework_role>, "reservation" : { "principal" : <framework_principal> }, "disk": { "persistence": { "id" : <persistent_volume_id> }, "volume" : { "container_path" : <container_path>, "mode" : <mode> } } } ] } }
The persistent volume will be destroyed, but the disk resources will still be reserved. As such, the subsequent resource offer will contain the following reserved disk resources:
{ "id" : <offer_id>, "framework_id" : <framework_id>, "slave_id" : <slave_id>, "hostname" : <hostname>, "resources" : [ { "name" : "disk", "type" : "SCALAR", "scalar" : { "value" : 2048 }, "role" : <framework_role>, "reservation" : { "principal" : <framework_principal> } } ] }
Note that in 0.23, even after you destroy the persistent volume, its content will still be on the disk. The garbage collection for persistent volumes is coming soon: MESOS-2048.
/create
(Coming Soon)/destroy
(Coming Soon)