Primary storage is associated with a cluster or zone, and it stores the virtual disks for all the VMs running on hosts in that cluster/zone.
Secondary storage stores the following:
ROOT volumes correspond to the boot disk of a VM. They are created automatically by CloudStack during VM creation. ROOT volumes are created based on a system disk offering, corresponding to the service offering the user VM is based on. We may change the ROOT volume disk offering but only to another system created disk offering.
DATA volumes correspond to additional disks. These can be created by users and then attached/detached to VMs. DATA volumes are created based on a user-defined disk offering.
The StorPool plugin consists of two parts:
Source directory: ./apache-cloudstack-4.17-src/plugins/hypervisors/kvm
Source directory: ./apache-cloudstack-4.17.0-src/plugins/storage/volume
There is one plugin for both the CloudStack management and agents, in the hope that having all the source in one place will ease development and maintenance. The plugin itself though is separated into two mainly independent parts:
The plugin is intended to be self contained and non-intrusive, thus ideally deploying it would consist of only dropping the jar file into the appropriate places. This is the reason why all StorPool related communication (ex. data copying, volume resize) is done with StorPool specific commands even when there is a CloudStack command that does pretty much the same.
Note that for the present the StorPool plugin may only be used for a single primary storage cluster; support for multiple clusters is planned.
Go to the source directory and run:
mvn -Pdeveloper -DskipTests install
The resulting jar file is located in the target/ subdirectory.
Note: checkstyle errors: before compilation a code style check is performed; if this fails compilation is aborted. In short: no trailing whitespace, indent using 4 spaces, not tabs, comment-out or remove unused imports.
Note: Need to build both the KVM plugin and the StorPool plugin proper.
For each CloudStack management host:
scp ./target/cloud-plugin-storage-volume-storpool-{version}.jar {MGMT_HOST}:/usr/share/cloudstack-management/lib/
For each CloudStack agent host:
scp ./target/cloud-plugin-storage-volume-storpool-{version}.jar {AGENT_HOST}:/usr/share/cloudstack-agent/plugins/
Note: CloudStack managements/agents services must be restarted after adding the plugin to the respective directories
Note: Agents should have access to the StorPool management API, since attach and detach operations happens on the agent. This is a CloudStack design issue, can't do much about it.
Perform the StorPool installation following the StorPool Installation Guide.
Create a template to be used by CloudStack. Must set placeHead, placeAll, placeTail and replication. No need to set default volume size because it is determined by the CloudStack disks and services offering.
From the WEB UI, go to Infrastructure -> Primary Storage -> Add Primary Storage
Scope: select Zone-Wide Hypervisor: select KVM Zone: pick appropriate zone. Name: user specified name
Protocol: select SharedMountPoint Path: enter /dev/storpool (required argument, actually not needed in practice).
Provider: select StorPool Managed: leave unchecked (currently ignored) Capacity Bytes: used for accounting purposes only. May be more or less than the actual StorPool template capacity. Capacity IOPS: currently not used (may use for max IOPS limitations on volumes from this pool). URL: enter SP_API_HTTP=address:port;SP_AUTH_TOKEN=token;SP_TEMPLATE=template_name. At present one template can be used for at most one Storage Pool.
SP_API_HTTP - address of StorPool Api SP_AUTH_TOKEN - StorPool‘s token SP_TEMPLATE - name of StorPool’s template
Storage Tags: If left blank, the StorPool storage plugin will use the pool name to create a corresponding storage tag. This storage tag may be used later, when defining service or disk offerings.
NOTE: When using multicluster for each CloudStack cluster in its settings set the value of StorPool's SP_CLUSTER_ID in “sp.cluster.id”.
NOTE: Secondary storage could be bypassed with Configuration setting “sp.bypass.secondary.storage” set to true. In this case only snapshots won't be downloaded to secondary storage.
The snapshot exists only on PRIMARY (StorPool) storage. From this snapshot it will be created a template on SECONADRY.
TODO: Maybe we should not use CloudStack functionality, and to use that one when bypass option is enabled
This is independent of StorPool as snapshots exist on secondary.
When creating the first volume based on the given template, if snapshot of the template does not exists on StorPool it will be first downloaded (cached) to PRIMARY storage. This is mapped to a StorPool snapshot so, creating succecutive volumes from the same template does not incur additional copying of data to PRIMARY storage.
This cached snapshot is garbage collected when the original template is deleted from CloudStack. This cleanup is done by a background task in CloudStack.
We just need to create the volume. The ISO installation is handled by CloudStack.
DATA volumes are created by CloudStack the first time it is attached to a VM.
We use the fact that the snapshot already exists on PRIMARY, so no data is copied. We will copy snapshots from SECONDARY to StorPool PRIMARY, when there is no corresponding StorPool snapshot.
We need to send a resize cmd to agent, where the VM the volume is attached to is running, so that the resize is visible by the VM.
The snapshot is first created on the PRIMARY storage (i.e. StorPool), then backed-up on SECONDARY storage (tested with NFS secondary) if bypass option is not enabled. The original StorPool snapshot is kept, so that creating volumes from the snapshot does not need to copy the data again to PRIMARY. When the snapshot is deleted from CloudStack so is the corresponding StorPool snapshot.
Currently snapshots are taken in RAW format.
It's handled by StorPool
Tested with storage pools on NFS only.
StorPool supports consistent snapshots of volumes attached to a virtual machine.
Max IOPS are kept in StorPool's volumes with the help of custom service offerings, by adding IOPS limits to the corresponding system disk offering.
CloudStack has no way to specify max BW. Do they want to be able to specify max BW only is sufficient.
Supported Virtual machine operations - live migration of VM to another host, virtual machine snapshots (group snapshot without memory), revert VM snapshot, delete VM snapshot
Supported Volume operations - attach/detach volume, live migrate volume between two StorPool primary storages, volume snapshot, delete snapshot, revert snapshot
Note: volume snapshot are allowed only when sp.bypass.secondary.storage is set to true. This means that the snapshots are not backed up to secondary storage