ScaleIO: Storage Plugin (Phase 0+1) (#77) * scaleio: prototype storage plugin - plugin skeleton - add storage pool, create/attach data disk Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> * kvm: attach disk example Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com> * Updated ScaleIO storage plugin to support Volume operations * ScaleIO storage plugin - Support for VM operations and other updates * ScaleIO storage pool plugin changes - Added validation to check existing ScaleIO storage pool and update capacity details - Updated resize volume for ScaleIO to pick the rounded 8GB boundary size - Added support for setting ScaleIO storage pool statistics (bandwidthLimitInKbps, iopsLimit) * Fixed IOPS validation and volume size update when resizing ScaleIO volume * Removed connect/disconnect disk changes from ScaleIO storage adaptor - ScaleIO datastore driver does map/unmap ScaleIO volume (from MS) using grant/revoke access - Not required to map/unmap ScaleIO volume from the storage adaptor * Updated connect disk, to wait for ScaleIO volume to become available in the KVM host * Updated ScaleIO storage provider, pool type, url scheme and related paramters to the new "PowerFlex" brand * Fixed size rounding issue while creating PowerFlex volume and added validations to PowerFlex Gateway API client * Updated host sdc connection check for ScaleIO/PowerFlex pool on host connect * Updated volume snapshots support for volumes on ScaleIO/PowerFlex storage pool and Added some validations for ScaleIO disks in host * Added primary storage level configurable setting "storage.pool.disk.wait" to wait for disk availability - Confiure the disk availability wait time, mainly introduced for ScaleIO/PowerFlex storage pool (can be used for other managed storages), to wait for the disk to become available in the host before performing any operation on it * Enabled template spooling to ScaleIO/PowerFlex storage pool and create VM from the spooled template. Added ScaleIO SDC limits support for volumes using offering parameters: bandwidthLimitInKbps, iopsLimit. * Added support for VM snapshots on ScaleIO/PowerFlex storage pool Minor improvements for IOPS (SDC Limits) configuration * Updated access for ScaleIO/PowerFlex volumes on VM Start and Stop Added primary storage level configurable setting "storage.pool.client.timeout" for storage API client Enabled cluster wide storage pool support for ScaleIO/PowerFlex storage Minor improvements for ScaleIO/PowerFlex disk access in the KVM host * Added support for direct download of templates (raw, qcow2) on ScaleIO/PowerFlex storage pool * Added support for config drives in host cache for KVM - Changed configuration "vm.configdrive.primarypool.enabled" scope from Global to Zone level - Introduced new zone level configuration "vm.configdrive.force.host.cache.use" (default: false) to force host cache for config drives - Introduced new zone level configuration "vm.configdrive.use.host.cache.on.unsupported.pool" (default: true) to use host cache for config drives when storage pool doesn't support config drive - Added new parameter "vm.configdrive.host.cache.location" (default: /var/cache/cloud) in KVM agent.properties for specifying the host cache path for config drives * Updated disk access while migrating the VM with volumes on ScaleIO/PowerFlex storage pool Changed the parameter "vm.configdrive.host.cache.location" to "host.cache.location" (default: /var/cache/cloud) in KVM agent.properties to specify the host cache path Changes to create config drives on the "/config" directory on the host cache path Changes to suppport migrate VM with config drive on the host cache path * Additonal changes to support migrate VM with config drive on the host cache * Detect virtual size from the template URL while registering direct download qcow2 (of KVM hypervisor) templates Updated full deployment destination for preparing the network(s) on VM start * Propagate the direct download certificates uploaded to the newly added KVM hosts * Code improvements for ScaleIO/PowerFlex storage plugin * Updated storage stats collection and tests for ScaleIO/PowerFlex storage plugin * Fix for template size of direct download templates on capacity check for ScaleIO/PowerFlex storage pool Updated data object grant and revoke access for connected SDCs to ScaleIO/PowerFlex storage pool * Discover the template size for direct download templates using any available host from the zones specified on template registration When zones are not specified while registering template, template size discovery is performed using any available host, which is picked up randomly from one of the available zones * Maintain the config drive location and use it when required on any config drive operation (migrate, delete) * Ensure the volume to be expunged, is expunge ready on storage cleanup * Do not set the storage migration flag for the volumes on zone wide PowerFlex/ScaleIO pool when listing the hosts available for cross-cluster migration * Release the VM resources when VM is sync-ed to Stopped state on PowerReportMissing (after graceful period) * Added alerts for PowerFlex/ScaleIO SDC disconnection on the host(s) * Retry VM deployment/start when the host cannot access volume/template on the ScaleIO/PowerFlex storage * Changes to find a potential host that can access the ScaleIO/PowerFlex storage pool * Updated ScaleIO/PowerFlex storage pool stats for checking the available capacity and usage * Updated ScaleIO/PowerFlex volumes naming convention to avoid the naming conflicts on sharing * Mark never-used or downloaded templates as Destroyed on deletion, without sending any DeleteCommand - Do not trigger any DeleteCommand for never-used or downloaded templates as these doesn't exist and cannot be deleted from the datastore * Updated ScaleIO/PowerFlex storage pool capacity stats * Cleanup unused templates and host entries on PowerFlex/ScaleIO storage pool deletion * Check the router filesystem is writable or not, before performing health checks - Introduce a new test "filesystem.writable.test" to check the filesystem is writable or not - The router health checks keeps the config info at "/var/cache/cloud" and updates the monitor results at "/root" for health checks, both are different partitions. So, test at both the locations. * Updated the router filesystem writable check using script, instead cmd execution - Added new script: "filesystem_writable_check.py" at /opt/cloud/bin/ to check the filesystem is writable or not * Update volume stats (physical and virtual size) for the volumes on PowerFlex/ScaleIO storage pool Co-authored-by: Suresh Kumar Anaparti <suresh.anaparti@shapeblue.com>
Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. CloudStack is used by a number of service providers to offer public cloud services, and by many companies to provide an on-premises (private) cloud offering, or as part of a hybrid cloud solution.
CloudStack is a turnkey solution that includes the entire “stack” of features most organizations want with an IaaS cloud: compute orchestration, Network-as-a-Service, user and account management, a full and open native API, resource accounting, and a first-class User Interface (UI).
CloudStack currently supports the most popular hypervisors: VMware vSphere, KVM, XenServer, XenProject and Hyper-V as well as OVM and LXC containers.
Users can manage their cloud with an easy to use Web interface, command line tools, and/or a full-featured query based API.
For more information on Apache CloudStack, please visit the website
There are more than 150 known organizations using Apache CloudStack (or a commercial distribution of CloudStack). Our users include many major service providers running CloudStack to offer public cloud services, product vendors who incorporate or integrate with CloudStack in their own products, organizations who have used CloudStack to build their own private clouds, and systems integrators that offer CloudStack related services.
See our case studies highlighting successful deployments of Apache CloudStack.
See the up-to-date list of current users.
If you are using CloudStack in your organization and your company is not listed above, please complete our brief adoption survey. We're happy to keep your company name anonymous if you require.
Apache CloudStack project uses Git. The official Git repository is at:
https://gitbox.apache.org/repos/asf/cloudstack.git
And a mirror is hosted on Github:
https://github.com/apache/cloudstack
The Github mirror is strictly read only and provides convenience to users and developers to explore the code and for the community to accept contributions via Github pull requests.
Interested in helping out with Apache CloudStack? Great! We welcome participation from anybody willing to work The Apache Way and make a contribution. Note that you do not have to be a developer in order to contribute to Apache CloudStack. We need folks to help with documentation, translation, promotion etc. See our contribution page.
If you're interested in learning more or participating in the Apache CloudStack project, the mailing lists are the best way to do that. While the project has several communications channels, the mailing lists are the most active and the official channels for making decisions about the project itself.
Mailing lists:
IRC, join us on irc.freenode.net on:
#cloudstack: General Apache CloudStack conversation and end user support#cloudstack-dev: Development discussionsReport and/or check bugs on JIRA and check our developer page for contributing code.
If you've found an issue that you believe is a security vulnerability in a released version of CloudStack, please report it to security@cloudstack.apache.org with details about the vulnerability, how it might be exploited, and any additional information that might be useful.
For more details, please visit our security page.
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Please see the LICENSE file included in the root directory of the source tree for extended license details.
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
The following provides more details on the included cryptographic software: