| .. Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information# |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| http://www.apache.org/licenses/LICENSE-2.0 |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| |
| The Cloudian Connector Plugin |
| ============================= |
| |
| Introduction to the Cloudian Connector Plugin |
| --------------------------------------------- |
| |
| The Cloudian (HyperStore) Connector is a native CloudStack plugin that allow |
| integration between Apache CloudStack and Cloudian Management Console (CMC). The |
| Connector integrates Cloudian S3 Storage into the CloudStack Management GUI and |
| allows administrators to easily give their CloudStack users access to and manage |
| their own S3 storage areas. |
| |
| Compatibilty |
| ~~~~~~~~~~~~ |
| |
| The following table shows the compatiblity of Cloudian Connector with CloudStack. |
| |
| .. cssclass:: table-striped table-bordered table-hover |
| |
| +---------------------+----------------------+-------------------------+ |
| | Connector Version | CloudStack version | Cloudian Compatibility | |
| +=====================+======================+=========================+ |
| | 4.9_6.2-1 | 4.9 | Version 6.2 and onwards | |
| +---------------------+----------------------+-------------------------+ |
| | 4.11+ | 4.11+ | Version 6.2 and onwards | |
| +---------------------+----------------------+-------------------------+ |
| |
| Table: Support Matrix |
| |
| .. note:: |
| Starting CloudStack 4.11, the Connector will be part of the CloudStack |
| release and will not need to be externally installed. |
| |
| Connector Overview |
| ------------------ |
| |
| Single-Sign-On Integration |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The connector plugin adds a Cloudian Storage button to the CloudStack UI. This |
| button is available for all users on the bottom left of the menu. |
| |
| .. figure:: /_static/images/cloudian-tab.png |
| :align: center |
| :alt: a screenshot of an enabled Cloudian Connector |
| |
| When a user clicks this button, a new window or tab (depending on the web |
| browser preferences) is opened for the HyperStore CMC GUI. The CloudStack user |
| is automatically logged in to CMC as the correctly mapped HyperStore user using |
| Single-Sign-On (SSO). |
| |
| With the connector enabled, when the user clicks ‘Log Out’ in the CloudStack UI |
| this first logs the user out of CloudStack and then redirects the page to log |
| out any logged-in Cloudian user out of CMC GUI and finally redirects the page to |
| the CloudStack login page. |
| |
| Single-Sign-On is a technique where CloudStack and HyperStore are configured to |
| trust each other. This is achieved by configuring both HyperStore and the |
| CloudStack connector with the same SSO Shared Key. The CloudStack connector |
| creates a special login URL for CMC which it signs using this shared key. Upon |
| receiving the special signed login URL, CMC validates the request by comparing |
| the signature to its own copy of the shared key and the user is automatically |
| logged in. |
| |
| User Mapping and Provisioning/De-provisioning |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| CloudStack domains are mapped to Cloudian Groups. CloudStack accounts within |
| those domains are mapped to Cloudian users. The Cloudian user and group are |
| created on demand if it doesn’t already exist when the CloudStack user accesses |
| CMC through the Cloudian Storage button. When accounts and domains are created |
| or removed in CloudStack, they automatically create or remove users or groups in |
| CMC. |
| |
| .. cssclass:: table-striped table-bordered table-hover |
| |
| +---------------------+----------------------------+ |
| | CloudStack Entity | Equivalent Cloudian Entity | |
| +=====================+============================+ |
| | Account | User | |
| +---------------------+----------------------------+ |
| | Domain | Group | |
| +---------------------+----------------------------+ |
| |
| Table: Mapping Between Cloudian and CloudStack |
| |
| .. note:: |
| Adding groups or users directly through Cloudian does not add |
| corresponding CloudStack Domains or Accounts. The integration is driven |
| completely from the CloudStack side. |
| |
| Special Admin User Mapping |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The special CloudStack admin is are mapped to a special HyperStore Admin user |
| account which defaults to the user id admin. As the admin user on HyperStore is |
| configurable, there is a configuration option to control this mapping. This |
| mapping dictates which HyperStore user is automatically logged in using SSO when |
| the CloudStack admin user clicks "Cloudian Storage". |
| |
| .. note:: |
| The Cloudian Admin user default is called admin. Older versions of |
| Cloudian used to use admin@cloudian.com. |
| |
| DNS Resolution Requirements |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The CloudStack Management Server will need to be access the Cloudian admin |
| service. The Cloudian admin service is commonly run on the same nodes as your |
| Cloudian S3 servers. The admin service is used to provision and deprovision |
| Cloudian users and groups automatically by CloudStack. |
| |
| Additionally, your CloudStack users will need to be able to resolve your CMC |
| server hostname on their desktops so that they can access CMC. |
| |
| Example domain names that should resolve: |
| |
| .. cssclass:: table-striped table-bordered table-hover |
| |
| +---------------------+----------------------+------------------------------+ |
| | Resolvable Name | Required By | Description | |
| +=====================+======================+==============================+ |
| | mgmt.abc-cloud.com | User's browser | CloudStack Management Server | |
| +---------------------+----------------------+------------------------------+ |
| | cmc.abc-cloud.com | User's browser | Cloudian CMC | |
| +---------------------+----------------------+------------------------------+ |
| | admin.abc-cloud.com | Management Server | Cloudian Admin Server | |
| +---------------------+----------------------+------------------------------+ |
| |
| Table: DNS Name Resolution Example |
| |
| |
| Configuring the Cloudian Connector |
| ---------------------------------- |
| |
| Prerequisites |
| ~~~~~~~~~~~~~ |
| |
| Cloudian ships with SSO disabled by default. You will need to enable it on each |
| CMC server. Additionally, you will need to choose a unique SSO shared key that |
| you will also configure in the CloudStack connector further below. |
| |
| Edit Puppet config to enable SSO on all CMC servers: |
| |
| :: |
| |
| # vi /etc/cloudian-[version]-puppet/modules/cmc/templates/mts-ui.properties.erb |
| sso.enabled=true |
| sso.shared.key=YourSecretKeyHere |
| |
| |
| .. note:: |
| Once configured in Puppet, you should roll out out to each CMC server and |
| restart CMC services. Please refer to the HyperStore documentation for how to |
| do this. |
| |
| Connector Configuration |
| ~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The main way to configure, enable and disable the connector is using the |
| CloudStack global setting. The global settings provide an easy way to configure |
| the connector and synchronize setting across multiple management server(s). The |
| following global setting can be accessed and changed using the CloudStack UI: |
| |
| .. cssclass:: table-striped table-bordered table-hover |
| |
| +------------------------------+------------------------------------------------+ |
| | Global Setting | Description | |
| +==============================+================================================+ |
| | cloudian.connector.enabled | Setting to enable/disable the plugin | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.admin.host | The Cloudian admin server host | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.admin.port | The Cloudian admin server port, usually 19443 | |
| | | (https) or 18081 (http) | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.admin.protocol | The Cloudian admin server protocol, http/https | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.validate.ssl | Whether to validate SSL certificate of Cloudian| |
| | | admin service while making API calls | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.admin.user | Basic auth user name for Cloudian admin server | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.admin.password | Basic auth password for Cloudian admin server | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.api.request.timeout | The admin API request timeout in seconds | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.cmc.admin.user | The user id of the CMC admin that maps to | |
| | | CloudStack admin user | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.cmc.host | The Cloudian Management Console hostname | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.cmc.port | The Cloudian Management Console port | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.cmc.protocol | The Cloudian Management Console protocol | |
| +------------------------------+------------------------------------------------+ |
| | cloudian.sso.key | The shared secret as configured in Cloudian CMC| |
| +------------------------------+------------------------------------------------+ |
| |
| Table: Cloudian Connector Global Settings |
| |
| .. note:: |
| Change in only ‘cloudian.connector.enabled’ setting requires restarting of |
| all the CloudStack management server(s), rest of the setting can be changed |
| dynamically without requiring to restart the CloudStack management server(s). |
| |
| Enabling the Cloudian Connector |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The Cloudian Connector comes disabled by default, enabling the connector is the |
| last step. You should have already configured the Cloudian Connector global |
| settings. To enable the connector, ensure that the global setting |
| "cloudian.connector.enabled" is set to true. Finally, restart each of the |
| management server(s) to reload and enable the connector. |
| |
| For example, here is how you can restart the CloudStack management server |
| installed on CentOS7: |
| |
| :: |
| |
| # systemctl restart cloudstack-management |
| |
| |
| Troubleshooting |
| ~~~~~~~~~~~~~~~~ |
| |
| Most of the trouble you may run into will be configuration related. |
| |
| There are a few things which can go wrong for SSO. Here are the most common |
| problems and things to check: |
| |
| - Does the global settings cloudian.cmc.admin.user point to the correct Cloudian |
| (admin) user? |
| |
| - Is SSO configured and enabled on Cloudian HyperStore CMC? |
| |
| - Check for errors in the CMC log file. |
| |
| - Are both CloudStack and HyperStore CMC configured with the same cloudian.sso.key? |
| |
| - Check the /var/log/cloudstack/management/management-server.log file and |
| search for errors relating to SSO. |
| |
| - Try access the CMC host directly from the problem users host using the |
| configured cloudian.cmc.host, cloudian.cmc.port and cloudian.cmc.protocol |
| configured in the CloudStack global settings. |
| |
| - If you log out of the management server and log in again, does the Cloudian |
| Storage button work? |
| |
| |
| Adding/Deleting Domains or Accounts fails: These operations use the Cloudian |
| Admin Server. It's likely that something has changed with the connection or the |
| admin server is down. Check list: |
| |
| - Is the admin server alive and listening? |
| |
| - Try access the admin server host directly from the problem users host using |
| the configured cloudian.admin.host, cloudian.admin.port and |
| cloudian.admin.protocol configured in the CloudStack global settings. Check the |
| configured auth settings cloudian.admin.user and cloudian.admin.password. |
| |
| - If you’re experiencing timeout issues, trying changing the API timeout value |
| defined in cloudian.api.request.timeout global setting. |
| |
| - Look for errors in the admin log file /var/log/cloudian/cloudian-admin.log. |
| |
| |
| ------------ |
| |
| |
| Cloudian as CloudStack Secondary Storage |
| ---------------------------------------- |
| |
| This section is a supplementary guide for CloudStack and describes how to |
| configure CloudStack to use Cloudian HyperStore as Secondary Storage. Please |
| also review CloudStack’s documentation (Getting Started Guide) for configuring |
| and using S3 as Secondary Storage. |
| |
| CloudStack, as of version 4.2.1, can utilize Cloudian HyperStore as S3 Secondary |
| Storage out of the box. There is no need for any modifications or to install any |
| connectors. Secondary Storage is used to store ISOs, Templates, Snapshots and |
| Volumes. |
| |
| .. note:: |
| CloudStack still requires an NFS Secondary Storage Staging Server with is |
| mentioned in the requirements below. |
| |
| Requirements: |
| |
| - CloudStack 4.5+ (installed/configured and running) |
| - Cloudian HyperStore 5.0 or greater (installed/configured and running) |
| |
| NFS Secondary Storage Staging Server Requirement |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The use of S3 as Secondary Storage for CloudStack also requires an NFS server. |
| The NFS server is required because the various hypervisors cannot yet talk |
| directly to S3 but instead talk through the standard file system API. As such, |
| CloudStack requires an NFS staging server which the Hypervisors use to read and |
| write data from/to. The NFS storage requirements for the staging server are |
| small however as space is only required while objects are staged (moving) |
| between the S3 server and the VMs. |
| |
| |
| DNS Name Resolution Requirement |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| All CloudStack Management Servers, system VMs and customer VMs (if required) |
| must be able to resolve your S3 bucket names. Usually, if you already have |
| Cloudian installed and running in your environment, this is already working. |
| At a minimum the following names should resolve to the correct IP addresses |
| using the DNS server that your Management Server and System VMs are using. |
| |
| .. cssclass:: table-striped table-bordered table-hover |
| |
| +---------------------+------------------------------+ |
| | Example Name | DNS Name Types | |
| +=====================+==============================+ |
| | s3.mycloud.com | Cloudian S3 Endpoint | |
| +---------------------+------------------------------+ |
| | sec.s3.mycloud.com | Bucket for Secondary Storage | |
| +---------------------+------------------------------+ |
| | s3-admin.mycloud.com| Cloudian Admin Server | |
| +---------------------+------------------------------+ |
| |
| Table: Example Domain Names that should Resolve on CloudStack Servers |
| |
| Adding Cloudian as CloudStack Secondary Storage |
| ----------------------------------------------- |
| |
| Setup a Cloudian User and Bucket for Secondary Storage |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| S3 Secondary Storage stores the CloudStack templates, snapshots etc in a |
| dedicated S3 Bucket. To properly configure CloudStack you will need to know the |
| S3 Bucket name and how to access your S3 Server (the S3 endpoint, access key and |
| secret key). |
| |
| Below, we will create a dedicated Cloudian user and a dedicated bucket which we |
| will assign for use as Secondary Storage. |
| |
| Create a dedicated user/group: |
| |
| - Login to the Cloudian Management Console (CMC) as the Cloudian admin user. |
| |
| - Create a new group called cloudstack. Any group name is ok. |
| |
| - Create a new user called cloudstack in the cloudstack group. Any user name is ok. |
| |
| |
| Create a dedicated bucket: |
| |
| - Login to CMC as the cloudstack user created above. |
| |
| - Create a bucket called secondary. Any bucket name will do. |
| |
| - On the top menu bar on the right hand side, use the drop down menu under your |
| user name to select Security Credentials and copy and paste your Access and |
| Secret Keys to a note for later use. CloudStack will need these when you attach |
| Cloudian as Secondary Storage in a later step below. |
| |
| Open Up Access to your S3 Network from Secondary Storage |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| If your S3 server is on a different network to your Secondary Storage VM, you |
| will need to open up access to the S3 network. This also allows users to |
| download templates from their S3 object store areas. |
| |
| .. figure:: /_static/images/cloudian-ss_globalopt.png |
| :align: center |
| :alt: a screenshot of changing global settings |
| |
| .. note:: |
| Editing the Global Settings requires you to restart the management server(s). |
| |
| Add an NFS Secondary Storage Staging Server |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| As mentioned previously, S3 Secondary Storage currently requires the use of an |
| NFS Secondary Staging Server. Add NFS Secondary Storage Staging Server: |
| |
| - Login to CloudStack Management Server as the admin user. |
| |
| - Navigate to Infrastructure → Secondary Storage. |
| |
| - Click Select View and select Secondary Staging Store. |
| |
| - Click Add Secondary Staging Store. |
| |
| - Configure the zone, server and path for your desired secondary staging store. |
| For example nfs.mycloud.com and /export/staging. |
| |
| .. figure:: /_static/images/cloudian-s3_ss_cache.png |
| :align: center |
| :alt: a screenshot of adding secondary staging store |
| |
| Attach Cloudian as Secondary Storage |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| CloudStack supports using either S3 or NFS as Secondary Storage but not both. |
| The below instructions assume you are not using Secondary Storage on NFS and |
| that you can delete it to add the S3 storage. |
| |
| .. note:: |
| Already using NFS for Secondary Storage with CloudStack? You need to migrate |
| your Secondary Storage. Refer to CloudStack’s instructions for migrating |
| existing NFS Secondary Storage to an S3 object storage. CloudStack 4.5 onwards |
| supports migrating data via special commands which are described in the Getting |
| Started Guide in a section titled Upgrading from NFS to Object Storage. |
| |
| Adding S3 Secondary Storage: |
| |
| - Login to CloudStack Management Server as the admin user. |
| |
| - Navigate to Infrastructure → Secondary Storage. |
| |
| - If it exists, select and delete any existing NFS Secondary Storage server |
| setting. NOTE: Do not do this if you want to migrate existing NFS secondary |
| storage to S3. Instead, see warning above. |
| |
| - Click the Add Secondary Storage button. This will open up a pop-up form which |
| you can fill out similarly to below. |
| |
| .. figure:: /_static/images/cloudian-s3_ss_config.png |
| :align: center |
| :alt: a screenshot of configuring S3 secondary storage |
| |
| .. note:: |
| CloudStack doesn’t currently allow you to re-edit the S3 configuration so take |
| time to double check what you enter. If you make a mistake the only options |
| currently are either a) delete and recreate the storage or b) directly edit the |
| entry in the database. |
| |
| When you have finished adding Cloudian as Secondary Storage in the previous |
| steps, CloudStack will populate the new secondary storage with the system and |
| default templates. This can take some time do download as the templates are |
| quite big. |
| |
| .. note:: |
| You can check if the system template and the default template have properly |
| downloaded to the new secondary storage by navigating to Templates, selecting a |
| template, clicking on the Zones tab and checking its Status is Ready 100% |
| Downloaded. |
| |
| .. note:: |
| Should you continue to have problems, sometimes it is necessary to restart |
| the Secondary Storage VM. You can do this by navigating to Infrastructure, |
| System VMs, selecting and rebooting the Secondary Storage VM. |
| |
| CloudStack should now ready to use Cloudian HyperStore for S3 Secondary Storage. |
| |
| Revision History |
| ---------------- |
| |
| - Fri Oct 6 2017 Rohit Yadav rohit.yadav@shapeblue.com Documentation |
| created for 4.11.0 version of the Cloudian Connector Plugin |