| .. Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information# |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| http://www.apache.org/licenses/LICENSE-2.0 |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| |
| Deploying CloudStack with Ansible |
| ================================= |
| |
| In this article, `Paul Angus <https://twitter.com/CloudyAngus>`__ Cloud |
| Architect at ShapeBlue takes a look at using Ansible to Deploy an |
| Apache CloudStack cloud. |
| |
| |
| What is Ansible |
| --------------- |
| |
| Ansible is a deployment and configuration management tool similar in |
| intent to Chef and Puppet. It allows (usually) DevOps teams to |
| orchestrate the deployment and configuration of their environments |
| without having to re-write custom scripts to make changes. |
| |
| Like Chef and Puppet, Ansible is designed to be idempotent. This means |
| that you determine the state you want a host to be in and Ansible will |
| decide if it needs to act in order to achieve that state. |
| |
| |
| There’s already Chef and Puppet, so what’s the fuss about Ansible? |
| ------------------------------------------------------------------ |
| |
| Let’s take it as a given that configuration management makes life much |
| easier (and is quite cool), Ansible only needs an SSH connection to the |
| hosts that you’re going to manage to get started. While Ansible requires |
| Python 2.4 or greater on the host you’re going to manage in order to |
| leverage the vast majority of its functionality, it is able to connect |
| to hosts which don’t have Python installed in order to then install |
| Python, so it’s not really a problem. This greatly simplifies the |
| deployment procedure for hosts, avoiding the need to pre-install agents |
| onto the clients before the configuration management can take over. |
| |
| Ansible will allow you to connect as any user to a managed host (with |
| that user’s privileges) or by using public/private keys – allowing fully |
| automated management. |
| |
| There also doesn’t need to be a central server to run everything, as |
| long as your playbooks and inventories are in-sync you can create as |
| many Ansible servers as you need (generally a bit of Git pushing and |
| pulling will do the trick). |
| |
| Finally – its structure and language is pretty simple and clean. I’ve |
| found it a bit tricky to get the syntax correct for variables in some |
| circumstances, but otherwise I’ve found it one of the easier tools to |
| get my head around. |
| |
| |
| So let’s see something |
| ---------------------- |
| |
| For this example we’re going to create an Ansible server which will then |
| deploy a CloudStack server. Both of these servers will be CentOS 6.4 |
| virtual machines. |
| |
| |
| Installing Ansible |
| ------------------ |
| |
| Installing Ansible is blessedly easy. We generally prefer to use CentOS |
| so to install Ansible you run the following commands on the Ansible |
| server. |
| |
| :: |
| |
| # rpm -ivh http://www.mirrorservice.org/sites/dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm |
| # yum install -y ansible |
| |
| And that’s it. |
| |
| *(There is a commercial version which has more features such as callback |
| to request configurations and a RESTful API and also support. The |
| installation of this is different)* |
| |
| By default Ansible uses /etc/ansible to store your playbooks, I tend to |
| move it, but there’s no real problem with using the default location. |
| Create yourself a little directory structure to get started with. The |
| documentation recommends something like this: |
| |
| |
| Playbooks |
| --------- |
| |
| Ansible uses playbooks to specify the state in which you wish the target |
| host to be in to be able to accomplish its role. Ansible playbooks are |
| written in YAML format. |
| |
| |
| Modules |
| ------- |
| |
| To get Ansible to do things you specify the hosts a playbook will act |
| upon and then call modules and supply arguments which determine what |
| Ansible will do to those hosts. |
| |
| To keep things simple, this example is a cut-down version of a full |
| deployment. This example creates a single management server with a local |
| MySQL server and assumes you have your secondary storage already |
| provisioned somewhere. For this example I’m also not going to include |
| securing the MySQL server, configuring NTP or using Ansible to configure |
| the networking on the hosts either. Although normally we’d use Ansible |
| to do exactly that. |
| |
| The pre-requisites to this CloudStack build are: |
| |
| - A CentOS 6.4 host to install CloudStack on |
| |
| - An IP address already assigned on the ACS management host |
| |
| - The ACS management host should have a resolvable FQDN (either through |
| DNS or the host file on the ACS management host) |
| |
| - Internet connectivity on the ACS management host |
| |
| |
| Planning |
| -------- |
| |
| The first step I use is to list all of the tasks I think I’ll need and |
| group them or split them into logical blocks. So for this deployment of |
| CloudStack I’d start with: |
| |
| - Configure selinux |
| |
| - (libselinux-python required for Ansible to work with selinux enabled |
| hosts) |
| |
| - Install and configure MySQL |
| |
| - (Python MySQL-DB required for Ansible MySQL module) |
| |
| - Install cloud-client |
| |
| - Seed secondary storage |
| |
| Ansible is built around the idea of hosts having roles, so generally you |
| would group or manage your hosts by their roles. So now to create some |
| roles for these tasks |
| |
| I’ve created: |
| |
| - cloudstack-manager |
| |
| - mysql |
| |
| First up we need to tell Ansible where to find our CloudStack management |
| host. In the root Ansible directory there is a file called ‘hosts’ |
| (/etc/Ansible/hosts) add a section like this: |
| |
| :: |
| |
| [acs-manager] |
| xxx.xxx.xxx.xxx |
| |
| where xxx.xxx.xxx.xxx is the ip address of your ACS management host. |
| |
| |
| MySQL |
| ----- |
| |
| So let’s start with the MySQL server. We’ll need to create a task |
| within the mysql role directory called main.yml. The ‘task’ in this case |
| to have MySQL running and configured on the target host. The contents of |
| the file will look like this: |
| |
| :: |
| |
| -name: Ensure mysql server is installed |
| |
| yum: name=mysql-server state=present |
| |
| - name: Ensure mysql python is installed |
| |
| yum: name=MySQL-python state=present |
| |
| |
| - name: Ensure selinux python bindings are installed |
| |
| yum: name=libselinux-python state=present |
| |
| - name: Ensure cloudstack specfic my.cnf lines are present |
| |
| lineinfile: dest=/etc/my.cnf regexp=’$item’ insertafter=”symbolic-links=0″ line=’$item’ |
| |
| with\_items: |
| |
| – skip-name-resolve |
| |
| – default-time-zone=’+00:00′ |
| |
| – innodb\_rollback\_on\_timeout=1 |
| |
| – innodb\_lock\_wait\_timeout=600 |
| |
| – max\_connections=350 |
| |
| – log-bin=mysql-bin |
| |
| – binlog-format = ‘ROW’ |
| |
| |
| - name: Ensure MySQL service is started |
| |
| service: name=mysqld state=started |
| |
| - name: Ensure MySQL service is enabled at boot |
| |
| service: name=mysqld enabled=yes |
| |
| |
| |
| - name: Ensure root password is set |
| |
| mysql\_user: user=root password=$mysql\_root\_password host=localhost |
| |
| ignore\_errors: true |
| |
| - name: Ensure root has sufficient privileges |
| |
| mysql\_user: login\_user=root login\_password=$mysql\_root\_password user=root host=% password=$mysql\_root\_password priv=\*.\*:GRANT,ALL state=present |
| |
| This needs to be saved as `/etc/ansible/roles/mysql/tasks/main.yml` |
| |
| As explained earlier, this playbook in fact describes the state of the |
| host rather than setting out commands to be run. For instance, we |
| specify certain lines which must be in the my.cnf file and allow Ansible |
| to decide whether or not it needs to add them. |
| |
| Most of the modules are self-explanatory once you see them, but to run |
| through them briefly; |
| |
| The ‘yum’ module is used to specify which packages are required, the |
| ‘service’ module controls the running of services, while the |
| ‘mysql\_user’ module controls mysql user configuration. The ‘lineinfile’ |
| module controls the contents in a file. |
| |
| We have a couple of variables which need declaring. You could do that |
| within this playbook or its ‘parent’ playbook, or as a higher level |
| variable. I’m going to declare them in a higher level playbook. More on |
| this later. |
| |
| That’s enough to provision a MySQL server. Now for the management |
| server. |
| |
| |
| CloudStack Management server service |
| ------------------------------------ |
| |
| For the management server role we create a main.yml task like this: |
| |
| :: |
| |
| - name: Ensure selinux python bindings are installed |
| |
| yum: name=libselinux-python state=present |
| |
| |
| - name: Ensure the Apache Cloudstack Repo file exists as per template |
| |
| template: src=cloudstack.repo.j2 dest=/etc/yum.repos.d/cloudstack.repo |
| |
| |
| - name: Ensure selinux is in permissive mode |
| |
| command: setenforce permissive |
| |
| |
| - name: Ensure selinux is set permanently |
| |
| selinux: policy=targeted state=permissive |
| |
| |
| -name: Ensure CloudStack packages are installed |
| |
| yum: name=cloud-client state=present |
| |
| |
| - name: Ensure vhdutil is in correct location |
| |
| get\_url: url=http://download.cloudstack.org/tools/vhd-util dest=/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/vhd-util mode=0755 |
| |
| |
| Save this as `/etc/ansible/roles/cloudstack-management/tasks/main.yml` |
| |
| Now we have some new elements to deal with. The Ansible template module |
| uses Jinja2 based templating. As we’re doing a simplified example here, |
| the Jinja template for the cloudstack.repo won’t have any variables in |
| it, so it would simply look like this: |
| |
| :: |
| |
| [cloudstack] |
| name=cloudstack |
| baseurl=http://download.cloudstack.org/rhel/4.2/ |
| enabled=1 |
| gpgcheck=0 |
| |
| This is saved in |
| `/etc/ansible/roles/cloudstack-manager/templates/cloudstack.repo.j2` |
| |
| That gives us the packages installed, we need to set up the database. To |
| do this I’ve created a separate task called setupdb.yml |
| |
| :: |
| |
| - name: cloudstack-setup-databases |
| command: /usr/bin/cloudstack-setup-databases cloud:{{mysql\_cloud\_password }}@localhost –deploy-as=root:{{mysql\_root\_password }} |
| |
| - name: Setup CloudStack manager |
| command: /usr/bin/cloudstack-setup-management |
| |
| |
| Save this as: `/etc/ansible/roles/cloudstack-management/tasks/setupdb.yml` |
| |
| As there isn’t (as yet) a CloudStack module, Ansible doesn’t inherently |
| know whether or not the databases have already been provisioned, |
| therefore this step is not currently idempotent and will overwrite any |
| previously provisioned databases. |
| |
| There are some more variables here for us to declare later. |
| |
| |
| System VM Templates: |
| -------------------- |
| |
| Finally we would want to seed the system VM templates into the secondary |
| storage. The playbook for this would look as follows: |
| |
| :: |
| |
| - name: Ensure secondary storage mount exists |
| file: path={{ tmp\_nfs\_path }} state=directory |
| |
| |
| - name: Ensure NFS storage is mounted |
| mount: name={{ tmp\_nfs\_path }} src={{ sec\_nfs\_ip }}:{{sec\_nfs\_path }} fstype=nfs state=mounted opts=nolock |
| |
| |
| - name: Seed secondary storage |
| command: |
| /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m {{ tmp\_nfs\_path }} -u http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2 -h kvm -F |
| |
| command: |
| /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m {{ tmp\_nfs\_path }} -u http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2 -h xenserver -F |
| |
| command: |
| /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m {{ tmp\_nfs\_path }} -u http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ov -h vmware -F |
| |
| |
| Save this as `/etc/ansible/roles/cloudstack-manager/tasks/seedstorage.yml` |
| |
| Again, there isn’t a CloudStack module so Ansible will always run this |
| even if the secondary storage already has the templates in it. |
| |
| |
| Bringing it all together |
| ------------------------ |
| |
| Ansible can use playbooks which run other playbooks, this allows us to |
| group these playbooks together and declare variables across all of the |
| individual playbooks. So in the Ansible playbook directory create a file |
| called deploy-cloudstack.yml, which would look like this: |
| |
| :: |
| |
| -hosts: acs-manager |
| |
| vars: |
| |
| mysql\_root\_password: Cl0ud5tack |
| mysql\_cloud\_password: Cl0ud5tack |
| tmp\_nfs\_path: /mnt/secondary |
| sec\_nfs\_ip: IP\_OF\_YOUR\_SECONDARY\_STORAGE |
| sec\_nfs\_path: PATH\_TO\_YOUR\_SECONDARY\_STORAGE\_MOUNT |
| |
| |
| roles: |
| |
| – mysql |
| – cloudstack-manager |
| |
| tasks: |
| |
| – include: /etc/ansible/roles/cloudstack-manager/tasks/setupdb.yml |
| – include: /etc/ansible/roles/cloudstack-manager/tasks/seedstorage.yml |
| |
| |
| Save this as `/etc/ansible/deploy-cloudstack.yml` inserting the IP |
| address and path for your secondary storage and changing the passwords |
| if you wish to. |
| |
| To run this go to the Ansible directory (cd /etc/ansible ) and run: |
| |
| :: |
| |
| # ansible-playbook deploy-cloudstack.yml -k |
| |
| ‘-k’ tells Ansible to ask you for the root password to connect to the |
| remote host. |
| |
| Now log in to the CloudStack UI on the new management server. |
| |
| |
| How is this example different from a production deployment? |
| ----------------------------------------------------------- |
| |
| In a production deployment, the Ansible playbooks would configure |
| multiple management servers connected to master/slave replicating MySQL |
| databases along with any other infrastructure components required and |
| deploy and configure the hypervisor hosts. We would also have a |
| dedicated file describing the hosts in the environment and a dedicated |
| file containing variables which describe the environment. |
| |
| The advantage of using a configuration management tool such as Ansible |
| is that we can specify components like the MySQL database VIP once and |
| use it multiple times when configuring the MySQL server itself and other |
| components which need to use that information. |
| |
| |
| Acknowledgements |
| ---------------- |
| |
| Thanks to Shanker Balan for introducing me to Ansible and a load of |
| handy hints along the way. |