| .. Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information# |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| http://www.apache.org/licenses/LICENSE-2.0 |
| Unless required by applicable law or agreed to in writing, |
| software distributed under the License is distributed on an |
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations |
| under the License. |
| |
| |
| Working with System Virtual Machines |
| ==================================== |
| |
| CloudStack uses several types of system virtual machines to perform |
| tasks in the cloud. In general CloudStack manages these system VMs and |
| creates, starts, and stops them as needed based on scale and immediate |
| needs. However, the administrator should be aware of them and their |
| roles to assist in debugging issues. |
| |
| |
| The System VM Template |
| ---------------------- |
| |
| The System VMs come from a single template. The System VM has the |
| following characteristics: |
| |
| - Debian 7.8 ("wheezy"), 3.2.0 kernel with the latest security |
| patches from the Debian security APT repository |
| |
| - Has a minimal set of packages installed thereby reducing the attack |
| surface |
| |
| - 64-bit for enhanced performance on Xen/VMWare |
| |
| - pvops kernel with Xen PV drivers, KVM virtio drivers, and VMware |
| tools for optimum performance on all hypervisors |
| |
| - Xen tools inclusion allows performance monitoring |
| |
| - Latest versions of HAProxy, iptables, IPsec, and Apache from debian |
| repository ensures improved security and speed |
| |
| - Latest version of JRE from Sun/Oracle ensures improved security and |
| speed |
| |
| |
| Changing the Default System VM Template |
| --------------------------------------- |
| |
| Using the 64-bit template should be use with a System Offering of at least 512MB |
| of memory. |
| |
| #. Based on the hypervisor you use, download the 64-bit template from |
| the following location: |
| |
| .. cssclass:: table-striped table-bordered table-hover |
| |
| ========== ================================================================================================ |
| Hypervisor Download Location |
| ========== ================================================================================================ |
| XenServer |sysvm64-url-xen| |
| KVM |sysvm64-url-kvm| |
| VMware |sysvm64-url-vmware| |
| Hyper-V |sysvm64-url-hyperv| |
| ========== ================================================================================================ |
| |
| #. As an administrator, log in to the CloudStack UI |
| |
| #. Register the 64 bit template. |
| |
| For example: KVM64bitTemplate |
| |
| #. While registering the template, select Routing. |
| |
| #. Navigate to Infrastructure > Zone > Settings. |
| |
| #. Set the name of the 64-bit template, KVM64bitTemplate, in the |
| *``router.template.kvm``* global parameter. |
| |
| If you are using a XenServer 64-bit template, set the name in the |
| *``router.template.xen``* global parameter. |
| |
| Any new virtual router created in this Zone automatically picks up |
| this template. |
| |
| #. Restart the Management Server. |
| |
| |
| Multiple System VM Support for VMware |
| ------------------------------------- |
| |
| Every CloudStack zone has single System VM for template processing tasks |
| such as downloading templates, uploading templates, and uploading ISOs. |
| In a zone where VMware is being used, additional System VMs can be |
| launched to process VMware-specific tasks such as taking snapshots and |
| creating private templates. The CloudStack management server launches |
| additional System VMs for VMware-specific tasks as the load increases. |
| The management server monitors and weights all commands sent to these |
| System VMs and performs dynamic load balancing and scaling-up of more |
| System VMs. |
| |
| |
| Console Proxy |
| ------------- |
| |
| The Console Proxy is a type of System Virtual Machine that has a role in |
| presenting a console view via the web UI. It connects the user’s browser |
| to the VNC port made available via the hypervisor for the console of the |
| guest. Both the administrator and end user web UIs offer a console |
| connection. |
| |
| Clicking a console icon brings up a new window. The AJAX code downloaded |
| into that window refers to the public IP address of a console proxy VM. |
| There is exactly one public IP address allocated per console proxy VM. |
| The AJAX application connects to this IP. The console proxy then proxies |
| the connection to the VNC port for the requested VM on the Host hosting |
| the guest. |
| |
| .. note:: |
| The hypervisors will have many ports assigned to VNC usage so that |
| multiple VNC sessions can occur simultaneously. |
| |
| There is never any traffic to the guest virtual IP, and there is no need |
| to enable VNC within the guest. |
| |
| The console proxy VM will periodically report its active session count |
| to the Management Server. The default reporting interval is five |
| seconds. This can be changed through standard Management Server |
| configuration with the parameter consoleproxy.loadscan.interval. |
| |
| Assignment of guest VM to console proxy is determined by first |
| determining if the guest VM has a previous session associated with a |
| console proxy. If it does, the Management Server will assign the guest |
| VM to the target Console Proxy VM regardless of the load on the proxy |
| VM. Failing that, the first available running Console Proxy VM that has |
| the capacity to handle new sessions is used. |
| |
| Console proxies can be restarted by administrators but this will |
| interrupt existing console sessions for users. |
| |
| |
| Using a SSL Certificate for the Console Proxy |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| By default, the console viewing functionality uses plaintext HTTP. In |
| any production environment, the console proxy connection should be |
| encrypted via SSL at the mininum. |
| |
| A CloudStack administrator has 2 ways to secure the console proxy |
| communication with SSL: |
| |
| - Set up a SSL wild-card certificate and domain name resolution |
| |
| - Set up SSL certificate for specific FQDN and configure load-balancer |
| |
| |
| Changing the Console Proxy SSL Certificate and Domain |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The administrator can configure SSL encryption by selecting a domain |
| and uploading a new SSL certificate and private key. The domain must |
| run a DNS service that is capable of resolving queries for addresses |
| of the form aaa-bbb-ccc-ddd.your.domain to an IPv4 IP address in the |
| form aaa.bbb.ccc.ddd, for example, 202.8.44.1. To change the console |
| proxy domain, SSL certificate, and private key: |
| |
| #. Set up dynamic name resolution or populate all possible DNS names in |
| your public IP range into your existing DNS server with the format |
| aaa-bbb-ccc-ddd.consoleproxy.company.com -> aaa.bbb.ccc.ddd. |
| |
| .. note:: |
| In these steps you will notice *consoleproxy.company.com* -For |
| security best practices, we recommend creating a wildcard SSL |
| certificate on a separate subdomain so in the event that the |
| certificate is compromised, a malicious user cannot impersonate |
| a company.com domain. |
| |
| #. Generate the private key and certificate signing request (CSR). When |
| you are using openssl to generate private/public key pairs and CSRs, |
| for the private key that you are going to paste into the CloudStack |
| UI, be sure to convert it into PKCS#8 format. |
| |
| #. Generate a new 2048-bit private key |
| |
| .. code:: bash |
| |
| openssl genrsa -des3 -out yourprivate.key 2048 |
| |
| #. Generate a new certificate CSR. Ensure the creation of a wildcard |
| certificate, eg ``*.consoleproxy.company.com`` |
| |
| .. code:: bash |
| |
| openssl req -new -key yourprivate.key -out yourcertificate.csr |
| |
| #. Head to the website of your favorite trusted Certificate |
| Authority, purchase an SSL certificate, and submit the CSR. You |
| should receive a valid certificate in return |
| |
| #. Convert your private key format into PKCS#8 encrypted format. |
| |
| .. code:: bash |
| |
| openssl pkcs8 -topk8 -in yourprivate.key -out yourprivate.pkcs8.encrypted.key |
| |
| #. Convert your PKCS#8 encrypted private key into the PKCS#8 format |
| that is compliant with CloudStack |
| |
| .. code:: bash |
| |
| openssl pkcs8 -in yourprivate.pkcs8.encrypted.key -out yourprivate.pkcs8.key |
| |
| #. In the Update SSL Certificate screen of the CloudStack UI, paste the |
| following: |
| |
| - The certificate you've just generated. |
| |
| - The private key you've just generated. |
| |
| - The desired domain name, prefixed with ``*.``; for example, ``*.consoleproxy.company.com`` |
| |
| |update-ssl.png| |
| |
| #. This stops all currently running console proxy VMs, then restarts |
| them with the new certificate and key. Users might notice a brief |
| interruption in console availability. |
| |
| The Management Server generates URLs of the form |
| "aaa-bbb-ccc-ddd.consoleproxy.company.com" after this change is made. |
| The new console requests will be served with the new DNS domain name, |
| certificate, and key. |
| |
| Uploading ROOT CA and Intermediate CA |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| If you need to upload custom certificate with ROOT CA and intermediate CA, you can find more details here: |
| https://cwiki.apache.org/confluence/display/CLOUDSTACK/Procedure+to+Replace+realhostip.com+with+Your+Own+Domain+Name |
| |
| IMPORTANT NOTES: |
| |
| In order to avoid errors and problems while uploading custom certificates, please check following: |
| |
| 1. While doing URL encoding of ROOT CA and any Intermediate CA, be sure that the plus signs ("+") inside certificates |
| are not replaced by space (" "), because some URL/string encoding tools tend to do that. |
| |
| 2. If you are renewing certificates it might happen you need to upload new ROOT CA and Intermediate CA, together with new Server Certificate and key. |
| In this case please be sure to use same names for certificates during API upload of certificate, example: |
| |
| http://123.123.123.123:8080/client/api?command=uploadCustomCertificate&...&name=root1... |
| http://123.123.123.123:8080/client/api?command=uploadCustomCertificate&...&name=intermed1... |
| |
| Here names are "root1" and "intermed1". |
| If you used other names previously, please check the cloud.keystore table to obtain used names. |
| |
| If you still have problems and folowing errors in management.log while destroying CPVM: |
| |
| - Unable to build keystore for CPVMCertificate due to CertificateException |
| - Cold not find and construct a valid SSL certificate |
| |
| that means that still some of the Root/intermediate/server certificates or the key is not in a good format, or incorrectly encoded or multiply Root CA/Intemediate CA present in database by mistake. |
| |
| Other way to renew Certificates (Root,Intermediates,Server certificates and key) - although not recommended |
| unless you fill comfortable - is to directly edit the database, |
| while still respect the main requirement that the private key is PKCS8 encoded, while Root CA, Intemediate and Server certificates |
| are still in default PEM format (no URL encoding needed here). |
| After editing the database, please restart management server, and destroy SSVM and CPVM after that, |
| so the new SSVM and CPVM with new certificates are created. |
| |
| Load-balancing Console Proxies |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| An alternative to using dynamic DNS or creating a range of DNS entries |
| as described in the last section would be to create a SSL certificate |
| for a specific domain name, configure CloudStack to use that particular |
| FQDN, and then configure a load balancer to load balance the console |
| proxy's IP address behind the FQDN. As the functionality for this is |
| still new, please see |
| https://cwiki.apache.org/confluence/display/CLOUDSTACK/Realhost+IP+changes |
| for more details. |
| |
| |
| Virtual Router |
| -------------- |
| |
| The virtual router is a type of System Virtual Machine. The virtual |
| router is one of the most frequently used service providers in |
| CloudStack. The end user has no direct access to the virtual router. |
| Users can ping the virtual router and take actions that affect it (such |
| as setting up port forwarding), but users do not have SSH access into |
| the virtual router. |
| |
| There is no mechanism for the administrator to log in to the virtual |
| router. Virtual routers can be restarted by administrators, but this |
| will interrupt public network access and other services for end users. A |
| basic test in debugging networking issues is to attempt to ping the |
| virtual router from a guest VM. Some of the characteristics of the |
| virtual router are determined by its associated system service offering. |
| |
| |
| Configuring the Virtual Router |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| You can set the following: |
| |
| - IP range |
| |
| - Supported network services |
| |
| - Default domain name for the network serviced by the virtual router |
| |
| - Gateway IP address |
| |
| - How often CloudStack fetches network usage statistics from CloudStack |
| virtual routers. If you want to collect traffic metering data from |
| the virtual router, set the global configuration parameter |
| router.stats.interval. If you are not using the virtual router to |
| gather network usage statistics, set it to 0. |
| |
| |
| Upgrading a Virtual Router with System Service Offerings |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| When CloudStack creates a virtual router, it uses default settings which |
| are defined in a default system service offering. See `“System Service |
| Offerings” <#system-service-offerings>`_. All the |
| virtual routers in a single guest network use the same system service |
| offering. You can upgrade the capabilities of the virtual router by |
| creating and applying a custom system service offering. |
| |
| #. Define your custom system service offering. |
| See `“Creating a New System Service Offering” |
| <#creating-a-new-system-service-offering>`_. |
| In System VM Type, choose Domain Router. |
| |
| #. Associate the system service offering with a network offering. See |
| `“Creating a New Network Offering” |
| <networking.html#creating-a-new-network-offering>`_. |
| |
| #. Apply the network offering to the network where you want the virtual |
| routers to use the new system service offering. If this is a new |
| network, follow the steps in Adding an Additional Guest Network on |
| page 66. To change the service offering for existing virtual routers, |
| follow the steps in `“Changing the Network Offering on a Guest Network” |
| <networking2.html#changing-the-network-offering-on-a-guest-network>`_. |
| |
| |
| Best Practices for Virtual Routers |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| - WARNING: Restarting a virtual router from a hypervisor console |
| deletes all the iptables rules. To work around this issue, stop the |
| virtual router and start it from the CloudStack UI. |
| |
| - .. warning:: |
| Do not use the destroyRouter API when only one router is available |
| in the network, because restartNetwork API with the cleanup=false |
| parameter can't recreate it later. If you want to destroy and |
| recreate the single router available in the network, use the |
| restartNetwork API with the cleanup=true parameter. |
| |
| |
| Service Monitoring Tool for Virtual Router |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Various services running on the CloudStack virtual routers can be |
| monitored by using a Service Monitoring tool. The tool ensures that |
| services are successfully running until CloudStack deliberately disables |
| them. If a service goes down, the tool automatically restarts the |
| service, and if that does not help bringing up the service, an alert as |
| well as an event is generated indicating the failure. A new global |
| parameter, ``network.router.enableservicemonitoring``, has been |
| introduced to control this feature. The default value is false, implies, |
| monitoring is disabled. When you enable, ensure that the Management |
| Server and the router are restarted. |
| |
| Monitoring tool can help to start a VR service, which is crashed due to |
| an unexpected reason. For example: |
| |
| - The services crashed due to defects in the source code. |
| |
| - The services that are terminated by the OS when memory or CPU is not |
| sufficiently available for the service. |
| |
| .. note:: |
| Only those services with daemons are monitored. The services that are |
| failed due to errors in the service/daemon configuration file cannot |
| be restarted by the Monitoring tool. VPC networks are not supported. |
| |
| The following services are monitored in a VR: |
| |
| - DNS |
| |
| - HA Proxy |
| |
| - SSH |
| |
| - Apache Web Server |
| |
| The following networks are supported: |
| |
| - Isolated Networks |
| |
| - Shared Networks in both Advanced and Basic zone |
| |
| .. note:: VPC networks are not supported |
| |
| This feature is supported on the following hypervisors: XenServer, |
| VMware, and KVM. |
| |
| |
| Enhanced Upgrade for Virtual Routers |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Upgrading VR is made flexible. The CloudStack administrators will be |
| able to control the sequence of the VR upgrades. The sequencing is based |
| on Infrastructure hierarchy, such as by Cluster, Pod, or Zone, and |
| Administrative (Account) hierarchy, such as by Tenant or Domain. As an |
| administrator, you can also determine when a particular customer |
| service, such as VR, is upgraded within a specified upgrade interval. |
| Upgrade operation is enhanced to increase the upgrade speed by allowing |
| as many upgrade operations in parallel as possible. |
| |
| During the entire duration of the upgrade, users cannot launch new |
| services or make changes to an existing service. |
| |
| Additionally, using multiple versions of VRs in a single instance is |
| supported. In the Details tab of a VR, you can view the version and |
| whether it requires upgrade. During the Management Server upgrade, |
| CloudStack checks whether VR is at the latest version before performing |
| any operation on the VR. To support this, a new global parameter, |
| *``router.version.check``*, has been added. This parameter is set to |
| true by default, which implies minimum required version is checked |
| before performing any operation. No operation is performed if the VR is |
| not at the required version. Services of the older version VR continue |
| to be available, but no further operations can be performed on the VR |
| until it is upgraded to the latest version. This will be a transient |
| state until the VR is upgraded. This will ensure that the availability |
| of VR services and VR state is not impacted due to the Management Server |
| upgrade. |
| |
| The following service will be available even if the VR is not upgraded. |
| However, no changes for any of the services can be sent to the VR, until |
| it is upgraded: |
| |
| - SecurityGroup |
| |
| - UserData |
| |
| - DHCP |
| |
| - DNS |
| |
| - LB |
| |
| - Port Forwarding |
| |
| - VPN |
| |
| - Static NAT |
| |
| - Source NAT |
| |
| - Firewall |
| |
| - Gateway |
| |
| - NetworkACL |
| |
| |
| Supported Virtual Routers |
| ^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| - VR |
| |
| - VPC VR |
| |
| - Redundant VR |
| |
| |
| Upgrading Virtual Routers |
| ^^^^^^^^^^^^^^^^^^^^^^^^^ |
| |
| #. Download the latest System VM template. |
| |
| #. Download the latest System VM to all the primary storage pools. |
| |
| #. Upgrade the Management Server. |
| |
| #. Upgrade CPVM and SSVM either from the UI or by using the following |
| script: |
| |
| .. code:: bash |
| |
| # cloudstack-sysvmadm -d <IP address> -u cloud -p -s |
| |
| Even when the VRs are still on older versions, existing services will |
| continue to be available to the VMs. The Management Server cannot |
| perform any operations on the VRs until they are upgraded. |
| |
| #. Selectively upgrade the VRs: |
| |
| #. Log in to the CloudStack UI as the root administrator. |
| |
| #. In the left navigation, choose Infrastructure. |
| |
| #. On Virtual Routers, click View More. |
| |
| All the VRs are listed in the Virtual Routers page. |
| |
| #. In Select View drop-down, select desired grouping based on your |
| requirement. |
| |
| You can use either of the following: |
| |
| - Group by zone |
| |
| - Group by pod |
| |
| - Group by cluster |
| |
| - Group by account |
| |
| #. Click the group which has the VRs to be upgraded. |
| |
| For example, if you have selected Group by zone, select the name |
| of the desired zone. |
| |
| #. Click the Upgrade button to upgrade all the VRs. |vr-upgrade.png| |
| |
| #. Click OK to confirm. |
| |
| |
| Secondary Storage VM |
| -------------------- |
| |
| In addition to the hosts, CloudStack’s Secondary Storage VM mounts and |
| writes to secondary storage. |
| |
| Submissions to secondary storage go through the Secondary Storage VM. |
| The Secondary Storage VM can retrieve templates and ISO images from URLs |
| using a variety of protocols. |
| |
| The secondary storage VM provides a background task that takes care of a |
| variety of secondary storage activities: downloading a new template to a |
| Zone, copying templates between Zones, and snapshot backups. |
| |
| The administrator can log in to the secondary storage VM if needed. |
| |
| |
| .. |update-ssl.png| image:: _static/images/update-ssl.png |
| :alt: Updating Console Proxy SSL Certificate |
| .. |vr-upgrade.png| image:: _static/images/vr-upgrade.png |
| :alt: Button to upgrade VR to use the new template. |
| |
| .. include:: /_global.rst |