en-US/storage-setup.xml

<?xml version='1.0' encoding='utf-8' ?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
<!ENTITY % BOOK_ENTITIES SYSTEM "cloudstack.ent">
%BOOK_ENTITIES;
]>
<!-- Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at
  http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing,
  software distributed under the License is distributed on an
  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  KIND, either express or implied.  See the License for the
  specific language governing permissions and limitations
  under the License.
-->
<chapter id="storage-setup">
  <title>Storage Setup</title>
  <para>&PRODUCT; is designed to work with a wide variety of commodity and enterprise-grade storage.  Local disk may be used as well, if supported by the selected hypervisor.  Storage type support for guest virtual disks differs based on hypervisor selection.</para>
  <informaltable>
      <tgroup cols="4" align="left" colsep="1" rowsep="1">
      <thead>
          <row>
          <entry><para></para></entry>
          <entry><para>XenServer</para></entry>
          <entry><para>vSphere</para></entry>
          <entry><para>KVM</para></entry>
          </row>
      </thead>
      <tbody>
          <row>
          <entry><para>NFS</para></entry>
          <entry><para>Supported</para></entry>
          <entry><para>Supported</para></entry>
          <entry><para>Supported</para></entry>
          </row>
          <row>
          <entry><para>iSCSI</para></entry>
          <entry><para>Supported</para></entry>
          <entry><para>Supported via VMFS</para></entry>
          <entry><para>Supported via Clustered Filesystems</para></entry>
          </row>
          <row>
          <entry><para>Fiber Channel</para></entry>
          <entry><para>Supported via Pre-existing SR</para></entry>
          <entry><para>Supported</para></entry>
          <entry><para>Supported via Clustered Filesystems</para></entry>
          </row>
          <row>
          <entry><para>Local Disk</para></entry>
          <entry><para>Supported</para></entry>
          <entry><para>Supported</para></entry>
          <entry><para>Supported</para></entry>
          </row>
      </tbody>
      </tgroup>
  </informaltable>
  <para>The use of the Cluster Logical Volume Manager (CLVM) for KVM is not officially supported with &PRODUCT;.</para>
  <section id="storage-set-small-scale">
      <title>Small-Scale Setup</title>
      <para>In a small-scale setup, a single NFS server can function as both primary and secondary storage. The NFS server just needs to export two separate shares, one for primary storage and the other for secondary storage.</para>
  </section>
  <section id="storage-set-secondary">
      <title>Secondary Storage</title>
      <para>&PRODUCT; is designed to work with any scalable secondary storage system. The only requirement is the secondary storage system supports the NFS protocol.</para>
      <note>
      <para>The storage server should be a machine with a large number of disks. The disks should ideally be managed by a hardware RAID controller. Modern hardware RAID controllers support hot plug functionality independent of the operating system so you can replace faulty disks without impacting the running operating system.</para>
      </note>
  </section>
  <section id="storage-set-example-config">
      <title>Example Configurations</title>
      <para>In this section we go through a few examples of how to set up storage to work properly on a few types of NFS and iSCSI storage systems.</para>
      <section id="storage-set-example-config-local-das">
      <title>Linux NFS on Local Disks and DAS</title>
      <para>This section describes how to configure an NFS export on a standard Linux installation. The exact commands might vary depending on the operating system version.</para>
      <orderedlist>
          <listitem><para>Install the RHEL/CentOS distribution on the storage server.</para></listitem>
          <listitem><para>If the root volume is more than 2 TB in size, create a smaller boot volume to install RHEL/CentOS. A root volume of 20 GB should be sufficient.</para></listitem>
          <listitem><para>After the system is installed, create a directory called /export. This can each be a directory in the root partition itself or a mount point for a large disk volume.</para></listitem>
          <listitem><para>If you have more than 16TB of storage on one host, create multiple EXT3 file systems and multiple NFS exports. Individual EXT3 file systems cannot exceed 16TB.</para></listitem>
          <listitem>
          <para>After /export directory is created, run the following command to configure it as an NFS export.</para>
          <programlisting># echo "/export &lt;CIDR&gt;(rw,async,no_root_squash,no_subtree_check)" >> /etc/exports</programlisting>
          <para>Adjust the above command to suit your deployment needs.</para>
          <itemizedlist>
              <listitem>
              <para><emphasis role="bold">Limiting NFS export.</emphasis> It is highly recommended that you limit the NFS export to a particular subnet by specifying a subnet mask (e.g.,”192.168.1.0/24”). By allowing access from only within the expected cluster, you avoid having non-pool member mount the storage. The limit you place must include the management network(s) and the storage network(s). If the two are the same network then one CIDR is sufficient. If you have a separate storage network you must provide separate CIDR’s for both or one CIDR that is broad enough to span both.</para>
              <para>The following is an example with separate CIDRs:</para>
              <programlisting>/export 192.168.1.0/24(rw,async,no_root_squash,no_subtree_check) 10.50.1.0/24(rw,async,no_root_squash,no_subtree_check)</programlisting>
              </listitem>
              <listitem>
              <para><emphasis role="bold">Removing the async flag.</emphasis> The async flag improves performance by allowing the NFS server to respond before writes are committed to the disk. Remove the async flag in your mission critical production deployment.</para>
              </listitem>
          </itemizedlist>
          </listitem>
          <listitem>
          <para>Run the following command to enable NFS service.</para>
          <programlisting># chkconfig nfs on</programlisting>
          </listitem>
          <listitem>
          <para>Edit the /etc/sysconfig/nfs file and uncomment the following lines.</para>
          <programlisting>LOCKD_TCPPORT=32803
LOCKD_UDPPORT=32769
MOUNTD_PORT=892
RQUOTAD_PORT=875
STATD_PORT=662
STATD_OUTGOING_PORT=2020</programlisting>
          </listitem>
          <listitem>
          <para>Edit the /etc/sysconfig/iptables file and add the following lines at the beginning of the INPUT chain.</para>
          <programlisting>
-A INPUT -m state --state NEW -p udp --dport 111 -j ACCEPT
-A INPUT -m state --state NEW -p tcp --dport 111 -j ACCEPT
-A INPUT -m state --state NEW -p tcp --dport 2049 -j ACCEPT
-A INPUT -m state --state NEW -p tcp --dport 32803 -j ACCEPT
-A INPUT -m state --state NEW -p udp --dport 32769 -j ACCEPT
-A INPUT -m state --state NEW -p tcp --dport 892 -j ACCEPT
-A INPUT -m state --state NEW -p udp --dport 892 -j ACCEPT
-A INPUT -m state --state NEW -p tcp --dport 875 -j ACCEPT
-A INPUT -m state --state NEW -p udp --dport 875 -j ACCEPT
-A INPUT -m state --state NEW -p tcp --dport 662 -j ACCEPT
-A INPUT -m state --state NEW -p udp --dport 662 -j ACCEPT
         </programlisting>
          </listitem>
          <listitem>
          <para>Reboot the server.</para>
          <para>An NFS share called /export is now set up.</para>
          </listitem>
      </orderedlist>
      <note><para>When copying and pasting a command, be sure the command has pasted as a single line before executing. Some document viewers may introduce unwanted line breaks in copied text.</para></note>
      </section>
      <section id="storage-set-example-config-iscsi">
      <title>Linux NFS on iSCSI</title>
      <para>Use the following steps to set up a Linux NFS server export on an iSCSI volume. These steps apply to RHEL/CentOS 5 distributions.</para>
      <orderedlist>
          <listitem>
          <para>Install iscsiadm.</para>
          <programlisting>
# yum install iscsi-initiator-utils
# service iscsi start
# chkconfig --add iscsi
# chkconfig iscsi on
          </programlisting>
          </listitem>
          <listitem>
          <para>Discover the iSCSI target.</para>
          <programlisting># iscsiadm -m discovery -t st -p &lt;iSCSI Server IP address&gt;:3260</programlisting>
          <para>For example:</para>
          <programlisting># iscsiadm -m discovery -t st -p 172.23.10.240:3260
          172.23.10.240:3260,1 iqn.2001-05.com.equallogic:0-8a0906-83bcb3401-16e0002fd0a46f3d-rhel5-test </programlisting>
          </listitem>
          <listitem>
          <para>Log in.</para>
          <programlisting># iscsiadm -m node -T &lt;Complete Target Name&gt; -l -p &lt;Group IP&gt;:3260</programlisting>
                  <para>For example:</para>
          <programlisting># iscsiadm -m node -l -T iqn.2001-05.com.equallogic:83bcb3401-16e0002fd0a46f3d-rhel5-test -p 172.23.10.240:3260 </programlisting>
          </listitem>
          <listitem>
          <para>Discover the SCSI disk. For example:</para>
          <programlisting>
# iscsiadm -m session -P3 | grep Attached
Attached scsi disk sdb State: running
          </programlisting>
          </listitem>
          <listitem>
          <para>Format the disk as ext3 and mount the volume.</para>
          <programlisting># mkfs.ext3 /dev/sdb
# mkdir -p /export
# mount /dev/sdb /export
          </programlisting>
          </listitem>
          <listitem>
          <para>Add the disk to /etc/fstab to make sure it gets mounted on boot.</para>
          <programlisting>/dev/sdb /export ext3 _netdev 0 0</programlisting>
          </listitem>
      </orderedlist>
      <para>Now you can set up /export as an NFS share.</para>
      <itemizedlist>
          <listitem>
          <para><emphasis role="bold">Limiting NFS export.</emphasis> In order to avoid data loss, it is highly recommended that you limit the NFS export to a particular subnet by specifying a subnet mask (e.g.,”192.168.1.0/24”). By allowing access from only within the expected cluster, you avoid having non-pool member mount the storage and inadvertently delete all its data. The limit you place must include the management network(s) and the storage network(s). If the two are the same network then one CIDR is sufficient. If you have a separate storage network you must provide separate CIDRs for both or one CIDR that is broad enough to span both. </para>
          <para>The following is an example with separate CIDRs:</para>
          <programlisting>/export 192.168.1.0/24(rw,async,no_root_squash,no_subtree_check) 10.50.1.0/24(rw,async,no_root_squash,no_subtree_check)</programlisting>
          </listitem>
          <listitem><para><emphasis role="bold">Removing the async flag.</emphasis> The async flag improves performance by allowing the NFS server to respond before writes are committed to the disk. Remove the async flag in your mission critical production deployment.</para></listitem>
      </itemizedlist>
      </section>
  </section>
  </chapter>