| <?xml version='1.0' encoding='utf-8' ?> |
| <!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "file:///C:/Program%20Files%20(x86)/Publican/DocBook_DTD/docbookx.dtd" [ |
| <!ENTITY % BOOK_ENTITIES SYSTEM "cloudstack.ent"> |
| %BOOK_ENTITIES; |
| ]> |
| <chapter id="storage-plugins"> |
| <title>Writing a Storage Plugin</title> |
| <para>This section gives an outline of how to implement a plugin |
| to integrate a third-party storage provider. |
| For details and an example, you will need to read the code.</para> |
| <note> |
| <para>Example code is available at: |
| plugins/storage/volume/sample</para> |
| </note> |
| <para/> |
| <para>Third party storage providers can integrate with &PRODUCT; to provide |
| either primary storage or secondary storage. |
| For example, &PRODUCT; provides plugins for |
| Amazon Simple Storage Service (S3) or OpenStack |
| Object Storage (Swift). The S3 plugin can be used for any object storage |
| that supports the Amazon S3 interface.</para> |
| <para>Additional third party object storages that |
| do not support the S3 interface can be integrated with &PRODUCT; |
| by writing plugin software that uses the object storage plugin framework. |
| Several new interfaces are available so that |
| storage providers can develop vendor-specific plugins based on well-defined |
| contracts that can be seamlessly managed by &PRODUCT;.</para> |
| <para>Artifacts such as templates, ISOs and snapshots are kept in storage which &PRODUCT; |
| refers to as secondary storage. To improve scalability and performance, as when a number |
| of hosts access secondary storage concurrently, object storage can be used for secondary |
| storage. Object storage can also provide built-in high availability capability. When using |
| object storage, access to secondary storage data can be made available across multiple |
| zones in a region. This is a huge benefit, as it is no longer necessary to copy templates, |
| snapshots etc. across zones as would be needed in an environment |
| using only zone-based NFS storage.</para> |
| <para>The user enables a storage plugin through the UI. |
| A new dialog box choice is offered to select the storage |
| provider. Depending on which provider is selected, additional input fields may appear so that |
| the user can provide the additional details required by that provider, such as a user name and |
| password for a third-party storage account. |
| </para> |
| <section id="storage-plugin-steps"> |
| <title>Overview of How to Write a Storage Plugin</title> |
| <para>To add a third-party storage option to &PRODUCT;, follow these general steps (explained in more detail |
| later in this section):</para> |
| <orderedlist> |
| <listitem><para>Implement the following interfaces in Java:</para> |
| <itemizedlist> |
| <listitem><para>DataStoreDriver</para></listitem> |
| <listitem><para>DataStoreLifecycle</para></listitem> |
| <listitem><para>DataStoreProvider</para></listitem> |
| <listitem><para>VMSnapshotStrategy (if you want to customize the VM snapshot functionality)</para></listitem> |
| </itemizedlist></listitem> |
| <listitem><para>Hardcode your plugin's required additional |
| input fields into the code for the Add Secondary Storage |
| or Add Primary Storage dialog box.</para></listitem> |
| <listitem><para>Place your .jar file in plugins/storage/volume/ or plugins/storage/image/.</para></listitem> |
| <listitem><para>Edit /client/tomcatconf/componentContext.xml.in.</para></listitem> |
| <listitem><para>Edit client/pom.xml.</para></listitem> |
| </orderedlist> |
| </section> |
| <section id="datastoredriver"> |
| <title>Implementing DataStoreDriver</title> |
| <para>DataStoreDriver contains the code that &PRODUCT; will use to provision the object store, when needed.</para> |
| <para>You must implement the following methods:</para> |
| <itemizedlist> |
| <listitem><para>getTO()</para></listitem> |
| <listitem><para>getStoreTO()</para></listitem> |
| <listitem><para>createAsync()</para></listitem> |
| <listitem><para>deleteAsync()</para></listitem> |
| </itemizedlist> |
| <para>The following methods are optional:</para> |
| <itemizedlist> |
| <listitem><para>resize()</para></listitem> |
| <listitem><para>canCopy() is optional. If you set it to true, then you must implement copyAsync().</para></listitem> |
| </itemizedlist> |
| </section> |
| <section id="datastorelifecycle"> |
| <title>Implementing DataStoreLifecycle</title> |
| <para>DataStoreLifecycle contains the code to manage the storage operations for ongoing use of the storage. |
| Several operations are needed, like create, maintenance mode, delete, etc.</para> |
| <para>You must implement the following methods:</para> |
| <itemizedlist> |
| <listitem><para>initialize()</para></listitem> |
| <listitem><para>maintain()</para></listitem> |
| <listitem><para>cancelMaintain()</para></listitem> |
| <listitem><para>deleteDataStore()</para></listitem> |
| <listitem><para>Implement one of the attach*() methods depending on what scope you want the storage to have: attachHost(), attachCluster(), or attachZone().</para></listitem> |
| </itemizedlist> |
| </section> |
| <section id="datastoreprovider"> |
| <title>Implementing DataStoreProvider</title> |
| <para>DataStoreProvider contains the main code of the data store.</para> |
| <para>You must implement the following methods:</para> |
| <itemizedlist> |
| <listitem><para>getDatastoreLifeCycle()</para></listitem> |
| <listitem><para>getDataStoreDriver()</para></listitem> |
| <listitem><para>getTypes(). Returns one or more types of storage for which this data store provider can be used. |
| For secondary object storage, return IMAGE, and for a Secondary Staging Store, return ImageCache.</para></listitem> |
| <listitem><para>configure(). First initialize the lifecycle implementation and the driver implementation, |
| then call registerDriver() to register the new object store provider instance with &PRODUCT;.</para></listitem> |
| <listitem><para>getName(). Returns the unique name of your provider; for example, |
| this can be used to get the name to display in the UI.</para></listitem> |
| </itemizedlist> |
| <para>The following methods are optional:</para> |
| <itemizedlist> |
| <listitem><para>getHostListener() is optional; it's for monitoring the status of the host.</para></listitem> |
| </itemizedlist> |
| </section> |
| <section id="implement-vmsnapshotstrategy"> |
| <title>Implementing VMSnapshotStrategy</title> |
| <para>VMSnapshotStrategy has the following methods:</para> |
| <itemizedlist> |
| <listitem><para>takeVMSnapshot()</para></listitem> |
| <listitem><para>deleteVMSnapshot()</para></listitem> |
| <listitem><para>revertVMSnapshot()</para></listitem> |
| <listitem><para>canHandle(). For a given VM snapshot, |
| tells whether this implementation of VMSnapshotStrategy can handle it.</para></listitem> |
| </itemizedlist> |
| </section> |
| <section id="storage-plugin-code-location"> |
| <title>Place the .jar File in the Right Directory</title> |
| <para>For a secondary storage plugin, place your .jar file here:</para> |
| <programlisting>plugins/storage/image/</programlisting> |
| <para>For a primary storage plugin, place your .jar file here:</para> |
| <programlisting>plugins/storage/volume/</programlisting> |
| </section> |
| <section id="storage-plugin-componentcontext-xml"> |
| <title>Edit Configuration Files</title> |
| <para>First, edit the following file tell &PRODUCT; to include your .jar file. |
| Add a line to this file to tell the &PRODUCT; Management Server that it now has a dependency on your code:</para> |
| <programlisting>client/pom.xml</programlisting> |
| <para>Place some facts about your code in the following file so &PRODUCT; can run it:</para> |
| <programlisting>/client/tomcatconf/componentContext.xml.in</programlisting> |
| <para>In the section “Deployment configurations of various adapters,” add this:</para> |
| <programlisting><bean>id=”some unique ID” class=”package name of your implementation of DataStoreProvider”</bean></programlisting> |
| <para>In the section “Storage Providers,” add this:</para> |
| <programlisting><property name=”providers”> |
| <ref local=”same ID from the bean tag's id attribute”> |
| </property></programlisting> |
| </section> |
| <section> |
| <title>Minimum Required Interfaces</title> |
| <para>The classes, interfaces, and methods used by &PRODUCT; from the Amazon Web Services (AWS) Java SDK |
| are listed in this section. |
| An object storage that supports the S3 interface |
| is minimally required to support the below in order to be compatible with &PRODUCT;.</para> |
| <section> |
| <title>Interface AmazonS3</title> |
| <para><ulink url="http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3.html">http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3.html</ulink></para> |
| <informaltable> |
| <tgroup cols="2" align="left" colsep="1" rowsep="1"> |
| <colspec colwidth="1.0*" colname="1" colnum="1"/> |
| <colspec colwidth="3.35*" colname="2" colnum="2"/> |
| <thead> |
| <row> |
| <entry><para>Modifier and Type</para></entry> |
| <entry><para>Method and Description</para></entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry><para>Bucket</para></entry> |
| <entry><para>createBucket(String bucketName)</para> |
| <para>Creates a new Amazon S3 bucket with the specified name in the default (US) region, Region.US_Standard.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>void</para></entry> |
| <entry><para>deleteObject(String bucketName, String key)</para> |
| <para>Deletes the specified object in the specified bucket.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>ObjectMetadata</para></entry> |
| <entry><para>getObject(GetObjectRequest getObjectRequest, File destinationFile)</para> |
| <para>Gets the object metadata for the object stored in Amazon S3 |
| under the specified bucket and key, and saves the object contents to the specified file.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>S3Object</para></entry> |
| <entry><para>getObject(String bucketName, String key)</para> |
| <para>Gets the object stored in Amazon S3 under the specified bucket and key.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>URL</para></entry> |
| <entry><para>generatePresignedUrl(String bucketName, String key, Date expiration, HttpMethod method)</para> |
| <para>Returns a pre-signed URL for accessing an Amazon S3 resource.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>void</para></entry> |
| <entry><para>deleteBucket(String bucketName)</para> |
| <para>Deletes the specified bucket.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>List<Bucket></para></entry> |
| <entry><para>listBuckets()</para> |
| <para>Returns a list of all Amazon S3 buckets that the authenticated sender of the request owns.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>ObjectListing</para></entry> |
| <entry><para>listObjects(String bucketName, String prefix)</para> |
| <para>Returns a list of summary information about the objects in the specified bucket.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>PutObjectResult</para></entry> |
| <entry><para>putObject(PutObjectRequest putObjectRequest)</para> |
| <para>Uploads a new object to the specified Amazon S3 bucket.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>PutObjectResult</para></entry> |
| <entry><para>putObject(String bucketName, String key, File file)</para> |
| <para>Uploads the specified file to Amazon S3 under the specified bucket and key name.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>PutObjectResult</para></entry> |
| <entry><para>putObject(String bucketName, String key, InputStream input, ObjectMetadata metadata)</para> |
| <para>Uploads the specified input stream and object metadata to Amazon S3 under the specified bucket and key name.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>void</para></entry> |
| <entry><para>setEndpoint(String endpoint)</para> |
| <para>Overrides the default endpoint for this client.</para> |
| </entry> |
| </row> |
| <row> |
| <entry><para>void</para></entry> |
| <entry><para>setObjectAcl(String bucketName, String key, CannedAccessControlList acl)</para> |
| <para>Sets the CannedAccessControlList for the specified object in Amazon S3 using one of the pre-configured CannedAccessControlLists.</para> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section> |
| <title>Class TransferManager</title> |
| <para><ulink url="http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html">http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html</ulink></para> |
| <informaltable> |
| <tgroup cols="2" align="left" colsep="1" rowsep="1"> |
| <colspec colwidth="1.0*" colname="1" colnum="1"/> |
| <colspec colwidth="3.35*" colname="2" colnum="2"/> |
| <thead> |
| <row> |
| <entry><para>Modifier and Type</para></entry> |
| <entry><para>Method and Description</para></entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry><para>Upload</para></entry> |
| <entry><para>upload(PutObjectRequest putObjectRequest)</para> |
| <para>Schedules a new transfer to upload data to Amazon S3.</para> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| <section> |
| <title>Class PutObjectRequest</title> |
| <para><ulink url="http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/PutObjectRequest.html">http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/PutObjectRequest.html</ulink></para> |
| <informaltable> |
| <tgroup cols="2" align="left" colsep="1" rowsep="1"> |
| <colspec colwidth="1.0*" colname="1" colnum="1"/> |
| <colspec colwidth="3.35*" colname="2" colnum="2"/> |
| <thead> |
| <row> |
| <entry><para>Modifier and Type</para></entry> |
| <entry><para>Method and Description</para></entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry><para>Upload</para></entry> |
| <entry><para>upload(PutObjectRequest putObjectRequest)</para> |
| <para>Schedules a new transfer to upload data to Amazon S3.</para> |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </informaltable> |
| </section> |
| </section> |
| </chapter> |
