The 1.1.0 release of jclouds includes results of 2 months effort by our contributors. A total of 30 Issues were addressed in this release. Highlights include more robust ssh connections via sshj driver, a more robust EC2 implementation, and support for the Green House Data vCloud. Many thanks to our contributors, Shikhar from sshj, and the cloud providers for facilitating the release.
As always, we keep our examples site up to date so you can see how to work this stuff. Next release will be in about a month and look for progress including CloudStack 2.2.8 and Softlayer providers in the next version.
Please submit your own ideas and let us know if there are features you'd like to see, need help on, or are interested in contributing. Make sure you follow us on Twitter for updates.
Green House Data run a completely wind-powered public vCloud offering in Wyoming, accessible as greenhousedata-element-vcloud in jclouds. Check out their site for more details.
Troubleshooting ssh connections consumed a lot of time during past releases of jclouds. This is because clouds have different operating system configurations, and sometimes distant or complicated networks between you and the nodes. Having a reliable ssh client implementation can significantly cut debugging time when problems occur. We‘ve found [https://github.com/shikhar/sshj sshj] the most reliable and hackable open source java ssh library, and importantly with best feedback when failures occur. Here’s how to configure it, noting you'll need to include the org.jclouds.driver/jclouds-sshj dependency in your codebase.
java
{% highlight java %} compute = new ComputeServiceContextFactory().createContext(provider, identity, credential, ImmutableSet. of(new SshjSshClientModule()), properties); {% endhighlight %}
clojure
{% highlight clojure %} (def compute (compute-service provider provider-identity provider-credential :sshj)) {% endhighlight %}
Those writing shell scripts from data inside !NodeMetadata often need to know what the local hostname is of the operating system you are working against. We now return hostname data on clouds that assign this value predictably, or null if unpredictable.
As of 1.1.0, the following apis and providers assign hostnames predictably: byon, aws-ec2, cloudservers-us|uk, slicehost, and trmk-ecloud|vcloudexpress.
When working with a multi-site Terremark or vCloud provider, you may want to change the default datacenter or network to launch VMs into, rather than have jclouds pick one. With new configuration properties, you can specify the name of the Org, VDC, Catalog, and Network you prefer for a particular invocation of jclouds.
{% highlight java %} overrides = new Properties(); overrides.setProperty(VCloudConstants.PROPERTY_VCLOUD_DEFAULT_VDC, “MyCompany - MIA”); context = new ComputeServiceContextFactory().createContext(“trmk-ecloud”, email, pass, ImmutableSet. of(new SshjSshClientModule()), overrides) {% endhighlight %}
Even refining lists to a set of user ids, using EC2 can be bogged down, parsing the thousands of public images published by the Amazon EC2 community. Several users have expressed the desire to refine further, based on qualifications they choose. You can set a property to restrict the list to a subset matching any recognized image query.
{% highlight java %} overrides = new Properties(); // choose only amazon images that are ebs-backed overrides.setProperty(AWSEC2Constants.PROPERTY_EC2_AMI_QUERY, “owner-id=137112412989;state=available;image-type=machine;root-device-type=ebs”); context = new ComputeServiceContextFactory().createContext(“aws-ec2”, access, secret, ImmutableSet. of(new SshjSshClientModule()), overrides) {% endhighlight %}
See EC2 Image Filters for more.
For those using the “aws-ec2” provider, you'll notice that AWS now supports security groups constraints that are more fine grained than before. For example, you can now assign rules constrained by source group and also port. We now have an fluent api for creating and applying ingress rules. {% highlight java %} // get a hold of an ec2 client so you can access security group services securityGroupClient = AWSEC2Client.class.cast(context.getProviderSpecificContext().getApi()).getSecurityGroupServices();
// create a new group and get back its id groupId = securityGroupClient.createSecurityGroupInRegionAndReturnId(regionOrNull, groupName, groupDescription);
// add new rule(s) to the group securityGroupClient.authorizeSecurityGroupIngressInRegion(regionOrNull, groupId, IpPermissions.permit(IpProtocol.UDP).fromPort(11).to(53) .originatingFromSecurityGroupId(“otherGroup”)); {% endhighlight %}
See Issue 606 for more examples.
During the recent aws outage, instances which used EBS (elastic block store) were more impacted from an availability standpoint. We now have a template builder option that allows you to prefer instance store.
{% highlight java %} // match the smallest hardware configuration that will run amazon linux with an S3-backed image. // Note that t1.micro does not support instance store, therefore this will match m1.small. Template template = templateBuilder.osFamily(OsFamily.AMZN_LINUX) .imageMatches(EC2ImagePredicates.rootDeviceType(RootDeviceType.INSTANCE_STORE)).build(); {% endhighlight %}
Terremark vCloud Express was the first operable vCloud Express environment. Over time, the vCloud model changed as did Terremark offerings, in different directions. Troubleshooting and coding these models became complicated over time, with vain attempts to share code across incompatible models. In the jclouds 1.1.0, we‘ve cut vcloud models from Terremark models, simplifying maintenance of both. This doesn’t impact use of ComputeService, but it does impact the provider-specific terremark apis, as they've been repackaged under org.jclouds.trmk. Also, the maven dependencies of org.jclouds.api/vcloud, org.jclouds.provider/trmk-ecloud, and org.jclouds.provider/trmk-vcloudexpress have been refactored.
Many jclouds ComputeService users install middleware via our runScript functionality directly as opposed to using a configuration tool. In order to test our ability to deploy simple configurations, we formerly used JBoss version 6.0. However, this took a long while to startup, was a big image, and generally slowed testing. Seeing tweets about version 7, we were quite excited to see improvements in test execution, based on claims. We‘ve verified very fast startup, many times under 2 seconds using AS7 in our service-based testing. If you’d like more details or results, have a look at the [issue]http://code.google.com/p/jclouds/issues/detail?id=616) we migrated to AS7 in. Also, if you are interested in contributing a test using a different product, let us know!
Status of the release is tracked in issue 641
| maven dependency | abstraction | iso 3166 codes | result | notes |
|---|---|---|---|---|
| org.jclouds.provider/aws-ec2 | compute | US-VA,US-CA,IE,SG | pass | only failure was a timeout from my laptop |
| org.jclouds.provider/aws-s3 | blobstore | US,US-CA,IE,SG | pass | test failure was due to client-side network timeout |
| org.jclouds.provider/azureblob | blobstore | US-TX,US-IL,IE-D,SG,NL-NH,HK | pass | one glitch on integration test, likely due to slow wifi |
| org.jclouds.provider/bluelock-vcloud-vcenterprise | compute | US-IN | pass | |
| org.jclouds.provider/bluelock-vcloud-zone01 | compute | US-IN | pass | |
| org.jclouds.provider/cloudfiles-uk | blobstore | GB-SLG | pass | |
| org.jclouds.provider/cloudfiles-us | blobstore | US-TX | pass | |
| org.jclouds.provider/cloudloadbalancers-us | loadbalancer | US-IL,US-TX | pass | |
| org.jclouds.provider/cloudonestorage | blobstore | US-GA,US-TX | pass | see note on atmos. operations except ones requiring ListContainerOptions.recursive operate |
| org.jclouds.provider/cloudservers-uk | compute | GB-SLG | pass | |
| org.jclouds.provider/cloudservers-us | compute | US-IL,US-TX | pass | |
| org.jclouds.provider/cloudsigma-zrh | compute | CH-ZH | unknown | operations seem to work fine, but since images don't support automated ssh login, we cannot verify |
| org.jclouds.provider/elastichosts-lon-b | compute | GB-LND | pass | |
| org.jclouds.provider/elastichosts-lon-p | compute | GB-LND | pass | |
| org.jclouds.provider/elastichosts-sat-p | compute | US-TX | pass | |
| org.jclouds.provider/eucalyptus-partnercloud-ec2 | compute | US-CA | unknown | account doesn't have enough resources to verify things operate |
| org.jclouds.provider/eucalyptus-partnercloud-s3 | blobstore | US-CA | unstable | acls are not compatible with s3 (testPublicReadOnObject,testPublicWriteOnObject,testUpdateObjectACL); lack of NextMarker makes list continuations unreliable (testListContainerMarker testPutMoreThanSingleListing); different http code on etag conditional (testGetIfMatch) |
| org.jclouds.provider/gogrid | compute | US-CA,US-VA | unstable | sometimes metadata like running status flip off our tests. Since I cannot get a clean run after several tries, I marking this unstable conceding most tests do pass. |
| org.jclouds.provider/greenhousedata-element-vcloud | compute | US-WY | pass | |
| org.jclouds.provider/ninefold-storage | blobstore | AU-NSW | pass | see note on atmos. operations except ones requiring ListContainerOptions.recursive operate |
| org.jclouds.provider/openhosting-east1 | compute | US-VA | pass | works from a computeService perspective, but don‘t expect openjdk to install properly on their default image, as the tests show there’s problems installing it |
| org.jclouds.provider/rimuhosting | compute | NZ-AUK,US-TX,AU-NSW,GB-LND | unstable | rimuhosting works while servers are launched sequentially. During the tests, servers launched sequentially operated as expected, but when 2 were launched at the same time, we started to receive unexpected content back from the rest api |
| org.jclouds.provider/savvis-symphonyvpdc | compute | US,CA | unstable | NOTE this provider does not yet implement ComputeService interface, so you have to use the provider api; a few minor tests fail, and since we don't have comprehensive support due to lacking ComputeService impl, marking unstable |
| org.jclouds.provider/serverlove-z1-man | compute | GB-MAN | pass | |
| org.jclouds.provider/skalicloud-sdg-my | compute | MY-10 | pass | |
| org.jclouds.provider/slicehost | compute | US-IL,US-TX,US-MO | pass | slicehost slice status is sometimes inconsistent and reports pending when in active state |
| org.jclouds.provider/stratogen-vcloud-mycloud | compute | GB | pass | passes critical tests, and a re-run should go clean |
| org.jclouds.provider/synaptic-storage | blobstore | US-VA,US-TX | pass | see note on atmos. operations except ones requiring ListContainerOptions.recursive operate |
| org.jclouds.provider/trmk-ecloud | compute | US-FL,NL-NH | pass | functionality works, with a few glitches: overriding credentials via runScriptOnNodes doesn't seem to apply (testAScriptExecutionAfterBootWithBasicTemplate);NPE attempting to create a new node after closing and recreating your context in the same VM (testCreateAnotherNodeWithANewContextToEnsureSharedMemIsntRequired) |
| org.jclouds.provider/trmk-vcloudexpress | compute | US-FL | pass | same glitches as ecloud |
| maven dependency | abstraction | endpoint tested | result | notes |
|---|---|---|---|---|
| org.jclouds.api/atmos | blobstore | pass | one exception ListContainerOptions.recursive listings via BlobStore Map interfaces do not actually recurse. This is issue 654 | |
| org.jclouds.api/byon | compute | pass | ||
| org.jclouds.api/cloudfiles | blobstore | pass | didn't test directly, rather via cloudfiles-* | |
| org.jclouds.api/cloudservers | blobstore | pass | didn't test directly, rather via cloudservers-* | |
| org.jclouds.api/deltacloud | compute | unknown | didn‘t test, but suspect this doesn’t work as we were on a snapshot version of deltacloud last time | |
| org.jclouds.api/elasticstack | compute | pass | didn't test directly, rather through elastichosts-* | |
| org.jclouds.api/eucalyptus | compute | unknown | was not able to get a hold of a stable env | |
| org.jclouds.api/filesystem | blobstore | pass | ||
| org.jclouds.api/nova | compute | pass | ||
| org.jclouds.api/swift | blobstore | http://50.16.106.110:11000 | pass | |
| org.jclouds.api/vcloud | compute | pass | tested via greenhousedata-element-vcloud | |
| org.jclouds.api/walrus | blobstore | unstable | see eucalyptus-partnercloud-s3 |