While they're implemented in apis/ec2, the tests are in
providers/aws-ec2, generally, to make sure ec2-alike clones won't barf
on them. We're exercising creation of volumes, images and instances
with the new options. I also had to do some sketchy wait-and-loop'ing
in AMIAPILiveTest.testCreateAndListEBSBackedImage() due to what seems
to be a delay on new AMIs showing up in filtered DescribeImages calls,
though they'll show up instantly when you specify the image ID. Go figure.
Images were cached in memory using a memoized supplier. To allow growing
this cache with the discovered images, the ImageCacheSupplier class has
been created. It provides an in-memory cache with all discovered images
and acts as a view over the image cache that also provides access to
them.
The in-memory cache for the discovered images expires with the session,
just as the image cache does.
The default memoized image supplier has been changed to the
ImageCacheSupplier, to make sure all providers get injected the right
instance, and the old supplier has been qualified with the 'imageCache'
name, in case a provider needs the basic image cache.
If the TemplateBuilderImpl is given an imageId but the image can not be
found in the image cache, fallback to the GetImageStrategy to perform a
call to the provider to try to get it.
We've seen that in some cases images are not returned in the image list
but they actually exist in the provider. This fix won't make them
available when filtering by other properties such as the operating system,
etc, but at least will make them available if their id is known.
Added the new ElasticHosts regions.
Updated the ElasticStack api to get the list of standard
drives using an API call. All providers except ServerLove
support the new API call, so the old logic in the ElasticStack
api has been moved to that provider. The rest of providers will now
extract all the OperatingSystem information by parsing the name of the
StandardDrive.
A unit test has been added to the ElasticStack api with all the images
that were hardcoded, to make sure all names are still parsed as expected
and all information in the existing providers is kept.
Modified the default template for all ElasticHosts providers to
match newer Ubuntu images and updated the Template*Live tests
accordingly.
Also refactored the WellKnownImage map to a supplier to lazy load it
when needed and avoid unexpected errors when building the Guice injector
if there are authentication errors or similar.
Callers should instead explicitly set contentMD5, usually with the
results from Guava Hashing.md5(). This narrows the API and removes a
strange IOException from callers. Further it removes a dangerous
rebuffering of arbitrarily-large non-repeatable Payloads.
The only users of this seem to be
org.jclouds.atmos.blobstore.strategy.FindMD5InUserMetadata and
org.jclouds.azureblob.blobstore.strategy.FindMD5InBlobProperties which
are themselves unused.
- Adds networks field/methods to TemplateOptions.
- Adds them to children as well for legacy reasons.
- Deprecates CloudStackTemplateOptions#networkIds methods in favor of #networks.
- TODO: Modify compute abstraction layer for provisioning for nova,
EC2, et al to take advantage of this.
* Wrap getOrderTemplate in VirtualGuestToNodeMetadata.getImage with try/catch
* Address comments from @nacx and @demobox:
- remove the fields of the custom SoftLayerOrderItemDuplicateException and propagate the HttpResponseException
- use of order = null instead of return null
- extend RuntimeException instead IllegalStateException in SoftLayerOrderItemDuplicateException
Conflicts solved:
providers/softlayer/src/main/java/org/jclouds/softlayer/compute/functions/VirtualGuestToNodeMetadata.java
* Wrap getOrderTemplate in VirtualGuestToNodeMetadata.getImage with try/catch
* Address comments from @nacx and @demobox:
- remove the fields of the custom SoftLayerOrderItemDuplicateException and propagate the HttpResponseException
- use of order = null instead of return null
- extend RuntimeException instead IllegalStateException in SoftLayerOrderItemDuplicateException
Conflicts solved:
providers/softlayer/src/main/java/org/jclouds/softlayer/compute/functions/VirtualGuestToNodeMetadata.java
So there were a few problems, but the core ugly one is that what you
pass in for creating a rule allowing groups' access is not the same
thing you get back from a group with such a rule, which makes mapping
between the arguments and the output insanely painful. So now, well,
we do some insanely painful stuff.
Involves some non-trivial reworking of EC2 security group
translation/binding, but it was necessary.
Still problems with a number of tests:
- org.jclouds.aws.ec2.compute.AWSEC2TemplateBuilderLiveTest#testTemplateBuilderWithLessRegions
- org.jclouds.ec2.compute.EC2TemplateBuilderLiveTest#testTemplateBuilderCanUseImageIdWithoutFetchingAllImages
Those seem to be failing because there's caching going on that they're
not expecting, but I could be wrong.
Also failing:
- org.jclouds.aws.ec2.features.PlacementGroupApiLiveTest in general
- org.jclouds.aws.ec2.features.SpotInstanceApiLiveTest - a couple tests
This removes
org.jclouds.ec2.compute.functions.PasswordCredentialsFromWindowsInstanceLiveTest
due to there not being an easily detectable valid AMI for it to run,
and low interest in actually running it in the first place.
Not all S3-compatible providers support virtual host buckets and thus
we should disable this feature by default. Continue to enable virtual
host buckets for AWS-S3 which supports this although this feature
suffers from DNS settling issues. Ran ran integration tests against
AWS-S3 and Scality using its S3 API.
Refactored the domain model to be immutable and addressed some
inconsistences with the Chef Server API model.
Removed all HEAD methods, as they have been removed from newer versions
of Chef. They were used to test the existance of a given resource, and
with newer versions the only way to do that is via a GET operation.
Now all live tests are passing for Community Chef 0.10.8, 11.0.6 and
Enterprise Chef.
As recommended at
http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html:
"You can also send a signature as a URL-encoded query-string parameter
in the URL for the request."
Also deals with some of the craziness of URL encoding/decoding in
jclouds.
References JCLOUDS-200
Introduces AWSS3BlobRequestSigner, which reuses the
RequestAuthorizeSignature filter for most of the heavy lifting.
Other implementation details based on [1].
Tested with AWSS3BlobSignerLiveTest, in particular,
testSign(Get|Put)UrlWithTime.
Closes JCLOUDS-200
[1] http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html
Large blob support for AzureClient; the next step of this is to
support PutOptions.multipart and digest a blob into 4M parts. This
just implements the Azure interaction.
...and EC2-related elsewhere. Also moved *Client -> *Api, and moved
everything from .../services to .../features, and threw in a bunch of
Optionals to fit the EC2Api approach.
And a very big tip of the hat to nacx for figuring out the generics
stuff my brain just could not handle. =)
- Adds the SecurityGroupExtension to compute, with tests and stub
support.
- Gets everything else to actually build against this.
- Unifies on compute's IpPermission/IpProtocol, eliminating EC2's.
- Converters from EC2/Nova/CloudStack SecurityGroup (and rules, for
the latter two) to the compute SecurityGroup (and rules, etc).
- EC2SecurityGroupExtension and tests.
- AWSEC2SecurityGroupExtension and tests - depends on JCLOUDS-99.
- Added AWSEC2CreateSecurityGroupIfNeeded, using
AWSSecurityGroupClient's ability to call
authorizeSecurityGroupInRegion with Iterable<IpPermission>
- Added a utility method,
ComputeServiceUtils.getPortRangesInList(int... ports), to get pairs of
start/end ports covering all ranges in a list of ports. Used above.
Andrea Turli
Jeremy Daggett
Andrew Gaul
Roman C. Coedo
Chris Custine
Andrew Bayer
Ignasi Barrera
Matheus Cunha
Bhathiya90
Zack Shoylev
Andrew Phillips
jolly
Diwaker Gupta
Everett Toews
Adrian Cole
Aled Sage
Tom Manville
Andrew Phillips
John Kew
Ignasi Barrera
Andrew Gaul