This changeset introduces an alternative to PayloadSlicer,
IterablePayloadSlicer, with a method for returning a Payload iterator.
...swift.blobstore.strategy.internal.SequentialMultipartUploadStrategy
has been updated to to use a payload iterator.
The number of retries here is not the same as for 500 errors; expected
behavior is a quick fail while retaining some robustness. This fix
should not reintroduce JCLOUDS-231.
This patch moves the Invokable Parameter cache to Reflection2 and adds
a convenience method for it to allow it to be shared by multiple
callers. The subsequent ability of S3Utils to use this cache results
in a ~40% improvement in performance for generating signed GETs and
PUTs for S3. This commit also converts a few others calls to
Invokable.getParameters() but the observed benefit from those was
small in microbenchmarks.
Atmos does not return a location header when writing zero-length
objects, which normally throws an HttpResponseException: no uri in
headers or content.
BlobStore.createContainerInLocation is supposed to return True if the
container was newly created and False if the container already
existed. This commit makes that happen for Swift blobstores.
So there were a few problems, but the core ugly one is that what you
pass in for creating a rule allowing groups' access is not the same
thing you get back from a group with such a rule, which makes mapping
between the arguments and the output insanely painful. So now, well,
we do some insanely painful stuff.
Involves some non-trivial reworking of EC2 security group
translation/binding, but it was necessary.
Still problems with a number of tests:
- org.jclouds.aws.ec2.compute.AWSEC2TemplateBuilderLiveTest#testTemplateBuilderWithLessRegions
- org.jclouds.ec2.compute.EC2TemplateBuilderLiveTest#testTemplateBuilderCanUseImageIdWithoutFetchingAllImages
Those seem to be failing because there's caching going on that they're
not expecting, but I could be wrong.
Also failing:
- org.jclouds.aws.ec2.features.PlacementGroupApiLiveTest in general
- org.jclouds.aws.ec2.features.SpotInstanceApiLiveTest - a couple tests
This removes
org.jclouds.ec2.compute.functions.PasswordCredentialsFromWindowsInstanceLiveTest
due to there not being an easily detectable valid AMI for it to run,
and low interest in actually running it in the first place.
Not all S3-compatible providers support virtual host buckets and thus
we should disable this feature by default. Continue to enable virtual
host buckets for AWS-S3 which supports this although this feature
suffers from DNS settling issues. Ran ran integration tests against
AWS-S3 and Scality using its S3 API.
By caching the results from Invokable.getParameters(), this commit
improves request signing performance (GETs and PUTs) for S3 by >
3X. These performance problems were seen in production and diagnosed
using the YourKit profiler.
Also:
- Make SwiftBlobIntegrationLiveTest.testMultipartChunkedFileStream more realistic by uploading a file large enough to be split into parts.
- JavaDoc fixes for SwiftBlobStore: don't reference nonexistent methods.
This avoids a 'java.io.IOException: Unrecognized character: "' when a quoted ETag is encountered (which happens when getting a multipart blob from Rackspace Cloud).
Also adds support for the new options to
CloudStackComputeServiceAdapter, unsurprisingly. Also got rid of the
in fact wrong mutual exclusivity of dataDiskSize and diskOfferingId in
DeployVirtualMachineOptions - that's a misleading bit from the
CloudStack API docs.
Introduces AWSS3BlobRequestSigner, which reuses the
RequestAuthorizeSignature filter for most of the heavy lifting.
Other implementation details based on [1].
Tested with AWSS3BlobSignerLiveTest, in particular,
testSign(Get|Put)UrlWithTime.
Closes JCLOUDS-200
[1] http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html
- ACS 4.x doesn't like taking SSH pub keys from the filesystem, so
generate them on the fly.
- vm.getDisplayName() can be null now.
- Add new possible resource limit types.
- Default to looking template=osFamily=CENTOS, since that's the only
template guaranteed to be there in the simulator.
- Use adminJobComplete instead of jobComplete in admin tests
- Accept capacity/usage/etc of 0.
- Premium configuration category not present in ACS.
- Sleep a bit between deleting a domain and verifying it's not there
any more. Also expect an IllegalStateException.
- Given that there are issues deleting zones at the moment (through
the UI, too), use a different zone for pod and zone tests.
Still failing tests:
- pretty much everything that creates a VM and expects to log into it,
but that's simulator-specific.
- Zone deletion, due to a bug in ACS, apparently.
- Registering and creating templates
- creating volumes from snapshots, and attaching volumes
Andrew Gaul
Andrew Bayer
Hyeonu Jeong
Everett Toews
Zack Shoylev
Ignacio Mulas
Eric Evans
Adrian Cole
Alex Heneveld
jolly
istolber
Niraj Tolia
Shri Javadekar
Francis Devereux
einsdo
Andrew Phillips
Diwaker Gupta