The existing approach for deleting objects in a container suffers
from a head-of-line blocking problem. This commit implements a better
scheme which does not have that problem. This scheme uses a counting
semaphore for making sure that a certain number of futures are
issued in parallel. As each of these futures is completed, one
permit of the semaphore is released.
Added unit tests for testing this new scheme.
When issuing many simultaneous requests to Synaptic Atmos I observed:
HTTP/1.1 failed with code 500, error: AtmosError
[code=1040, message=The server is busy. Please try again.]
Previously all clients slept for fixed intervals and thus retried
around the same time. This commit adds a random delay which should
better distribute load on the provider.
Previously jclouds could use an unlimited number of threads on its
user ExecutorService. While this ExecutorService will go away when we
complete deasyncafication, we should prevent jclouds from misbehaving
until that time.
FileBackedOutputStream.asByteSource.getInput returns a FileInputStream
which we do not close. We later call FileBackedOutputStream.reset
which removes the underlying File. This fails on Windows which does
not support deleting an open file and leaks resources on other
platforms. Eagerly close to address this issue.
Payload.getInput must always call openStream to handle overridden
methods correctly. Previously this caused errors in jclouds-chef in
BaseCipherPayload.
Also deprecate byte[], File, InputSupplier<InputStream>, and String
Payloads. Callers should instead provide a ByteSource via
ByteSource.wrap(byte[]) and Files.asByteSource(File)
We plan to transition Payload to ByteSource in the next major release.
Unfortunately Payload.getInput masks its checked exception and
ByteSource.getInput is final so we cannot continue to mask the
exceptions. Deprecation of getInput and addition openStream allows us
to transition callers from the former to the latter.
This changeset introduces an alternative to PayloadSlicer,
IterablePayloadSlicer, with a method for returning a Payload iterator.
...swift.blobstore.strategy.internal.SequentialMultipartUploadStrategy
has been updated to to use a payload iterator.
This allows jclouds to factor out common headers into the Caller so they don't
have to be repeated in the Callee.
The Produces/Consumes annotations (Content-Type/Accept headers) will also
propagate from the Caller to the Callee.
This patch moves the Invokable Parameter cache to Reflection2 and adds
a convenience method for it to allow it to be shared by multiple
callers. The subsequent ability of S3Utils to use this cache results
in a ~40% improvement in performance for generating signed GETs and
PUTs for S3. This commit also converts a few others calls to
Invokable.getParameters() but the observed benefit from those was
small in microbenchmarks.
Atmos does not return a location header when writing zero-length
objects, which normally throws an HttpResponseException: no uri in
headers or content.
BlobStore.createContainerInLocation is supposed to return True if the
container was newly created and False if the container already
existed. This commit makes that happen for Swift blobstores.
Not all S3-compatible providers support virtual host buckets and thus
we should disable this feature by default. Continue to enable virtual
host buckets for AWS-S3 which supports this although this feature
suffers from DNS settling issues. Ran ran integration tests against
AWS-S3 and Scality using its S3 API.
Andrew Gaul
Pasi Niemi
Shri Javadekar
Andrew Gaul
Andrew Phillips
Zack Shoylev
Ignasi Barrera
Andrew Bayer
Eric Evans
Everett Toews
Ignacio Mulas
Niraj Tolia