03248b4c94
JCLOUDS-930: LocalStore -- fixup for prefixes.
...
When filtering results, we have to consider that a result was added to
the common prefixes list in its entirety (i.e. a directory). Such
results should be filtered out from the delimiter test.
An example that demonstrates this problem is if one creates a
directory "foo" and an object within it called "file". When listing
the results, they will include the directory object "foo" and the
object under "foo/file". During a non-recursive listing, we create a
list of common prefixes ("foo"). Subsequently, jclouds should remove
all objects that include the delimiter ("/"), however, that would not
apply to "foo". With the change to include the delimiter in the
listing, we need to be careful not to return two values "foo" and
"foo/".
A unit test for the local blob store to highlight this problem is
included. An integration test "testDirectory" also catches this issue.
2015-07-07 16:31:04 +02:00
3009da3761
JCLOUDS-930: Expect a trailing "/" in tests.
...
We should expect a trailing prefix in delimiter tests, as the object
stores will return common prefixes up to and including the delimiter
character.
2015-07-07 16:30:59 +02:00
14087e56bd
Relax membership assertion for the prefix test.
...
Relaxes the membership check in prefix testing. The check is no longer
sensitive to ordering of the results, as different providers may
append, prepend, or insert in sorted order the relative prefixes.
2015-07-06 15:10:29 -07:00
bf93cadec6
Use correct default in testPublicAccess
...
Regression from d57b6e7b8f
2015-07-02 16:25:49 -07:00
bc9cda2e37
JCLOUDS-930: Remove bogus dependsOnMethods
2015-07-02 16:21:03 -07:00
d109dd6326
Use prefix instead of marker to list MPU parts
2015-06-30 13:43:01 -07:00
aade18b76d
JCLOUDS-930: LocalBlobStore -- marker regression.
...
We should not append a "/" to the marker when returning list results
in the case of directories (RELATIVE_PATH), as the names will already
include the delimiter.
Added a jclouds test to catch such regressions in the local store in
the future.
2015-06-30 08:46:46 -07:00
8677ffcb21
JCLOUDS-930: Implement prefix for LocalBlobStore.
...
Implements prefix support for the local blob store. The patch allows
for correctly parsing prefixes that may not terminate with a delimiter
(i.e. foo with delimiter "/" and a key foobar/key, should return
foobar/ as the common path) and ones that do (i.e. foo/).
NOTE: there is a small change in behavior in this patch.
LocalBlobStore used to return the common prefixes without the
delimiter character ("/"). However, other providers do include the
delimiter (I checked S3 and Google Cloud Storage) and LocalBlobStore
should include it as well.
2015-06-29 17:31:55 -07:00
db4e4e5ba7
Factor out the common prefixes method.
...
Refactoring the code in the list() method to create a common
extractCommonPrefixes method.
2015-06-29 17:31:55 -07:00
7c7d2e2df6
JCLOUDS-930: Add the prefix option.
...
Add a prefix option to the ListContainerOptions class.
2015-06-29 17:31:55 -07:00
e63474a824
JCLOUDS-931: Sleep only after creating containers.
...
jclouds should issue the requests to create all of the containers in
the container pool before sleeping. The patch modifies the
createContainerAndEnsureEmpty() method to take an additional
parameter, which specifies whether awaitConsistency() should be called
or not.
2015-06-26 14:04:43 -07:00
8b325d585e
object stores typically sort the list of containers
2015-06-24 15:08:23 -07:00
d57b6e7b8f
Use correct port in testPublicAccess
...
Also reduce timeout from 20 minutes to 1.
2015-06-24 14:51:16 -07:00
f155242838
Add missing value in Content-Range header
...
This is related with #776 . It's necessary to add
the 'bytes' string before the bytes.
2015-06-19 08:24:18 +02:00
3a410a25a8
Test for Content-Range header in single range get
...
Also emit this header in LocalBlobStore.
2015-06-17 11:43:17 -07:00
1f7f8c8c86
Refine HTTP 416 into IllegalArgumentException
...
Providers yield different messages so remove parsing from test. Also
narrow exception handling scope.TTP 416 into IllegalArgumentException
Providers yield different messages so remove parsing from test. Also
narrow exception handling scope.
2015-06-16 21:47:30 -07:00
b0dfae1092
Use a configurable timeout for awaitConsistency
...
This allows setting to zero for strongly-consistency implementations
of s3 and swift stores.
2015-06-15 15:45:17 -07:00
6ed95d8968
fix range get for manifest object
...
swift doesn't return the etag if the client is doing a range
get of a manifest (SLO or DLO), this fixes SwiftObject so
etag is optional
2015-06-10 13:38:55 -07:00
23ef39416d
use awaitConsistency in more places so provider can override it
2015-06-09 23:05:37 -07:00
140fb7ea7b
fix multipart put of a blob with content md5
...
previously the parts will carry the content md5 of the entire
object, because unsetting the md5 of a ContentMetadata actually
didn't do anything
2015-06-09 22:59:30 -07:00
573ceb51c3
metadata and content metadata should have the same length
...
for range get in a LocalBlobStore, metadata.getSize() has the
original blob size and contentMetadata.getContentLength() has
the actual payload size. Other blobstores have the same size
in both
2015-06-08 14:28:16 -07:00
f14514c068
JCLOUDS-894: Add portable multipart upload
...
This unifies the provider multipart upload code paths and removes code
duplication.
2015-06-06 16:32:16 -07:00
00734ee755
JCLOUDS-894: Move MultipartUploadSlicingAlgorithm to core
2015-06-06 16:32:16 -07:00
1ccdae46fa
remove blob uri from transient blobstore
...
container may not be valid hostnames and the current blob uri
code can fail. there isn't really a way to use uri for transient
blobstore anyway, and removing it is the easier option. for
reference, filesystem blobstore also doesn't do this
2015-05-30 22:24:48 -07:00
f19853c278
Revert "metadata and content metadata should have the same length"
...
This reverts commit bbafc34d46
2015-05-27 22:19:15 -07:00
1103a778d2
implemented some basic range validations in local blob store
...
also fixed a range get bug in local blob store
2015-05-27 16:30:05 -07:00
bbafc34d46
metadata and content metadata should have the same length
...
for range get in a LocalBlobStore, metadata.getSize() has the
original blob size and contentMetadata.getContentLength() has
the actual payload size. Other blobstores have the same size
in both
2015-05-26 16:45:34 -07:00
f397bebeb2
JCLOUDS-894 Multipart upload code fixes for swift
2015-05-12 19:13:15 -05:00
b9c5b5b4e9
JCLOUDS-894: Expose local blobstore multipart operations
2015-05-01 13:07:23 -07:00
71cdba1571
JCLOUDS-894: Expose multipart component operations
2015-05-01 13:07:23 -07:00
40ba156937
JCLOUDS-651: portable copy object content metadata
2015-04-08 17:58:41 -07:00
b35295c238
JCLOUDS-651: Add @Beta annotations to copy methods
2015-04-08 17:12:51 -07:00
b6fcac9667
JCLOUDS-651: Fix filesystem integration tests in OSX
2015-04-07 10:53:48 +02:00
d8f48c48b4
JCLOUDS-651: Copy Swift system metadata
2015-04-06 15:35:59 -07:00
a6a232a7a1
JCLOUDS-651: Ensure copy replaces user metadata
...
Some object stores have additive semantics for metadata. As currently
implemented Azure, S3, and Swift consistently replace user metadata
but have different behavior for system metadata, e.g., Content-Type.
2015-04-03 17:50:33 -07:00
3cf8abc22e
JCLOUDS-651: Add copyBlob to portable abstraction
...
Some providers optimize this operation with a server-side copy. Add a
fallback implementation which does a client-side copy.
2015-04-02 15:36:52 -07:00
914ad04ae9
Fixes missing consistency waits in testDirectory
2015-04-02 17:14:40 -05:00
279a984fee
JCLOUDS-732: Transient portable object ACLs
2015-03-31 15:33:34 -07:00
a062bf841a
JCLOUDS-660: Transient portable container ACLs
2015-03-31 15:33:33 -07:00
e7940d52f9
Add ReadOnlyBlobStore
...
This prohibits mutating operations.
2015-03-31 13:13:19 -07:00
9d9acd1559
Add ForwardingBlobStore
...
Useful for implementing various facades: encryption, read-only, etc.
2015-03-31 13:13:19 -07:00
f469cf8b51
Next development version 2.0.0-SNAPSHOT
2015-03-24 10:52:01 +01:00
bb41ed4341
Apache jclouds 1.9.0 release
2015-03-24 10:35:42 +01:00
9f93941525
Add content length to exception message
2015-03-02 16:51:59 -08:00
a2ee7341e3
fix non-recursive list of dir with empty prefix
2015-02-23 17:52:42 -08:00
d8aea21f72
JCLOUDS-732: Local blobstore portable object ACLs
...
Intentionally unimplemented.
2015-02-21 10:28:02 -08:00
49cd0aeba2
JCLOUDS-732: portable object ACLs
2015-02-21 10:28:01 -08:00
bc81919c58
Add @Beta annotations to BlobStore ACL methods
...
Also add missing @Override annotations.
2015-02-21 10:18:57 -08:00
be7b9f4cc4
fix non-recursive list with empty prefix
2015-02-20 23:29:48 -08:00
290a458e5e
Fixes more timing issues in the BaseBlobIntegrationTest
2015-02-18 04:53:40 -06:00
07f1fb69a7
Fixes more tests with timing problems
2015-02-18 03:46:37 -06:00
14d4ce635f
Refactors BaseBlobIntegrationTest and eventual consistency fix
2015-02-17 04:49:35 -06:00
dce4cd29ec
Refactors consitency pause in blob tests and more fixes
2015-02-16 13:21:06 -06:00
a8dde0550b
Adds a few more pauses for eventual consistency
2015-02-16 11:54:26 -06:00
9898e99b50
JCLOUDS-144: Portable multi-blob delete
2015-02-15 01:06:03 -08:00
f6d43f6962
Fixes blob-related tests that fail occasionally
2015-02-13 17:25:15 -06:00
994f84e4f5
JCLOUDS-660: Local blobstore portable container ACLs
...
Intentionally unimplemented.
2015-02-12 17:26:47 -08:00
a19795800a
JCLOUDS-660: portable container ACLs
2015-02-12 17:26:47 -08:00
0423ba802a
Makes testSignPutUrlWithTime much slower but passes live testing
2015-02-06 21:29:44 -06:00
1f35e959aa
Makes the test testSignGetUrlWithTime more robust
2015-02-06 15:37:27 -06:00
041b047a2d
Make testGetIfUnmodifiedSince a lot more robust (but slower)
2015-02-05 23:26:23 -06:00
a8c5e28129
Windows fixes, mostly separator related.
2015-01-28 19:47:19 -06:00
ca58f627d9
Fixes a problem on windows where the separator is different
2015-01-26 13:52:30 -06:00
2ed971d216
Correct test expectation
...
Fixed by 49d7d9d3b7
2015-01-23 21:07:45 -08:00
af1a01a0af
JCLOUDS-96: Handle us-standard in testAllLocations
2015-01-23 19:24:44 -08:00
49d7d9d3b7
Reorder list option handling for local blobstores
...
We should apply the prefix first, recursive second, marker third, and
max keys last. Also add some fixes to pass delimiter tests.
2015-01-22 11:37:46 -08:00
da3e7463fd
Add more tests for delimiter, prefixes, and marker
...
s3-tests uncovered interactions between these options which existing
tests do not exercise.
2015-01-22 11:37:46 -08:00
3cabb196ce
fix usermetadata check on OS X
...
Java on OS X does not support extended attributes, which the
filesystem blobstore uses to implement user metadata. This disable
the relevant test on OS X
2015-01-20 17:33:22 -08:00
0512fc6e35
Add @Nullable attribute to BlobStore methods
...
blobMetadata and getBlob return null when a blob does not exist and
callers should check for this.
2015-01-16 15:06:16 -08:00
9542fa5d31
Do not check object counts in multi-part tests
...
Some providers, e.g., GCS and Swift, create multi-part blobs with a
manifest pointing to other blobs. Thus a container will have
additional objects in its listing.
2015-01-16 05:45:27 -08:00
8c5391505d
JCLOUDS-805: Allow overriding addContentMetadata
...
For symmetry with checkContentMetadata.
2015-01-16 05:12:00 -08:00
58612bc473
Expand pagination test and enable for Atmos
...
Tested against atmos, aws-s3, azureblob, filesystem,
google-cloud-storage, rackspace-cloudfiles-us, and transient.
2015-01-12 17:43:42 -08:00
fae097e144
JCLOUDS-654: Add object size to StorageMetadata
...
This allows callers to read the content length during container
listing. Tested against: atmosonline, aws-s3, azureblob, filesystem,
and transient. Intentionally not implemented for legacy swift
provider.
2015-01-09 22:34:56 -08:00
91c47bfd92
JCLOUDS-801: Add portable multipart upload tests
...
Exercise both repeatable and non-repeatable payloads. Tested against
AWS-S3, Azure, and legacy Swift. Skipped on all other providers.
2015-01-09 21:59:25 -08:00
b7ab8b18a0
support directory blobs
...
make the filesystem blob store distinguish between a/ and a. a/
must be a directory blob with no content and only metadata
on listing, only directories with md5 attribute is considered a
blob and returned
2015-01-09 16:28:16 -08:00
77eaa2c1f9
Add container creation date to local blobstores
2015-01-09 14:10:33 -08:00
d3c1e2eab4
Use AutoService to generate service loader metadata
2015-01-09 11:22:12 -08:00
d6d06b71f4
Handle listing with marker after the last key
...
Also handle zero maxResults. Found with Ceph s3-tests and s3proxy.
2015-01-08 16:28:02 -08:00
8d0c7f9a6c
JCLOUDS-534 Folded overly explicit use of HostAndPort, which is also incompatible with guava 16.
2014-10-29 17:01:26 -07:00
177fd6d37f
Do not require Guava ByteSource in order to create a payload.
2014-10-29 10:20:39 -07:00
1702bddffa
Revert "Replace uses of ByteArrayPayload"
...
This reverts commit bd5b6853ee
2014-10-29 10:20:38 -07:00
2933a78a6b
Revert "Prefer ByteSource payload"
...
This reverts commit 0e3d1475ab
2014-10-25 10:29:22 -07:00
0a801d2ad2
Revert "JCLOUDS-652: Require JDK 7"
...
This reverts commit 5e8bd02c93
2014-10-25 10:28:34 -07:00
b9525a0877
JCLOUDS-40 unasync Fallback
2014-10-06 13:14:59 -07:00
ffc0df11c9
Restoring BlobStoreUtils.parseDirectoryFromPath
...
Removed in 56a2a8b
2014-10-06 09:50:24 -04:00
99e217b720
Move off confusing test names.
2014-10-05 17:22:18 -07:00
a4e3c1a2f8
JCLOUDS-40 Remove AsyncBlobStore
2014-10-05 13:17:54 -07:00
56a2a8bf4a
JCLOUDS-40 Remove internal usage of AsyncBlobStore.
2014-10-05 08:49:54 -07:00
dfb583b67a
JCLOUDS-40 remove all implementations of AsyncBlobStore except Submission in preparation for complete removal.
2014-10-05 08:49:38 -07:00
b6497556f6
JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3
2014-10-03 13:18:29 -07:00
0e3d1475ab
Prefer ByteSource payload
...
Also use randomByteSource for test input.
2014-09-09 15:32:39 -07:00
bd5b6853ee
Replace uses of ByteArrayPayload
...
Prefer ByteSourcePayload which offers a superset of its functionality.
Note that ByteArrayPayload implicitly set the contentLength while
users of ByteSourcePayload must do so explicitly.
2014-09-07 07:43:14 -07:00
0b89b1fd10
Ensure that Content-MD5 has 128 bits
...
Update many tests Content-MD5 headers and ETag responses.
2014-09-04 16:57:48 -07:00
34cfdbe266
Enforce Override annotation via Checkstyle
...
Also remove bogus inheritDoc on base and static methods.
2014-08-31 19:37:50 -07:00
550018a68d
Use randomByteSource to generate test data
...
This avoids buffering the entire bolus of data in-memory.
2014-08-27 15:59:36 -07:00
60e3763999
Throw ContainerNotFoundException from removeBlob
...
This makes removeBlob consistent with all other user-facing methods.
2014-08-21 16:18:12 -07:00
e711275fb1
Prefer isEmpty() for collections rather than size()
2014-08-20 11:07:48 -07:00
e5283df995
JCLOUDS-676: Fix reflection warnings / Clojure
...
JCLOUDS-676: Fix reflection warnings / Clojure
Also fixed a typo in a doc string
2014-08-17 12:03:30 -07:00
20bc145284
JCLOUDS-650: ContainerNotFoundException on putBlob
...
Found with Ceph s3-tests and s3proxy.
2014-08-09 00:25:10 -07:00
e3d9851216
Up to 2.0.0-SNAPSHOT after the 1.8.0 release
2014-08-05 09:10:24 +02:00
Timur Alperovich
Andrew Gaul
Ka-Hing Cheung
Iván López
Zack Shoylev
Ignasi Barrera
Jeremy Daggett
Adrian Cole
Adrian Cole
Andrew Phillips
John Walker