527 Commits

Author SHA1 Message Date
Andrew Gaul
49d7d9d3b7 Reorder list option handling for local blobstores
We should apply the prefix first, recursive second, marker third, and
max keys last.  Also add some fixes to pass delimiter tests.
2015-01-22 11:37:46 -08:00
Andrew Gaul
da3e7463fd Add more tests for delimiter, prefixes, and marker
s3-tests uncovered interactions between these options which existing
tests do not exercise.
2015-01-22 11:37:46 -08:00
Ka-Hing Cheung
3cabb196ce fix usermetadata check on OS X
Java on OS X does not support extended attributes, which the
filesystem blobstore uses to implement user metadata. This disable
the relevant test on OS X
2015-01-20 17:33:22 -08:00
Andrew Gaul
0512fc6e35 Add @Nullable attribute to BlobStore methods
blobMetadata and getBlob return null when a blob does not exist and
callers should check for this.
2015-01-16 15:06:16 -08:00
Andrew Gaul
9542fa5d31 Do not check object counts in multi-part tests
Some providers, e.g., GCS and Swift, create multi-part blobs with a
manifest pointing to other blobs.  Thus a container will have
additional objects in its listing.
2015-01-16 05:45:27 -08:00
Andrew Gaul
8c5391505d JCLOUDS-805: Allow overriding addContentMetadata
For symmetry with checkContentMetadata.
2015-01-16 05:12:00 -08:00
Andrew Gaul
58612bc473 Expand pagination test and enable for Atmos
Tested against atmos, aws-s3, azureblob, filesystem,
google-cloud-storage, rackspace-cloudfiles-us, and transient.
2015-01-12 17:43:42 -08:00
Andrew Gaul
fae097e144 JCLOUDS-654: Add object size to StorageMetadata
This allows callers to read the content length during container
listing.  Tested against: atmosonline, aws-s3, azureblob, filesystem,
and transient.  Intentionally not implemented for legacy swift
provider.
2015-01-09 22:34:56 -08:00
Andrew Gaul
91c47bfd92 JCLOUDS-801: Add portable multipart upload tests
Exercise both repeatable and non-repeatable payloads.  Tested against
AWS-S3, Azure, and legacy Swift.  Skipped on all other providers.
2015-01-09 21:59:25 -08:00
Ka-Hing Cheung
b7ab8b18a0 support directory blobs
make the filesystem blob store distinguish between a/ and a. a/
must be a directory blob with no content and only metadata

on listing, only directories with md5 attribute is considered a
blob and returned
2015-01-09 16:28:16 -08:00
Andrew Gaul
77eaa2c1f9 Add container creation date to local blobstores 2015-01-09 14:10:33 -08:00
Jeremy Daggett
d3c1e2eab4 Use AutoService to generate service loader metadata 2015-01-09 11:22:12 -08:00
Andrew Gaul
d6d06b71f4 Handle listing with marker after the last key
Also handle zero maxResults.  Found with Ceph s3-tests and s3proxy.
2015-01-08 16:28:02 -08:00
Adrian Cole
8d0c7f9a6c JCLOUDS-534 Folded overly explicit use of HostAndPort, which is also incompatible with guava 16. 2014-10-29 17:01:26 -07:00
Adrian Cole
177fd6d37f Do not require Guava ByteSource in order to create a payload. 2014-10-29 10:20:39 -07:00
Adrian Cole
1702bddffa Revert "Replace uses of ByteArrayPayload"
This reverts commit bd5b6853ee4d2397f49e27d68818b01123fa7bd2.
2014-10-29 10:20:38 -07:00
Adrian Cole
2933a78a6b Revert "Prefer ByteSource payload"
This reverts commit 0e3d1475ab50e1232d5eac16eee1d3a207e36f42.
2014-10-25 10:29:22 -07:00
Adrian Cole
0a801d2ad2 Revert "JCLOUDS-652: Require JDK 7"
This reverts commit 5e8bd02c93c7ebacb573342e77ef7ca4b02c4ffb.
2014-10-25 10:28:34 -07:00
Adrian Cole
b9525a0877 JCLOUDS-40 unasync Fallback 2014-10-06 13:14:59 -07:00
Andrew Phillips
ffc0df11c9 Restoring BlobStoreUtils.parseDirectoryFromPath
Removed in 56a2a8b but used in downstream repos, e.g. jclouds-karaf
2014-10-06 09:50:24 -04:00
Adrian Cole
99e217b720 Move off confusing test names. 2014-10-05 17:22:18 -07:00
Adrian Cole
a4e3c1a2f8 JCLOUDS-40 Remove AsyncBlobStore 2014-10-05 13:17:54 -07:00
Adrian Cole
56a2a8bf4a JCLOUDS-40 Remove internal usage of AsyncBlobStore. 2014-10-05 08:49:54 -07:00
Adrian Cole
dfb583b67a JCLOUDS-40 remove all implementations of AsyncBlobStore except Submission in preparation for complete removal. 2014-10-05 08:49:38 -07:00
Adrian Cole
b6497556f6 JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 2014-10-03 13:18:29 -07:00
Andrew Gaul
0e3d1475ab Prefer ByteSource payload
Also use randomByteSource for test input.
2014-09-09 15:32:39 -07:00
Andrew Gaul
bd5b6853ee Replace uses of ByteArrayPayload
Prefer ByteSourcePayload which offers a superset of its functionality.
Note that ByteArrayPayload implicitly set the contentLength while
users of ByteSourcePayload must do so explicitly.
2014-09-07 07:43:14 -07:00
Andrew Gaul
0b89b1fd10 Ensure that Content-MD5 has 128 bits
Update many tests Content-MD5 headers and ETag responses.
2014-09-04 16:57:48 -07:00
Andrew Gaul
34cfdbe266 Enforce Override annotation via Checkstyle
Also remove bogus inheritDoc on base and static methods.
2014-08-31 19:37:50 -07:00
Andrew Gaul
550018a68d Use randomByteSource to generate test data
This avoids buffering the entire bolus of data in-memory.
2014-08-27 15:59:36 -07:00
Andrew Gaul
60e3763999 Throw ContainerNotFoundException from removeBlob
This makes removeBlob consistent with all other user-facing methods.
2014-08-21 16:18:12 -07:00
Jeremy Daggett
e711275fb1 Prefer isEmpty() for collections rather than size() 2014-08-20 11:07:48 -07:00
John Walker
e5283df995 JCLOUDS-676: Fix reflection warnings / Clojure
JCLOUDS-676: Fix reflection warnings / Clojure

Also fixed a typo in a doc string
2014-08-17 12:03:30 -07:00
Andrew Gaul
20bc145284 JCLOUDS-650: ContainerNotFoundException on putBlob
Found with Ceph s3-tests and s3proxy.
2014-08-09 00:25:10 -07:00
Andrew Phillips
e3d9851216 Up to 2.0.0-SNAPSHOT after the 1.8.0 release 2014-08-05 09:10:24 +02:00
Andrew Gaul
9c82ded343 Correct comment 2014-07-27 21:25:35 -07:00
Andrew Gaul
7f31cd6051 Handle possibly null message from IOException 2014-07-27 19:41:45 -07:00
Andrew Gaul
1e1eb5a092 Add deleteContainerIfEmpty to BlobStore
This matches how most blobstores operate: delete container is a single
operation, not a compound operation which recursively deletes blobs.
Azure is the only provider which allows deleting a non-empty
container.
2014-07-25 23:02:39 -07:00
Andrew Phillips
5732bdcd94 Escaping regex special chars (esp. '\') in delimiter matching
Uncovered by 370194b
2014-07-22 13:35:52 -04:00
Andrew Gaul
f17c876d8d Replace const.txt file input with synthetic inputs
This commit replaces file resource-based test inputs with in-memory
equivalents.  This is more consistent and efficient than the previous
approach.  Also resized some test inputs to be partSize + 1 instead of
2 * partSize.  Tested against aws-s3, blobstore, core, cloudfiles-us,
and filesystem.
2014-07-19 18:35:25 -07:00
Andrew Gaul
dd3dc9790e Close streams in integration tests
Also remove bogus delete workaround.  Previously unclosed
FileInputStream caused test failures on Windows which cannot delete
open files.  Found with Kohsuke's file-leak-detector.
2014-07-19 05:25:20 -07:00
Andrew Gaul
fb60d76704 JCLOUDS-622: Remove most vestiges of InputSupplier
Guava 16 deprecated InputSupplier and a future release will remove it.
2014-07-16 16:07:59 -07:00
Andrew Gaul
a8b106c2bb JCLOUDS-622: remove calls to InputSupplier methods
Guava 16 deprecated these methods and Guava 18 will remove them.
2014-07-15 03:09:58 -07:00
Andrew Gaul
75928795f3 Address Checkstyle violations 2014-07-15 03:07:29 -07:00
Andrew Gaul
bdb5cbcbe7 Improve use of ByteSource with test resources
This commit ensures proper resource cleanup, simplifies
initialization, and paves the way for purely synthetic inputs in a
subsequent commit.
2014-07-11 09:10:51 -07:00
Andrew Gaul
cf830af4d2 Remove unneeded InputSupplier helper
Also convert byte[] to HashCode.
2014-07-10 22:36:32 -07:00
Andrew Gaul
f4eca0422d Enforce correct MD5 for local blobstores
Matches behavior of real blobstores.
2014-07-10 22:19:42 -07:00
Andrew Gaul
66dab8d96e JCLOUDS-410: Remove InputStreamSupplierPayload
ByteSourcePayload replaces this.  jclouds 1.7 deprecated
InputStreamSupplierPayload and Guava 16 deprecated InputSupplier
itself.
2014-07-10 20:43:08 -07:00
Andrew Gaul
8fa209b15e Use more specific testng asserts where possible
These asserts yield more informative error messages.  Found with:

grep -rI 'assertTrue(.* =='
grep -rI 'assertTrue(.* !='
grep -rI 'assertTrue(.*\.equals('
2014-06-24 13:54:01 -07:00
Chris Custine
309c053c77 JCLOUDS-584: HP Cloud Object Storage Fixes for 13.5 2014-06-24 12:01:51 -07:00