Commit Graph

677 Commits

Author SHA1 Message Date
Andrew Gaul 95c4011982 Include part size in listMultipartUpload result
Clients use this during completeMultipartUpload.
2016-11-15 20:30:50 -08:00
Ignasi Barrera d290705419 Next development version 2.1.0-SNAPSHOT 2016-11-15 01:12:59 +01:00
Ignasi Barrera 6a3a37f4e0 Apache jclouds 2.0.0-rc3 release 2016-11-09 22:30:45 +01:00
Andrew Gaul 69aa5d6426 JCLOUDS-1065: Deprecate fake directory support
Superseded by prefix and delimiter support.
2016-10-26 21:30:52 -07:00
Andrew Gaul 3c9f66b8e9 Avoid lower-case l literal suffix
Readers can confuse this with 1.  Found via error-prone.  Fixed via:

find -name \*.java | xargs sed -i 's/\( [0-9][0-9]*\)l/\1L/g'
find -name \*.java | xargs sed -i 's/\(([0-9][0-9]*\)l/\1L/g'
2016-10-23 14:24:35 -07:00
Shri Javadekar 7826d22d30 JCLOUDS-1161: Make AWSS3BlobRequestSignerV4 the default signer.
Added new live tests and fixed some unit tests accordingly.
2016-10-18 20:06:34 -07:00
Andrew Gaul 50e6d44433 JCLOUDS-945: fix local blobstore marker handling
Previously using prefix markers would not correctly find the next key.
Add test for this behavior.
2016-10-12 22:22:28 -07:00
Andrew Gaul 3b42bafe25 Add streaming methods to ForwardingBlobStore
Follow on to 05c05e3de2.
2016-09-04 22:16:40 -07:00
Zack Shoylev 05c05e3de2 Allows users to download large files efficiently and directly to disk. 2016-09-02 14:05:26 -05:00
Zack Shoylev 42079e1392 Parallel upload for BaseBlobStore 2016-07-15 04:15:17 -05:00
Andrew Gaul 4874a1eb18 JCLOUDS-1137: Handle TOCTOU during blobMetadata
A similar issue exists when getting a blob payload when a caller
simultaneously removes the blob.
2016-07-11 22:02:15 -07:00
Zack Shoylev 6bff97b6d3 Changes the upload behavior to parallel, a TODO 2016-07-01 11:26:56 -05:00
Andrew Gaul 984b6ae8fb Handle null ETag in copyBlob and conditional get
Mac OS X and Docker aufs do not support xattr and thus have null
ETags.  Fixes andrewgaul/s3proxy#143.
2016-06-28 09:55:53 -07:00
Andrew Gaul 6df0472ab5 JCLOUDS-1125: local blobstore list MPUs 2016-06-23 21:45:04 -07:00
Andrew Gaul a845471b88 Create Payload per thread in testPutBlobParallel
Fixes a regression from 45bcc3ce22.
2016-06-23 05:26:21 -07:00
Andrew Gaul 45bcc3ce22 Use in-memory payload for testPutBlobParallel
Also simplify parts of the test.
2016-06-22 22:50:22 -07:00
Andrew Gaul de04c69141 Consistently name test container names
Previously jclouds allocated the initial container pool as
$USERNAME-blobstore-1, 2, etc. and subsequent containers with
$USERNAME-blobstore-$RANDOM.  Use only the former instead.
2016-06-22 22:50:22 -07:00
Andrew Gaul 7a1c3a7b06 Remove listed multipart parts in abort
Previously we unconditionally removed all possible part names.
2016-06-21 10:01:00 -07:00
Andrew Gaul 04c2394a10 Annotate @BeforeSuite with groups as @AfterSuite
Previously this caused unexpected failures in the B2 integration
tests.
2016-06-16 12:57:38 -07:00
Andrew Gaul 52de3b5766 JCLOUDS-1125: Upload two parts in list MPU test
This allows B2 to call complete multipart upload.
2016-06-13 17:00:50 -07:00
Andrew Gaul 0bd2959410 JCLOUDS-1125: portable list multipart uploads
Only Azure, B2, and S3 support this operation.  Some MultipartUpload
fields become nullable.
2016-06-13 16:26:43 -07:00
Andrew Gaul ed7ff648e8 Ensure MPU input size meets minimum for test 2016-06-02 11:27:21 -07:00
Andrew Gaul d983d8d0a7 Revert "Handle when total length is less than part length"
This reverts commit bc1f12b7bd.
2016-06-02 07:21:48 -07:00
Andrew Gaul bc1f12b7bd Handle when total length is less than part length 2016-06-01 22:01:38 -07:00
Andrew Gaul 63bfc2f6e8 Use prefix option in prefix test
Directory is deprecated.
2016-06-01 17:44:43 -07:00
Andrew Gaul 421764dfd1 Remove disposition checks from testFileGetParallel
Some providers like B2 do not support Content-Disposition and other
tests exercise this functionality.
2016-06-01 16:15:47 -07:00
Andrew Gaul 5a85f5b5f9 Use provided minimum part size in MPU tests 2016-06-01 16:15:15 -07:00
Andrew Gaul 1ac73d3093 Ensure MPU has a minimum size 2016-06-01 16:14:58 -07:00
Andrew Gaul 209152e30a Abort multipart upload on putMultipartBlob error
This avoids dangling incomplete MPU.
2016-06-01 13:34:53 -07:00
Andrew Gaul bf7d864c41 Allow null ETag in MultipartPart
Some storage backends, notably filesystem provider using either
Mac OS X or NFS, do not support extended attributes and cannot store
the ETag.  References andrewgaul/s3proxy#135.
2016-06-01 12:47:02 -07:00
Andrew Gaul fb186d82de Consistently compare quoted ETags in CopyBlob 2016-05-28 12:17:28 -07:00
Andrew Gaul 00a36c3e45 JCLOUDS-654: Add size to BlobMetadata constructor
Follow on to fae097e144.
2016-05-27 17:27:06 -07:00
Andrew Gaul cd50ad905d Avoid NPE in BaseBlobStore.copyBlob
Resolves copyBlob failures in Atmos.
2016-05-12 12:48:07 -07:00
Andrew Gaul 62410d31ae Handle missing containers in LocalBlobStore.list
Also fix FilesystemStorageStrategyImpl.getContainerMetadata to return
null on missing container as jdbc and transient already do.
2016-05-06 21:36:28 -07:00
Andrew Gaul a4c40e15cd Handle unimplemented signed URL tests in providers
This commit makes it evident in source code which providers do not
support this feature.
2016-05-06 10:31:37 -07:00
Andrew Gaul 35f4f2bbf1 Use more unique names in containerDoesntExist
Previously this failed on AWS due to another container named
"forgetaboutit".
2016-03-27 13:48:42 -07:00
Andrew Gaul 2638ba4e69 Consistently quote ETags during comparison
Regression from e0a7ea7fdf.
2016-02-16 23:10:16 -08:00
Andrew Gaul 66609e6d70 JCLOUDS-651: Local blobstore support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul 8945258d79 JCLOUDS-651: Portable support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul 293d3f864e Convert CopyOptions into an AutoValue
This commit requires an interface change since AutoValue lacks support
for Optional and uses Nullable annotations instead.
2016-02-16 16:29:54 -08:00
Andrew Gaul a697396e8c Correctly remove prefix in DelimiterFilter
Previously this called String.replaceFirst which uses a regular
expression and incorrectly handles characters like *.  Also remove
other correct but unnecessary call to String.replaceFirst.
2016-02-09 10:48:01 -08:00
Andrew Gaul de333e8b3c Check Content-Length in local blobstore putBlob 2016-02-03 22:03:48 -08:00
quod3 77eef902b4 Use parseLong instead of parseInt in range parser
Use parseLong instead of parseInt when parsing open-ended byte ranges in LocalBlobStore. Without this fix, any "from byte x" or "to byte x" getBlob() call will throw a NumberFormatException if x is too big to fit into an int (2 GB).

Fixes https://issues.apache.org/jira/browse/JCLOUDS-1073
2016-01-30 20:11:07 -05:00
Andrew Gaul eb6f16e2f9 Enable CollectionIncompatibleType
Also suppress two false positives.
2016-01-29 08:50:09 -08:00
Andrew Gaul b6920ca44b JCLOUDS-1065: Deprecate inDirectory option
Superseded by prefix and delimiter support.
2016-01-28 22:59:12 -08:00
Andrew Gaul a3376d4efe Consistently implement domain object methods
Including equals, hashCode, and toString.
2016-01-24 21:58:39 -08:00
Andrew Gaul b50c518f7e Avoid sleeping in signed URL tests
Also bump expiration to a consistent 60 seconds.
2016-01-23 13:48:43 -08:00
Andrew Gaul b2c0786fc0 Enable range tests for azureblob 2016-01-23 12:38:50 -08:00
Andrew Gaul 52dc1a3ccf Add BaseBlobIntegrationTest.testSetBlobAccess
Similar to BaseContainerIntegrationTest.testSetContainerAccess.
2016-01-19 14:40:55 -08:00
Andrew Gaul 688890819e Fix replacing metadata in BaseBlobStore
Fixes Atmos errors.  Uncovered by removing unneeded addContentMetadata
call in BaseBlobIntegrationTest.testCopyBlobReplaceMetadata.
2016-01-15 13:22:15 -08:00