8b94febfeb
Throw consistent exception on invalid range read
...
Previously S3, Swift, and local blobstores threw a generic
IllegalArgumentException for this uncommon error. Instead
consistently throw HttpResponseException.
2017-11-10 21:39:47 -08:00
343897d6d8
Use different content on overwrite.
...
When testing blobstore overwrite behavior, jclouds should use a blob
with different content (but same name).
2017-11-06 17:22:32 -08:00
83d10e8194
JCLOUDS-1337: restore tier from stub blob
...
This allows completeMultipartUpload to propagate the tier set in
initiateMultipartUpload.
2017-11-03 09:52:42 -07:00
7fbef10d57
JCLOUDS-1337: Include tier in object listing
...
This requires hoisting Tier from BlobMetadata to StorageMetadata.
2017-11-02 16:24:34 -07:00
9a0b51a4d4
Include trailing slash in prefix test
2017-10-30 18:31:40 -07:00
af05e24ae9
Implement partial prefix support for Atmos
...
Atmos only supports listing by directories while other blobstores
allow listing via arbitrary prefixes. Allow requests which list
directories via both prefix and delimiter = "/" to succeed instead of
failing all requests. Also change a test which specified recursive to
instead be delimiter = "/". Fixes gaul/s3proxy#244 .
2017-10-30 17:51:35 -07:00
f7b74d95c9
Exclude tier tests on OSX
2017-10-25 08:30:49 +02:00
f513bf7c40
JCLOUDS-1337: Atmos putBlob portable storage tiers
2017-10-23 21:31:48 -07:00
6158b60954
JCLOUDS-1337: Portable storage tiers
2017-10-11 12:04:02 -07:00
7704893650
Remove Inject on constructors for abstract classes
...
This is not meaningful since these classes cannot be instantiated.
Found via error-prone.
2017-09-07 23:37:24 -07:00
517323fbd5
JCLOUDS-1225: Address Guava 18 MoreExecutors changes
...
Fixed with:
find -name \*.java | xargs sed -i 's/sameThreadExecutor/newDirectExecutorService/g'
2017-08-22 17:14:22 -07:00
a493e2ca2f
JCLOUDS-1225: Address Guava 18 Objects changes
...
Fixed with:
find -name \*.java | xargs sed -i 's/Objects.[Tt]oStringHelper/More&/g'
find -name \*.java | xargs sed -i 's/Objects.firstNonNull/More&/g'
find -name \*.java | xargs sed -i 's/^\(import com.google.common.base.\)\(Objects.*\)/\1More\2\n\1\2/g'
find -name \*.java | xargs java -jar google-java-format-1.3-all-deps.jar -i --fix-imports-only --skip-sorting-imports
2017-08-22 17:12:53 -07:00
d6038487f7
JCLOUDS-1327: Add tests for zero-length blobs
2017-08-06 13:51:07 -07:00
d07c4a215e
Handle empty delimiter/prefix in FS store.
...
When delimiter/prefix is an empty string, jclouds filesystem blobstore
should treat them as not being set.
2017-08-03 14:37:12 -07:00
0bc935dd57
Remove clojure bindings
...
These have not seen any development in many years.
2017-07-10 11:39:11 -07:00
e446b5b8b4
JCLOUDS-1111: Force overwriting Atmos objects
...
This avoids a racy delete then write in the portable abstraction.
2017-07-08 15:50:04 -07:00
2b49bf3618
JCLOUDS-1299: Handle % in names during Swift list
...
uriBuilder.appendPath assumes an encoded path.
2017-05-16 17:29:02 -07:00
2e7de9f850
JCLOUDS-1271: Deprecate signRemoveBlob
...
URL signing should always use authentication parameters instead of
headers yet no provider supports DELETEs with the former.
2017-04-19 18:08:52 -07:00
97643823fc
Emit marker in PageSet toString
2017-03-18 18:55:12 -07:00
e058973abc
Ensure SOME_CONSTANTS are static final
...
Found via error-prone.
2017-03-17 16:59:41 -07:00
87eda15085
Return more metadata from listMultipartUpload
...
Include Content-Length and Last-Modified.
2017-03-08 20:35:24 -08:00
95c4011982
Include part size in listMultipartUpload result
...
Clients use this during completeMultipartUpload.
2016-11-15 20:30:50 -08:00
d290705419
Next development version 2.1.0-SNAPSHOT
2016-11-15 01:12:59 +01:00
6a3a37f4e0
Apache jclouds 2.0.0-rc3 release
2016-11-09 22:30:45 +01:00
69aa5d6426
JCLOUDS-1065: Deprecate fake directory support
...
Superseded by prefix and delimiter support.
2016-10-26 21:30:52 -07:00
3c9f66b8e9
Avoid lower-case l literal suffix
...
Readers can confuse this with 1. Found via error-prone. Fixed via:
find -name \*.java | xargs sed -i 's/\( [0-9][0-9]*\)l/\1L/g'
find -name \*.java | xargs sed -i 's/\(([0-9][0-9]*\)l/\1L/g'
2016-10-23 14:24:35 -07:00
7826d22d30
JCLOUDS-1161: Make AWSS3BlobRequestSignerV4 the default signer.
...
Added new live tests and fixed some unit tests accordingly.
2016-10-18 20:06:34 -07:00
50e6d44433
JCLOUDS-945: fix local blobstore marker handling
...
Previously using prefix markers would not correctly find the next key.
Add test for this behavior.
2016-10-12 22:22:28 -07:00
3b42bafe25
Add streaming methods to ForwardingBlobStore
...
Follow on to 05c05e3de2
2016-09-04 22:16:40 -07:00
05c05e3de2
Allows users to download large files efficiently and directly to disk.
2016-09-02 14:05:26 -05:00
42079e1392
Parallel upload for BaseBlobStore
2016-07-15 04:15:17 -05:00
4874a1eb18
JCLOUDS-1137: Handle TOCTOU during blobMetadata
...
A similar issue exists when getting a blob payload when a caller
simultaneously removes the blob.
2016-07-11 22:02:15 -07:00
6bff97b6d3
Changes the upload behavior to parallel, a TODO
2016-07-01 11:26:56 -05:00
984b6ae8fb
Handle null ETag in copyBlob and conditional get
...
Mac OS X and Docker aufs do not support xattr and thus have null
ETags. Fixes andrewgaul/s3proxy#143 .
2016-06-28 09:55:53 -07:00
6df0472ab5
JCLOUDS-1125: local blobstore list MPUs
2016-06-23 21:45:04 -07:00
a845471b88
Create Payload per thread in testPutBlobParallel
...
Fixes a regression from 45bcc3ce22
2016-06-23 05:26:21 -07:00
45bcc3ce22
Use in-memory payload for testPutBlobParallel
...
Also simplify parts of the test.
2016-06-22 22:50:22 -07:00
de04c69141
Consistently name test container names
...
Previously jclouds allocated the initial container pool as
$USERNAME-blobstore-1, 2, etc. and subsequent containers with
$USERNAME-blobstore-$RANDOM. Use only the former instead.
2016-06-22 22:50:22 -07:00
7a1c3a7b06
Remove listed multipart parts in abort
...
Previously we unconditionally removed all possible part names.
2016-06-21 10:01:00 -07:00
04c2394a10
Annotate @BeforeSuite with groups as @AfterSuite
...
Previously this caused unexpected failures in the B2 integration
tests.
2016-06-16 12:57:38 -07:00
52de3b5766
JCLOUDS-1125: Upload two parts in list MPU test
...
This allows B2 to call complete multipart upload.
2016-06-13 17:00:50 -07:00
0bd2959410
JCLOUDS-1125: portable list multipart uploads
...
Only Azure, B2, and S3 support this operation. Some MultipartUpload
fields become nullable.
2016-06-13 16:26:43 -07:00
ed7ff648e8
Ensure MPU input size meets minimum for test
2016-06-02 11:27:21 -07:00
d983d8d0a7
Revert "Handle when total length is less than part length"
...
This reverts commit bc1f12b7bd
2016-06-02 07:21:48 -07:00
bc1f12b7bd
Handle when total length is less than part length
2016-06-01 22:01:38 -07:00
63bfc2f6e8
Use prefix option in prefix test
...
Directory is deprecated.
2016-06-01 17:44:43 -07:00
421764dfd1
Remove disposition checks from testFileGetParallel
...
Some providers like B2 do not support Content-Disposition and other
tests exercise this functionality.
2016-06-01 16:15:47 -07:00
5a85f5b5f9
Use provided minimum part size in MPU tests
2016-06-01 16:15:15 -07:00
1ac73d3093
Ensure MPU has a minimum size
2016-06-01 16:14:58 -07:00
209152e30a
Abort multipart upload on putMultipartBlob error
...
This avoids dangling incomplete MPU.
2016-06-01 13:34:53 -07:00
bf7d864c41
Allow null ETag in MultipartPart
...
Some storage backends, notably filesystem provider using either
Mac OS X or NFS, do not support extended attributes and cannot store
the ETag. References andrewgaul/s3proxy#135 .
2016-06-01 12:47:02 -07:00
fb186d82de
Consistently compare quoted ETags in CopyBlob
2016-05-28 12:17:28 -07:00
00a36c3e45
JCLOUDS-654: Add size to BlobMetadata constructor
...
Follow on to fae097e144
2016-05-27 17:27:06 -07:00
cd50ad905d
Avoid NPE in BaseBlobStore.copyBlob
...
Resolves copyBlob failures in Atmos.
2016-05-12 12:48:07 -07:00
62410d31ae
Handle missing containers in LocalBlobStore.list
...
Also fix FilesystemStorageStrategyImpl.getContainerMetadata to return
null on missing container as jdbc and transient already do.
2016-05-06 21:36:28 -07:00
a4c40e15cd
Handle unimplemented signed URL tests in providers
...
This commit makes it evident in source code which providers do not
support this feature.
2016-05-06 10:31:37 -07:00
35f4f2bbf1
Use more unique names in containerDoesntExist
...
Previously this failed on AWS due to another container named
"forgetaboutit".
2016-03-27 13:48:42 -07:00
2638ba4e69
Consistently quote ETags during comparison
...
Regression from e0a7ea7fdf
2016-02-16 23:10:16 -08:00
66609e6d70
JCLOUDS-651: Local blobstore support for conditional copies
2016-02-16 16:29:54 -08:00
8945258d79
JCLOUDS-651: Portable support for conditional copies
2016-02-16 16:29:54 -08:00
293d3f864e
Convert CopyOptions into an AutoValue
...
This commit requires an interface change since AutoValue lacks support
for Optional and uses Nullable annotations instead.
2016-02-16 16:29:54 -08:00
a697396e8c
Correctly remove prefix in DelimiterFilter
...
Previously this called String.replaceFirst which uses a regular
expression and incorrectly handles characters like *. Also remove
other correct but unnecessary call to String.replaceFirst.
2016-02-09 10:48:01 -08:00
de333e8b3c
Check Content-Length in local blobstore putBlob
2016-02-03 22:03:48 -08:00
77eef902b4
Use parseLong instead of parseInt in range parser
...
Use parseLong instead of parseInt when parsing open-ended byte ranges in LocalBlobStore. Without this fix, any "from byte x" or "to byte x" getBlob() call will throw a NumberFormatException if x is too big to fit into an int (2 GB).
Fixes https://issues.apache.org/jira/browse/JCLOUDS-1073
2016-01-30 20:11:07 -05:00
eb6f16e2f9
Enable CollectionIncompatibleType
...
Also suppress two false positives.
2016-01-29 08:50:09 -08:00
b6920ca44b
JCLOUDS-1065: Deprecate inDirectory option
...
Superseded by prefix and delimiter support.
2016-01-28 22:59:12 -08:00
a3376d4efe
Consistently implement domain object methods
...
Including equals, hashCode, and toString.
2016-01-24 21:58:39 -08:00
b50c518f7e
Avoid sleeping in signed URL tests
...
Also bump expiration to a consistent 60 seconds.
2016-01-23 13:48:43 -08:00
b2c0786fc0
Enable range tests for azureblob
2016-01-23 12:38:50 -08:00
52dc1a3ccf
Add BaseBlobIntegrationTest.testSetBlobAccess
...
Similar to BaseContainerIntegrationTest.testSetContainerAccess.
2016-01-19 14:40:55 -08:00
688890819e
Fix replacing metadata in BaseBlobStore
...
Fixes Atmos errors. Uncovered by removing unneeded addContentMetadata
call in BaseBlobIntegrationTest.testCopyBlobReplaceMetadata.
2016-01-15 13:22:15 -08:00
5fd7b1aa78
JCLOUDS-948: Portable Cache-Control support
2016-01-15 04:41:01 -08:00
a90245afe5
Add marker and prefix to equals and toString
2016-01-11 19:18:14 -08:00
c0d72b6f53
JCLOUDS-844: Local blobstore putBlob ACLs
2015-12-22 00:46:33 -08:00
86491bc607
JCLOUDS-844: Portable putBlob ACLs
2015-12-22 00:46:33 -08:00
37f307ecd0
Skip testSetContainerAccess for local blobstores
...
These lack a public HTTP endpoint needed for this test.
2015-12-20 01:14:33 -08:00
13701bf7ab
Do not test marker contents
...
Some providers like Azure have opaque markers.
2015-12-19 15:24:12 -08:00
a7c66d5120
JCLOUDS-1042: Test reading from public container
...
Blob has default access but container is public-read.
2015-12-18 23:28:39 -08:00
4abd5dd16f
Revert "Delete parts when removing Swift multipart objects"
...
This reverts commit 3c1588527d
2015-12-08 10:08:22 +08:00
2dd231b02b
Filter out folders when listing a container
...
These folders represent directories in the filesystem blobstore which
real blobstores do not have. Their presence complicates providing the
correct marker for paginated listing. Future commits will remove
folders entirely.
2015-11-20 16:27:53 -08:00
4ce357ba69
Consistently quote ETags in comparisons
...
Addresses integration test regression in
25f4807df8
2015-11-19 19:00:34 -08:00
25f4807df8
Tolerate missing or extra quotes in ETags
...
References andrewgaul/s3proxy#77 .
2015-11-19 17:40:48 -08:00
8d87bfc61b
Avoid buffering ByteSource when not needed
...
This allows range requests of large files to work. Regression from
79fe91bd03
2015-11-19 15:10:23 -08:00
3c1588527d
Delete parts when removing Swift multipart objects
...
Swift implements multi-part upload with user-visible parts and an
explicit manifest. When deleting an MPU blob it can delete only the
manifest or both the manifest and parts. For consistency with other
providers, we now do the latter in the portable abstraction. Swift
ignores the multipart-manifest=delete parameter for single-part
objects. Fixes andrewgaul/s3proxy#92 .
2015-11-17 23:06:33 -08:00
41ce90ec36
Fixes tests failing on windows filesystems Makes windows behavior more consistent, especially for deletes
2015-11-11 21:01:13 -06:00
e0c959c212
Pagination for listing LocalBlobStore MPU parts
2015-10-20 14:01:32 -07:00
aaa164179c
JCLOUDS-217: Percent encoding changes.
...
jclouds should not check if the string is encoded, but rather expect
that all strings would be encoded prior to transmission. As part of
that change, we must make sure that no code relies on such behavior
within jclouds. This commit adds a blobstore test to check encoding
pattern for blobs. It also removes the encoding check in the Strings2
class and the related test.
2015-09-08 15:12:14 -07:00
edde721937
JCLOUDS-962: Make all @Provider methods final
2015-09-02 23:54:44 +02:00
3757a64abf
JCLOUDS-992: LocalBlobStore - use FOLDER for DIR.
...
Changed the LocalBlobStore to use FOLDER, as opposed to RELATIVE_PATH
when creating directories. Changed the delete keys strategy to treat
FOLDER blobs as regular blobs.
2015-08-20 12:42:11 -07:00
0c5a3db9df
JCLOUDS-992: remove special directory handling.
2015-08-20 12:42:11 -07:00
42bfeccdfd
Remove last marker test from ListContainerMarker.
...
Azure does not allow arbitrary marker specification and the generic
testListContainerMarker test should not assume that the behavior is
possible. There is already a separate test for such scenario that is
skipped on Azure.
2015-08-19 16:03:33 -07:00
53c3a4918a
Fix LocalBlobStore
2015-08-06 09:43:33 -05:00
79fe91bd03
JCLOUDS-979: Fix range get for LocalBlobStore
2015-08-04 11:40:57 +02:00
86f47394bf
Correct FindBugs violations
2015-07-31 14:53:58 -07:00
7e6284dd86
Add missing equals and hashCode methods
...
Possible cause of odd PageSet behavior.
2015-07-31 14:17:41 -07:00
5ad245dea5
Do not buffer range get in LocalBlobStore
...
This avoids OutOfMemoryError with large blobs and enables offsets over
2 GB.
2015-07-27 18:02:54 -07:00
2c578d7bcb
marker should be null when there are no more items
2015-07-21 15:46:58 -07:00
19d54369db
Propagate delimiter changes to LocalBlobStore.
...
Propagates the changes to add delimiter support to the local blob
store. The prefix option no longer works with the non-recursive
option. The caller _must_ set the delimiter option to enable the same
functionality.
2015-07-14 14:11:34 -07:00
e4161a59bb
Move tests to test the file system API.
...
Move the prefix and delimiter tests from BaseContainerLiveTest to
BaseContainerIntegrationTest, as that class is subclassed in
Filesystem integration tests. This will make sure the file system blob
store will not regress.
The ListContainerTest is updated to _not_ use a file separator
character, as that leads to a possible creation of directories.
Instead, "-" is used as the delimiter for the delimiter option tests.
2015-07-14 14:11:34 -07:00
613b6bf045
Add a test that sets delimiter and prefix.
...
Expands the delimiter tests to include a test that uses both prefix
and delimiter.
2015-07-14 14:11:34 -07:00
Andrew Gaul
Timur Alperovich
Ignasi Barrera
Chaithanya Ganta
Timur Alperovich
Shri Javadekar
Zack Shoylev
quod3
Oliver Gondža
Roman Coedo
Ka-Hing Cheung