645 Commits

Author SHA1 Message Date
Andrew Gaul
cd50ad905d Avoid NPE in BaseBlobStore.copyBlob
Resolves copyBlob failures in Atmos.
2016-05-12 12:48:07 -07:00
Andrew Gaul
62410d31ae Handle missing containers in LocalBlobStore.list
Also fix FilesystemStorageStrategyImpl.getContainerMetadata to return
null on missing container as jdbc and transient already do.
2016-05-06 21:36:28 -07:00
Andrew Gaul
a4c40e15cd Handle unimplemented signed URL tests in providers
This commit makes it evident in source code which providers do not
support this feature.
2016-05-06 10:31:37 -07:00
Andrew Gaul
35f4f2bbf1 Use more unique names in containerDoesntExist
Previously this failed on AWS due to another container named
"forgetaboutit".
2016-03-27 13:48:42 -07:00
Andrew Gaul
2638ba4e69 Consistently quote ETags during comparison
Regression from e0a7ea7fdf4554d919b23ce72391a37e41d7cad8.
2016-02-16 23:10:16 -08:00
Andrew Gaul
66609e6d70 JCLOUDS-651: Local blobstore support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul
8945258d79 JCLOUDS-651: Portable support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul
293d3f864e Convert CopyOptions into an AutoValue
This commit requires an interface change since AutoValue lacks support
for Optional and uses Nullable annotations instead.
2016-02-16 16:29:54 -08:00
Andrew Gaul
a697396e8c Correctly remove prefix in DelimiterFilter
Previously this called String.replaceFirst which uses a regular
expression and incorrectly handles characters like *.  Also remove
other correct but unnecessary call to String.replaceFirst.
2016-02-09 10:48:01 -08:00
Andrew Gaul
de333e8b3c Check Content-Length in local blobstore putBlob 2016-02-03 22:03:48 -08:00
quod3
77eef902b4 Use parseLong instead of parseInt in range parser
Use parseLong instead of parseInt when parsing open-ended byte ranges in LocalBlobStore. Without this fix, any "from byte x" or "to byte x" getBlob() call will throw a NumberFormatException if x is too big to fit into an int (2 GB).

Fixes https://issues.apache.org/jira/browse/JCLOUDS-1073
2016-01-30 20:11:07 -05:00
Andrew Gaul
eb6f16e2f9 Enable CollectionIncompatibleType
Also suppress two false positives.
2016-01-29 08:50:09 -08:00
Andrew Gaul
b6920ca44b JCLOUDS-1065: Deprecate inDirectory option
Superseded by prefix and delimiter support.
2016-01-28 22:59:12 -08:00
Andrew Gaul
a3376d4efe Consistently implement domain object methods
Including equals, hashCode, and toString.
2016-01-24 21:58:39 -08:00
Andrew Gaul
b50c518f7e Avoid sleeping in signed URL tests
Also bump expiration to a consistent 60 seconds.
2016-01-23 13:48:43 -08:00
Andrew Gaul
b2c0786fc0 Enable range tests for azureblob 2016-01-23 12:38:50 -08:00
Andrew Gaul
52dc1a3ccf Add BaseBlobIntegrationTest.testSetBlobAccess
Similar to BaseContainerIntegrationTest.testSetContainerAccess.
2016-01-19 14:40:55 -08:00
Andrew Gaul
688890819e Fix replacing metadata in BaseBlobStore
Fixes Atmos errors.  Uncovered by removing unneeded addContentMetadata
call in BaseBlobIntegrationTest.testCopyBlobReplaceMetadata.
2016-01-15 13:22:15 -08:00
Andrew Gaul
5fd7b1aa78 JCLOUDS-948: Portable Cache-Control support 2016-01-15 04:41:01 -08:00
Andrew Gaul
a90245afe5 Add marker and prefix to equals and toString 2016-01-11 19:18:14 -08:00
Andrew Gaul
c0d72b6f53 JCLOUDS-844: Local blobstore putBlob ACLs 2015-12-22 00:46:33 -08:00
Andrew Gaul
86491bc607 JCLOUDS-844: Portable putBlob ACLs 2015-12-22 00:46:33 -08:00
Andrew Gaul
37f307ecd0 Skip testSetContainerAccess for local blobstores
These lack a public HTTP endpoint needed for this test.
2015-12-20 01:14:33 -08:00
Andrew Gaul
13701bf7ab Do not test marker contents
Some providers like Azure have opaque markers.
2015-12-19 15:24:12 -08:00
Andrew Gaul
a7c66d5120 JCLOUDS-1042: Test reading from public container
Blob has default access but container is public-read.
2015-12-18 23:28:39 -08:00
Andrew Gaul
4abd5dd16f Revert "Delete parts when removing Swift multipart objects"
This reverts commit 3c1588527dfc0682faf696186ebc18d29f7ee0da.
2015-12-08 10:08:22 +08:00
Andrew Gaul
2dd231b02b Filter out folders when listing a container
These folders represent directories in the filesystem blobstore which
real blobstores do not have.  Their presence complicates providing the
correct marker for paginated listing.  Future commits will remove
folders entirely.
2015-11-20 16:27:53 -08:00
Andrew Gaul
4ce357ba69 Consistently quote ETags in comparisons
Addresses integration test regression in
25f4807df868cff9a9da599009adc0091f7bb52b.
2015-11-19 19:00:34 -08:00
Andrew Gaul
25f4807df8 Tolerate missing or extra quotes in ETags
References andrewgaul/s3proxy#77.
2015-11-19 17:40:48 -08:00
Andrew Gaul
8d87bfc61b Avoid buffering ByteSource when not needed
This allows range requests of large files to work.  Regression from
79fe91bd035461d9344c03bda276cb12a7a10b1a.
2015-11-19 15:10:23 -08:00
Andrew Gaul
3c1588527d Delete parts when removing Swift multipart objects
Swift implements multi-part upload with user-visible parts and an
explicit manifest.  When deleting an MPU blob it can delete only the
manifest or both the manifest and parts.  For consistency with other
providers, we now do the latter in the portable abstraction.  Swift
ignores the multipart-manifest=delete parameter for single-part
objects.  Fixes andrewgaul/s3proxy#92.
2015-11-17 23:06:33 -08:00
Zack Shoylev
41ce90ec36 Fixes tests failing on windows filesystems Makes windows behavior more consistent, especially for deletes 2015-11-11 21:01:13 -06:00
Andrew Gaul
e0c959c212 Pagination for listing LocalBlobStore MPU parts 2015-10-20 14:01:32 -07:00
Timur Alperovich
aaa164179c JCLOUDS-217: Percent encoding changes.
jclouds should not check if the string is encoded, but rather expect
that all strings would be encoded prior to transmission. As part of
that change, we must make sure that no code relies on such behavior
within jclouds. This commit adds a blobstore test to check encoding
pattern for blobs. It also removes the encoding check in the Strings2
class and the related test.
2015-09-08 15:12:14 -07:00
Oliver Gondža
edde721937 JCLOUDS-962: Make all @Provider methods final 2015-09-02 23:54:44 +02:00
Timur Alperovich
3757a64abf JCLOUDS-992: LocalBlobStore - use FOLDER for DIR.
Changed the LocalBlobStore to use FOLDER, as opposed to RELATIVE_PATH
when creating directories. Changed the delete keys strategy to treat
FOLDER blobs as regular blobs.
2015-08-20 12:42:11 -07:00
Timur Alperovich
0c5a3db9df JCLOUDS-992: remove special directory handling. 2015-08-20 12:42:11 -07:00
Timur Alperovich
42bfeccdfd Remove last marker test from ListContainerMarker.
Azure does not allow arbitrary marker specification and the generic
testListContainerMarker test should not assume that the behavior is
possible. There is already a separate test for such scenario that is
skipped on Azure.
2015-08-19 16:03:33 -07:00
Roman Coedo
53c3a4918a Fix LocalBlobStore 2015-08-06 09:43:33 -05:00
Roman Coedo
79fe91bd03 JCLOUDS-979: Fix range get for LocalBlobStore 2015-08-04 11:40:57 +02:00
Andrew Gaul
86f47394bf Correct FindBugs violations 2015-07-31 14:53:58 -07:00
Andrew Gaul
7e6284dd86 Add missing equals and hashCode methods
Possible cause of odd PageSet behavior.
2015-07-31 14:17:41 -07:00
Andrew Gaul
5ad245dea5 Do not buffer range get in LocalBlobStore
This avoids OutOfMemoryError with large blobs and enables offsets over
2 GB.
2015-07-27 18:02:54 -07:00
Ka-Hing Cheung
2c578d7bcb marker should be null when there are no more items 2015-07-21 15:46:58 -07:00
Timur Alperovich
19d54369db Propagate delimiter changes to LocalBlobStore.
Propagates the changes to add delimiter support to the local blob
store. The prefix option no longer works with the non-recursive
option. The caller _must_ set the delimiter option to enable the same
functionality.
2015-07-14 14:11:34 -07:00
Timur Alperovich
e4161a59bb Move tests to test the file system API.
Move the prefix and delimiter tests from BaseContainerLiveTest to
BaseContainerIntegrationTest, as that class is subclassed in
Filesystem integration tests. This will make sure the file system blob
store will not regress.

The ListContainerTest is updated to _not_ use a file separator
character, as that leads to a possible creation of directories.
Instead, "-" is used as the delimiter for the delimiter option tests.
2015-07-14 14:11:34 -07:00
Timur Alperovich
613b6bf045 Add a test that sets delimiter and prefix.
Expands the delimiter tests to include a test that uses both prefix
and delimiter.
2015-07-14 14:11:34 -07:00
Timur Alperovich
4dc33ab564 JCLOUDS-929: Implement delimiter support in Local.
Adds the delimiter support in the local blob store. The existing
directory implementation is preserved and is not compatible with the
delimiter option, as the existing implementation assumes the usage of
filesystem directories.
2015-07-13 14:34:59 -07:00
Timur Alperovich
a29d75a5d1 JCLOUDS-929: Implement generic delimiter support.
The patch adds delimiter option support in the generic blob store
interface. A live integration test is added to verify that jclouds
correctly lists objects separated by a delimiter.
2015-07-13 14:34:58 -07:00
Andrew Gaul
7d7deefb75 JCLOUDS-894: Handle part size when > maximum
Azure has a small part size that was not handled by the existing
logic.  This code is twisty and confusing and should be rewritten!
2015-07-09 18:00:37 -07:00