185 Commits

Author SHA1 Message Date
Andrew Gaul
4874a1eb18 JCLOUDS-1137: Handle TOCTOU during blobMetadata
A similar issue exists when getting a blob payload when a caller
simultaneously removes the blob.
2016-07-11 22:02:15 -07:00
Andrew Gaul
6df0472ab5 JCLOUDS-1125: local blobstore list MPUs 2016-06-23 21:45:04 -07:00
Andrew Gaul
0bd2959410 JCLOUDS-1125: portable list multipart uploads
Only Azure, B2, and S3 support this operation.  Some MultipartUpload
fields become nullable.
2016-06-13 16:26:43 -07:00
Andrew Gaul
62410d31ae Handle missing containers in LocalBlobStore.list
Also fix FilesystemStorageStrategyImpl.getContainerMetadata to return
null on missing container as jdbc and transient already do.
2016-05-06 21:36:28 -07:00
Andrew Gaul
6371235568 Null out tmpFile to avoid remove after rename 2016-02-05 12:35:27 -08:00
Andrew Gaul
6cefc99cf0 Always delete tmpFile in filesystem putBlob
Previously we would only do this on the exceptional path.  Also throw
IOException on failed rename for consistency.
2016-02-05 00:52:37 -08:00
Andrew Gaul
9feeee8347 Set xattr before rename to make blob create atomic
Previously concurrent operations could expose an object before put had
completed.  Note that the temporary file is still exposed to the
client.
2016-02-04 23:14:54 -08:00
Andrew Gaul
de333e8b3c Check Content-Length in local blobstore putBlob 2016-02-03 22:03:48 -08:00
Andrew Gaul
86e947dded Revert "Always return ETag from filesystem object store"
This reverts commit 496e27f1afa32b90d0656c3f21387cf68a30cb31.  This
commit imposed too much computational overhead when listing blobs.
2016-01-28 23:04:12 -08:00
Andrew Gaul
a8f5401f6c Enable filesystem service tests 2016-01-28 22:42:05 -08:00
Andrew Gaul
7b3a300d51 Skip testBlobAccess on filesytem
Follow on to 52dc1a3ccfc6413d3350bc8f1d092cda8d7a25d3.
2016-01-20 23:47:06 -08:00
Andrew Gaul
52dc1a3ccf Add BaseBlobIntegrationTest.testSetBlobAccess
Similar to BaseContainerIntegrationTest.testSetContainerAccess.
2016-01-19 14:40:55 -08:00
Andrew Gaul
82ad05e98e JCLOUDS-948: Filesystem Cache-Control support 2016-01-15 04:43:39 -08:00
Andrew Gaul
37f307ecd0 Skip testSetContainerAccess for local blobstores
These lack a public HTTP endpoint needed for this test.
2015-12-20 01:14:33 -08:00
Ignasi Barrera
90423d7b9b Fix filesystem api in OSX 2015-11-24 15:38:10 +01:00
Zack Shoylev
ab25fc7259 putblob fails when file exists 2015-11-23 16:01:46 -06:00
Ka-Hing Cheung
ca70d44143 don't remove directory if it's a blob
the filesystem blob store was always removing a/ when a/b was
removed, even if the client explicitly created a/. this patch
fixes that
2015-11-20 17:27:19 -08:00
Andrew Gaul
f706d9c13a JCLOUDS-835: Atomically put filesystem blobs
Write to a temporary file and rename to the desired destination to
better match object store semantics.
2015-11-20 14:44:02 -08:00
Zack Shoylev
2efcb2c5a9 Fixes a windows locale bug with the "Everyone" principal 2015-11-19 11:50:15 -06:00
Zack Shoylev
41ce90ec36 Fixes tests failing on windows filesystems Makes windows behavior more consistent, especially for deletes 2015-11-11 21:01:13 -06:00
Timur Alperovich
85357cf92a JCLOUDS-217: Filesystem: avoid trailing spaces.
On Windows, we need to avoid trailing spaces, as the test fails to
create the required blob. Specifically, we should not test blobs named
" " and "%20 ".
2015-09-08 15:19:03 -07:00
Nate Rosenblum
496e27f1af Always return ETag from filesystem object store
If the Content-MD5 extended attribute is missing for an object,
recompute it and return as the ETag. Primarily affects OS X.
2015-08-20 14:46:17 -07:00
Zack Shoylev
6945b04243 Fixes a small test bug on windows 2015-07-08 20:19:33 -05:00
Timur Alperovich
5383148d9e Filesystem: disable directory integration test.
There is no notion of marker blobs in the file system blob store,
which means that when listing results after issuing the
"createDirectory" request, the marker blob will not be in the result
set.
2015-07-07 18:00:32 -07:00
Timur Alperovich
8c9344b501 JCLOUDS-930: Regression in a File System test.
Fix a regression in the file system blob store, where a test does not
expect the trailing delimiter character.
2015-06-29 21:41:33 -07:00
Andrew Gaul
1dcd4500ea JCLOUDS-930: disable testListDirectoryBlobsS3FS
Regression from 8677ffcb213f4182eee03039431a9562c2eee16a.
2015-06-29 19:36:20 -07:00
Ka-Hing Cheung
8b325d585e object stores typically sort the list of containers 2015-06-24 15:08:23 -07:00
Andrew Gaul
38fa41d056 JCLOUDS-941: Ignore metadata tests on Mac OS X
Also make testOverwriteBlobMetadata consistent with other tests.
2015-06-22 11:50:20 -07:00
Iván López
37a014ae00 JCLOUDS-941: Auto-detect filesystem Content-Type
When a filesystem blob does not have content metadata and when users
set jclouds.filesystem.auto-detect-content-type to tru, probe the file
type to return to clients.  This is useful when using jclouds to serve
an existing filesystem.
2015-06-21 22:55:57 -07:00
Zack Shoylev
786e5d8e25 Adds windows ACL code for file permissions, fixes build. 2015-04-28 14:35:58 -05:00
Andrew Gaul
fadbbad8ac JCLOUDS-732: Filesystem portable object ACLs 2015-03-31 15:33:34 -07:00
Andrew Gaul
deff8d8413 JCLOUDS-660: Filesystem portable container ACLs 2015-03-31 15:33:33 -07:00
Andrew Gaul
4f566d3b6e Address and suppress FindBugs warnings 2015-03-30 20:40:10 -07:00
Ka-Hing Cheung
a2ee7341e3 fix non-recursive list of dir with empty prefix 2015-02-23 17:52:42 -08:00
Andrew Gaul
d8aea21f72 JCLOUDS-732: Local blobstore portable object ACLs
Intentionally unimplemented.
2015-02-21 10:28:02 -08:00
Ka-Hing Cheung
be7b9f4cc4 fix non-recursive list with empty prefix 2015-02-20 23:29:48 -08:00
Andrew Gaul
994f84e4f5 JCLOUDS-660: Local blobstore portable container ACLs
Intentionally unimplemented.
2015-02-12 17:26:47 -08:00
Andrew Gaul
10690f9293 Handle null return value from File.listFiles
This matches the behavior of other call sites.
Fixes andrewgaul/s3proxy#39.
2015-02-03 20:22:58 -08:00
Zack Shoylev
a8c5e28129 Windows fixes, mostly separator related. 2015-01-28 19:47:19 -06:00
Ka-Hing Cheung
3cabb196ce fix usermetadata check on OS X
Java on OS X does not support extended attributes, which the
filesystem blobstore uses to implement user metadata. This disable
the relevant test on OS X
2015-01-20 17:33:22 -08:00
Ka-Hing Cheung
5edd5de38c make xattr work in docker volume
when you bind a host volume into docker, java does not correctly
detect that and checks the xattr support on the root fs instead
of the host fs. The root fs often does not support xattr, so
the check would fail even if the target really does support xattr.

the fix is just try setting the xattrs anyway, and let them fail
if there really isn't xattrs support
2015-01-13 18:08:55 -08:00
Ka-Hing Cheung
5cc3b243fc disable directory blob tests in OS X
On OS X, java doesn't support xattrs, which is required by
directory blobs. Disable those tests on OS X
2015-01-12 12:32:41 -08:00
Andrew Gaul
fae097e144 JCLOUDS-654: Add object size to StorageMetadata
This allows callers to read the content length during container
listing.  Tested against: atmosonline, aws-s3, azureblob, filesystem,
and transient.  Intentionally not implemented for legacy swift
provider.
2015-01-09 22:34:56 -08:00
Ka-Hing Cheung
b7ab8b18a0 support directory blobs
make the filesystem blob store distinguish between a/ and a. a/
must be a directory blob with no content and only metadata

on listing, only directories with md5 attribute is considered a
blob and returned
2015-01-09 16:28:16 -08:00
Andrew Gaul
77eaa2c1f9 Add container creation date to local blobstores 2015-01-09 14:10:33 -08:00
Jeremy Daggett
d3c1e2eab4 Use AutoService to generate service loader metadata 2015-01-09 11:22:12 -08:00
Ka-Hing Cheung
692691bcdc fix fs container validator test
previoisly it was testing blob key validator instead of container
name validator
2015-01-07 11:36:47 -08:00
Adrian Cole
0012e6f950 JCLOUDS-534 Avoid runtime incompatibility introduced by Guava's closeQuietly. 2014-10-29 17:01:26 -07:00
Adrian Cole
a22a725293 JCLOUDS-40 Remove AsyncBlobStore references from filesystem api 2014-10-05 13:18:02 -07:00
Adrian Cole
dfb583b67a JCLOUDS-40 remove all implementations of AsyncBlobStore except Submission in preparation for complete removal. 2014-10-05 08:49:38 -07:00