Commit Graph

2513 Commits

Author SHA1 Message Date
Zack Shoylev 42079e1392 Parallel upload for BaseBlobStore 2016-07-15 04:15:17 -05:00
Andrea Turli a515ce2f22 Change line endings on files recursively 2016-07-14 00:02:13 +02:00
Andrew Gaul 4874a1eb18 JCLOUDS-1137: Handle TOCTOU during blobMetadata
A similar issue exists when getting a blob payload when a caller
simultaneously removes the blob.
2016-07-11 22:02:15 -07:00
Andrew Gaul 08e78c979e Add S3Proxy profile to skip failing tests 2016-07-11 16:42:09 -07:00
Zack Shoylev 6bff97b6d3 Changes the upload behavior to parallel, a TODO 2016-07-01 11:26:56 -05:00
Duncan Grant 7a979ba87b ImageId can be image name or id and will pull from hub 2016-06-28 09:47:43 +02:00
Andrew Gaul fff12293f9 JCLOUDS-1125: Add missing expected exception 2016-06-23 23:06:13 -07:00
Andrew Gaul 6df0472ab5 JCLOUDS-1125: local blobstore list MPUs 2016-06-23 21:45:04 -07:00
Arvind Nadendla 112c64e92a Make links nullable 2016-06-20 18:11:12 -05:00
Ignasi Barrera bde70457d0 JCLOUDS-1105: Do not create the default security group if custom groups are specified 2016-06-17 22:18:35 +02:00
urban 71b3a2b631 JCLOUDS-1129:Add more interfaces of the same network 2016-06-17 22:16:59 +02:00
Andrew Donald Kennedy 496fbae646 Fix incorrect key format check in Docker 2016-06-17 00:01:35 +02:00
Andrew Gaul 5fec2346a6 JCLOUDS-1028: Configure idempotent methods
Enable POST for Atmos, S3, and Swift.
2016-06-15 16:35:22 -07:00
Andrew Gaul 0bd2959410 JCLOUDS-1125: portable list multipart uploads
Only Azure, B2, and S3 support this operation.  Some MultipartUpload
fields become nullable.
2016-06-13 16:26:43 -07:00
Andrew Gaul 445664c9f1 JCLOUDS-1125: S3 list multipart uploads 2016-06-13 16:26:43 -07:00
Aled Sage 730a4cfe87 Docker.ContainerToNodeMetadata: remove needless mocking
Previously it mocked the container and the StateToStatus. It just
asserted that getter methods were called, rather than confirming the
resulting NodeMetadata had the right values.

By removing some of the mocks, it not only simplifies the code but
improves code-coverage. (e.g. if StateToStatus was being passed the
wrong value, the previous test would not have noticed).
2016-06-13 17:18:01 +02:00
Aled Sage fc88756d5a docker: get container’s IPs from all networks 2016-06-13 17:17:56 +02:00
Duncan Grant 91339b200e Node json should be optional
The Node element of the container json is optional
as well as nullable.
2016-06-13 12:56:05 +02:00
Zack Shoylev c1ce819f61 updateTemporaryUrlKey test 2016-06-07 17:37:39 -05:00
Duncan Grant ebc8f7568f Use container host ip not manager ip
When providing node ip in metadata jclouds was using the
ip of the docker manager which was fine for docker machine
but fails when using a swarm cluster.  This uses the ip returned
from docker.
2016-06-06 16:16:44 +02:00
Andrew Gaul a67ae3f27b Make inner classes static where possible
Found via error-prone.
2016-06-01 20:03:54 -07:00
Josef Cacek c0469cc9b7 [JCLOUDS-1118] fix configuration options location; mark some fields Nullable 2016-05-26 12:07:39 +02:00
Josef Cacek d4cf774389 [JCLOUDS-1117] fix for advanced Docker configuration - the Config.Builder should stay unchanged if used 2016-05-26 12:07:33 +02:00
Andrew Gaul 4ef28251c5 Add missing import
Regression from c5d845d368adf01909fd8ab5eeb824f7c44fb69e.
2016-05-23 20:26:01 -07:00
Andrew Donald Kennedy 61fb3ec145 Support supplying key and certificate to Docker as data 2016-05-23 17:52:24 +02:00
Jim Spring f46b38dd89 client credentials JWT support 2016-05-20 00:10:37 +02:00
Zack Shoylev c96cfb6176 Fix cleanup when the security group extension is not available. 2016-05-19 12:08:36 -05:00
Andrew Gaul 41bdd9c5a9 Skip unimplemented signed URL tests in S3 provider
jclouds aws-s3 provider supports these but s3 does not.  Also expect
failure in atmos and azureblob.  Follow on to
a4c40e15cd.
2016-05-12 12:12:52 -07:00
Andrew Gaul e423bc949e Avoid use of primitive wrapper constructors
Deprecated in Java 9.
2016-05-09 22:47:27 -07:00
Andrew Gaul 62410d31ae Handle missing containers in LocalBlobStore.list
Also fix FilesystemStorageStrategyImpl.getContainerMetadata to return
null on missing container as jdbc and transient already do.
2016-05-06 21:36:28 -07:00
Andrew Gaul a4c40e15cd Handle unimplemented signed URL tests in providers
This commit makes it evident in source code which providers do not
support this feature.
2016-05-06 10:31:37 -07:00
Andrew Gaul 0af3380a60 Address error-prone CheckReturnValue errors
Found via Guava 20.0.
2016-04-28 14:16:22 -07:00
Andrew Gaul da386ad0e2 Correct FindBugs warnings 2016-04-27 20:28:37 -07:00
Ignasi Barrera 04f1bb2b49 JCLOUDS-1104: Extension namespaces are deprecated and can be null 2016-04-26 22:38:45 +02:00
Andrew Donald Kennedy 42d0576aa9 Add openStdin option to Docker template options and set all port bindings explicitly 2016-04-26 10:33:17 +02:00
Andrew Gaul 6446627aad Correct equals with incompatible types
Found via error-prone.
2016-04-24 23:28:55 -07:00
Andrew Gaul 66cda12b50 Avoid eager evaluation of Preconditions messages
Found via error-prone.
2016-04-24 23:28:55 -07:00
Andrew Donald Kennedy 49f7bc37af Make DockerTemplateOptions values null safe 2016-04-19 17:01:18 +02:00
Andrew Gaul 3cf4e3d79c JCLOUDS-1101: Correct generics wildcard errors
Always incorrect but previously broken on Java 9.  Reference:

https://bugs.openjdk.java.net/browse/JDK-8075793
2016-04-18 09:49:58 -07:00
Ignasi Barrera 819141a608 Fixed OAuth authentication flow injections 2016-04-07 10:10:10 +02:00
Andrew Donald Kennedy 047595c7bb Added volumesFrom to Docker template options 2016-04-05 10:22:56 +02:00
Jim Spring 0420741690 Add support for Azure AD authentication using Service Principal and Password 2016-04-01 14:50:09 +02:00
Oleg 74a2a2683f JCLOUDS-1102: Fix. Rackspace returns a new structure for volume types. 2016-04-01 14:48:12 +02:00
Fernando Ribeiro 80beb9b397 JCLOUDS-1098: Fix OpenStack Marconi Unit Tests 2016-03-23 16:28:34 +01:00
Fernando Ribeiro 92e6319cd7 JCLOUDS-1095: Update OpenStack KeyStone Support for Zaqar 2016-03-23 10:29:07 +01:00
Zack Shoylev 760995a46e Use non-privileged mode for compute 2016-03-21 12:49:07 -05:00
Andrew Gaul 6e7801ad58 JCLOUDS-766: Fix issues with chunked upload signer
Previously we created a new builder instead of using the one the
method modified and did not preserve content encoding.  Addresses
AWSS3BlobIntegrationLiveTest.testPutInputStream test failures.
2016-03-20 16:06:51 -07:00
Andrew Gaul 8053abb530 JCLOUDS-766: Check metadata and headers for MD5
Addresses AWSS3ClientLiveTest.testMultipartSynchronously test
failures.
2016-03-18 22:56:54 -07:00
Andrew Gaul 35c5d7ffc0 JCLOUDS-766: Include Content-MD5 in v4 signature
Addresses AWSS3BlobIntegrationLiveTest.deleteMultipleObjects test
failures.
2016-03-18 16:58:03 -07:00
Andrew Gaul feae011c4d JCLOUDS-766: URL encode path for v4 signatures
Addresses AWSS3BlobIntegrationLiveTest.deleteObject test failures.
2016-03-18 15:05:05 -07:00
IgnacioCalvo 7d2f1e8b4d JCLOUDS-1080: Force JSON format for requests
Also make all fields optional.
2016-03-17 20:40:51 -07:00
Josef Cacek f3ee898c13 [JCLOUDS-1089] Add image history support to Docker ImageApi 2016-03-08 23:06:21 +01:00
Josef Cacek 831cdc67c3 [JCLOUDS-1002] provide access to Docker container Config object in the Node template options 2016-03-08 23:01:52 +01:00
Zack Shoylev f8ebbfdf8a Fixes an intermittently failing swift live test. 2016-03-08 12:38:13 -06:00
Andrew Gaul c18371a793 JCLOUDS-766: Add eu-central-1 to AWS S3 regions
Also clean up some missing and duplicated properties.
2016-03-07 21:21:08 -08:00
Andrew Gaul dd670f3f78 JCLOUDS-1064: More helpful error for null length 2016-03-06 19:08:26 -08:00
Zack Shoylev 243afed278 Add alias fallback info to keystone and nova extensions. 2016-02-24 11:22:10 -06:00
Josef Cacek 3f1fe271ed [JCLOUDS-1084] Docker live tests fixed and made more robust 2016-02-22 17:16:40 +01:00
Josef Cacek e0ab5d848a [JCLOUDS-1079] Make Floating IPs public in the NodeMetadata in openstack-nova provider 2016-02-19 11:59:52 +01:00
Zack Shoylev 8bbcfed312 Ensures the Swift container list call requests JSON as per spec 2016-02-18 18:48:55 -06:00
Andrew Gaul e0a7ea7fdf Only quote ETag if it does not already have quotes 2016-02-16 16:29:54 -08:00
Andrew Gaul 7eb46cce36 JCLOUDS-651: Swift support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul 6cdb1216a7 JCLOUDS-651: S3 support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul cc8af838e8 JCLOUDS-651: Atmos support for conditional copies 2016-02-16 16:29:54 -08:00
Andrew Gaul 293d3f864e Convert CopyOptions into an AutoValue
This commit requires an interface change since AutoValue lacks support
for Optional and uses Nullable annotations instead.
2016-02-16 16:29:54 -08:00
Zack Shoylev c8bbb44f37 Enhance the way openstack extensions are resolved. Needed for new openstack versions. 2016-02-16 15:18:39 -06:00
rongallagher 27b3a844f8 JCLOUDS-1072: Add support for environment-specific run lists to Role 2016-02-10 12:09:19 +01:00
Andrew Gaul 2bd0550110 Add support for Swift conditional copy 2016-02-09 16:34:48 -08:00
Andrew Gaul 6371235568 Null out tmpFile to avoid remove after rename 2016-02-05 12:35:27 -08:00
Andrew Gaul 6cefc99cf0 Always delete tmpFile in filesystem putBlob
Previously we would only do this on the exceptional path.  Also throw
IOException on failed rename for consistency.
2016-02-05 00:52:37 -08:00
Andrew Gaul 9feeee8347 Set xattr before rename to make blob create atomic
Previously concurrent operations could expose an object before put had
completed.  Note that the temporary file is still exposed to the
client.
2016-02-04 23:14:54 -08:00
Andrew Gaul be96b9f270 Suppress spurious ignored return value errors
Found with Guava 20.
2016-02-04 14:57:46 -08:00
Reijhanniel Jearl Campos d83282ca53 JCLOUDS-691: Remove 404 fallbacks of PUT and POST calls from OpenStack APIs 2016-02-04 12:10:06 +01:00
Andrew Gaul de333e8b3c Check Content-Length in local blobstore putBlob 2016-02-03 22:03:48 -08:00
Andrew Gaul 6360023f09 Remove unnecessary static imports
Found via error-prone.
2016-01-29 09:14:55 -08:00
Andrew Gaul 86e947dded Revert "Always return ETag from filesystem object store"
This reverts commit 496e27f1af.  This
commit imposed too much computational overhead when listing blobs.
2016-01-28 23:04:12 -08:00
Andrew Gaul a8f5401f6c Enable filesystem service tests 2016-01-28 22:42:05 -08:00
Ignasi Barrera 773aa30c64 JCLOUDS-1067: Fix cookbook grouping mapping in Chef metadata 2016-01-28 15:12:29 +01:00
Andrew Gaul 02b2e80b27 Delete single- and multi-part objects in Swift
Due to Swift's wonky MPU implementation we must try the multi-part
delete first followed by a single-part delete if it fails.  Previously
delete would leave orphan subobjects.  Fixes andrewgaul/s3proxy#92.
2016-01-26 21:22:54 -08:00
Andrew Gaul 68ff250c38 Return response from StaticLargeObjectApi.delete 2016-01-25 13:26:03 -08:00
Andrew Gaul 3a65df1e2e JCLOUDS-988: Subclass StaticLargeObjectApiLiveTest in Cloudfiles providers
This allows tests to run.
2016-01-22 14:01:49 -08:00
Ignasi Barrera ef7b85130e JCLOUDS-1056: Configure OSGi packaging 2016-01-22 11:34:07 +01:00
Andrea Turli a4a255fa4a improve roll-back strategy when floating-ip are not available
- improve destroyNodes to clean up securityGroups and keyPair created explicitly for that node
- refactor clean up server in one place
2016-01-22 11:02:44 +01:00
Andrew Gaul 100d1dac6c Do not override S3 provider methods
Fixes Guice 4.0 support.
2016-01-21 10:12:19 -08:00
Andrew Gaul 7b3a300d51 Skip testBlobAccess on filesytem
Follow on to 52dc1a3ccf.
2016-01-20 23:47:06 -08:00
Andrew Gaul 52dc1a3ccf Add BaseBlobIntegrationTest.testSetBlobAccess
Similar to BaseContainerIntegrationTest.testSetContainerAccess.
2016-01-19 14:40:55 -08:00
Ignasi Barrera 40f31786c5 JCLOUDS-512: Implement the ImageCache
This commit refactors the ImageCacheSupplier to act as a
proper cache. It is used by the ImageExtesion and all operations
on the images are propagated to the cache.

A method has also been added to the TemplateBuilder to let users
force a cache refresh. There have been several requests to provide a way
to disable image caching in the compute abstraction, and this new method
should fix that.
2016-01-19 11:25:09 +01:00
Andrew Gaul 6bc2ff7e00 JCLOUDS-1042: Remove unused bucketAcls 2016-01-17 17:35:12 -08:00
Andrew Gaul 68e6539192 JCLOUDS-1042: Remove unneeded ACL swizzling
Follow on to 750c5d5316.
2016-01-17 17:30:06 -08:00
Andrew Gaul b63f74a6eb JCLOUDS-948: Swift Cache-Control support
Not supported.
2016-01-15 13:22:15 -08:00
Andrew Gaul f292408af4 JCLOUDS-948: S3 Cache-Control support
Deprecate older S3-specific Cache-Control mechanism.
2016-01-15 13:22:14 -08:00
Andrew Gaul 82ad05e98e JCLOUDS-948: Filesystem Cache-Control support 2016-01-15 04:43:39 -08:00
Andrew Gaul 77d3db0058 JCLOUDS-948: Atmos Cache-Control support
Not supported.
2016-01-15 04:41:01 -08:00
Zhao Jin 8bddbb496a JCLOUDS-480: AWS S3 v4 signature
This includes support for chunked uploads.
2016-01-13 15:08:18 -08:00
Andrew Gaul c20fcb8cda Do not compare MPU ETag contents
AWS S3 MPU ETag are hashes of the part ETag headers but some
implementations, specifically S3Proxy with the filesystem provider,
represent multi-part objects as a single object.  Remove these checks
since they add nothing.
2016-01-12 15:36:37 -08:00
Rita Zhang a12ce5a8b7 Add support for AzureBlobStore.getBlobAccess and RegionScopedSwiftBlobStore.getBlobAccess.
They are currently throwing UnsupportedOperationException exception.
2016-01-05 15:33:50 -08:00
Andrew Gaul 91b202f4ea JCLOUDS-844: Swift putBlob ACLs 2015-12-22 00:46:34 -08:00
Andrew Gaul 85951e2af9 JCLOUDS-844: S3 putBlob ACLs 2015-12-22 00:46:34 -08:00
Andrew Gaul cc0d0d40c5 JCLOUDS-844: Atmos putBlob ACLs 2015-12-22 00:46:33 -08:00
Andrew Gaul 86491bc607 JCLOUDS-844: Portable putBlob ACLs 2015-12-22 00:46:33 -08:00