Commit Graph

10331 Commits

Author SHA1 Message Date
Andrew Gaul 8cd68a3503 JCLOUDS-1369: improving slicing with many parts
Previously with GCS and its maximum 32 parts, the slicing algorithm
would sliced 3.2 GB blobs into (31) 32 MB parts and (1) 2.3 GB part.
With this algorithm it creates (31) 100 MB parts and (1) smaller part.
2018-01-09 12:56:00 -08:00
Andrew Gaul d05be89614 JCLOUDS-1368: Correct use of slicing algorithm
MultipartUploadSlicingAlgorithm creates multiple equal-sized parts and
a remaining amount.  BaseBlobStore.putMultipartBlob used this
interface incorrectly, which could create more parts than intended
since the remaining size could be larger than the part size.  This
manifested with Google Cloud Storage which only allows 32 parts.
2018-01-09 12:56:00 -08:00
Andrew Gaul 52c92a9eb5 Propagate access denied when creating container
Previously this method only reported whether it created a container or
not and callers could not determine whether there was an error or if
the container already existed.  References gaul/s3proxy#122.
2018-01-09 12:44:24 -08:00
Ignasi Barrera 3877303ede Update maven coordinates for promoted Azure ARM 2018-01-08 15:11:22 +01:00
Ignasi Barrera cc969052a7 Promote Azure ARM from jclouds-labs 2018-01-08 14:58:45 +01:00
Ignasi Barrera eebcc0017f JCLOUDS-1362: Proper password generation with custom constraints for each cloud 2018-01-08 08:48:37 +01:00
Ignasi Barrera 9fef6ed06b JCLOUDS-1362: Better password generation utility 2018-01-08 08:39:20 +01:00
Andrew Gaul 11640b6c2e Correct RandomByteSource.read return value
Previously read returned a value between -128 and 127.  -1 indicates
end of stream, causing issues for callers.  Instead return values
between 0 and 255 as intended.
2018-01-04 15:12:29 -08:00
Jim Spring a2dee2f84c Add Azure KeyVault support 2018-01-04 01:44:48 +01:00
Andrew Gaul 89e102810e Correct order of AutoValue parameters
Found via error-prone 2.1.3.
2017-12-23 23:47:17 +08:00
Yuval Kashtan 17cef5652f [JCLOUDS-1363] - Fix case sensitivity of blobstore metadata
HTTP headers are case insensitive by nature (see RFC 2616). When addUserMetadataTo check if this is indeed a user metadata header, it must properly ignore case.
The fix make sure that both metadataPrefix and the header key are compared with toLowerCase().

This solves issue with minio metadata read
2017-12-07 08:52:50 +01:00
Ignasi Barrera ac5f3a1452 Rename GraphRBAC live test class 2017-12-05 09:39:39 +01:00
Ignasi Barrera 3efce9a3a5 Configure the Graph RBAC API and allow mocking service endpoints 2017-12-04 10:12:16 +01:00
Ignasi Barrera 6c759930d7 Injectable current service principal 2017-12-01 14:21:22 +01:00
Ignasi Barrera 624367dfbf OAuth filter customization per request 2017-11-30 09:07:20 +01:00
Ignasi Barrera 715994b125 Fix GCE live tests 2017-11-30 09:06:23 +01:00
Ignasi Barrera a0f659faed
Fix DigitalOcean DropletApiLiveTest 2017-11-30 09:05:28 +01:00
Ignasi Barrera 6f7e4b92b0
Define the JDK version in a Maven property 2017-11-30 09:04:46 +01:00
Andrew Gaul 8ec3c520ce JCLOUDS-1337: revert legacy aws-s3 storage class
This functionality was deprecated in 2.1.x and we will remove it in
2.2.x.  Partially reverts 89053d9a8b.
2017-11-22 18:08:41 -08:00
William Chen a540daf3d6 This is the first commit of for SoftLayer Network API set
minor code improvement

Syntax fix with coding standards

- added more test coverage to NetworkApiMockTest class
- minor clean up
- minor code improvement
2017-11-22 14:32:22 +01:00
andreaturli ec03b710a7 [JCLOUDS-1360] add support for block device mappings to Nova 2017-11-22 11:29:49 +01:00
Andrew Gaul 9b59d099d1 JCLOUDS-1358: Handle 0-byte input with v4 signature
Previously this failed due to use of mark and reset on the
InputStream.
2017-11-21 11:36:11 -08:00
Andrew Gaul d29b79674d JCLOUDS-1358: Fix zero byte InputStream test input
Do not use ByteSource.empty() since it is backed by a
ByteArrayInputStream which supports reset.
2017-11-21 10:50:56 -08:00
jucolon 7368d58ad9 JCLOUDS-1342 : Azure-ARM Virtual Machine Scale Set Support 2017-11-21 10:20:07 +01:00
Nelson Araujo 5d82a3df97 JCLOUDS-1355: Enable VM creation without external IP address. 2017-11-21 09:58:53 +01:00
Andrew Gaul 73c3b6024c Add exception cause 2017-11-20 10:49:55 -08:00
Andrew Gaul a62494a117 JCLOUDS-1110: Inject timeStamp into Atmos provider
This addresses a unit test failure.
2017-11-14 10:51:58 -08:00
Andrew Gaul fda87d7dc7 JCLOUDS-1110: Atmos time-limited signed URLs 2017-11-13 14:44:02 -08:00
Andrew Gaul 3923fb0d71 JCLOUDS-598: Populate additional MD5 field
Follow-on to a6337fdcc4.
2017-11-13 09:21:19 -08:00
Andrew Gaul 2df35de45c Remove nonsense check for non-existent URL
This URL does not resolve and slows integration tests.
2017-11-12 17:51:49 -08:00
Andrew Gaul a6337fdcc4 JCLOUDS-598: Populate Atmos MD5 from wschecksum 2017-11-12 17:51:49 -08:00
Andrew Gaul 50026c8f2d Return Atmos objectID as ETag
Previously Atmos returned null.  Also rework the fix for JCLOUDS-339
which does not reproduce with AT&T Synaptic.  Fixes gaul/s3proxy#247.
2017-11-11 14:20:49 -08:00
Andrew Gaul e473d7df6a Tighten up test expectations
Follow on to 8b94febfeb.
2017-11-11 07:45:23 -08:00
Andrew Gaul 26570db680 Include HTTP status code in illegal range error 2017-11-10 22:49:45 -08:00
Andrew Gaul 8b94febfeb Throw consistent exception on invalid range read
Previously S3, Swift, and local blobstores threw a generic
IllegalArgumentException for this uncommon error.  Instead
consistently throw HttpResponseException.
2017-11-10 21:39:47 -08:00
khopade c468c60d51 Utilizing MD5 hash present in payload metadata, instead of generating it again from payload 2017-11-10 10:04:06 +01:00
Andrew Gaul 855ac7776c JCLOUDS-1080: remove spurious parameter from test
Follow on to 52044d23d9.
2017-11-07 11:49:00 -08:00
Andrew Gaul 52044d23d9 JCLOUDS-1080: request JSON only for list objects 2017-11-07 10:58:09 -08:00
Andrew Gaul 9dc85ecaab JCLOUDS-1281: Improve Swift DLO tests
Add DynamicLargeObjectApiLiveTest to Rackspace Cloud Files suite and
use correct regionId so that tests pass.  Also remove duplicative
test.
2017-11-07 10:50:10 -08:00
Archana Chinnaiah eaf3c779dc JCLOUDS-1281: Swift dynamic large objects 2017-11-07 10:50:10 -08:00
Andrew Gaul b2ced53e16 JCLOUDS-1111: Overwrite objects via remove and put
This partially reverts commit
e446b5b8b4.  AT&T Synaptic returns a
bogus error on with x-emc-force-overwrite:

HTTP 400, code=1012, message=There was a mismatch between the object
size and the specified extent size.
2017-11-06 18:05:47 -08:00
Timur Alperovich 343897d6d8 Use different content on overwrite.
When testing blobstore overwrite behavior, jclouds should use a blob
with different content (but same name).
2017-11-06 17:22:32 -08:00
Ignasi Barrera 8b33c44078 Add tags to virtual network creation 2017-11-06 09:09:30 +01:00
Andrew Gaul 46759f8bda Fix a few Coverity defects 2017-11-05 20:12:47 -08:00
Andrew Gaul 83d10e8194 JCLOUDS-1337: restore tier from stub blob
This allows completeMultipartUpload to propagate the tier set in
initiateMultipartUpload.
2017-11-03 09:52:42 -07:00
Richard Downer 98c289721c Recognise CentOS images on AWS Marketplace
CentOS’ officially-supported AMIs are hosted on the AWS Marketplace.
This adds support for those images, recognising the AMI naming
convention and ensuring the OS metadata is parsed correctly and the
correct SSH login name is used.

There is no change to the default jclouds configuration and the
official CentOS images will not be detected by default. To use these
images, you must alter the ami-query properties to include searching
the “AWS Marketplace”, which has an owner ID of 679593333241. You must
also manually log on to the AWS Marketplace, select your chosen CentOS
image, and “subscribe” to it (you can do this by proceeding as if to
launch an image, but stopping after you have agreed to the subscription
and before launching).
2017-11-03 10:23:37 +01:00
Richard Downer 045f1e9527 JCLOUDS-1351: improve OS Family parsing
Modifies OsFamily to have two tiers of known OSes, so that generic OS
names such as “Linux” cannot end up taking priority over more specific
OS names. This fixes the case where “CentOS Linux” was detected as LINUX
and not CENTOS.
2017-11-03 10:23:37 +01:00
Andrew Gaul 7fbef10d57 JCLOUDS-1337: Include tier in object listing
This requires hoisting Tier from BlobMetadata to StorageMetadata.
2017-11-02 16:24:34 -07:00
Andrew Gaul c0a7938b69 Address Checkstyle violation 2017-10-30 18:31:40 -07:00
Andrew Gaul 9a0b51a4d4 Include trailing slash in prefix test 2017-10-30 18:31:40 -07:00