Commit Graph

9651 Commits

Author SHA1 Message Date
Andrew Gaul ffa7c5c587 Update via canned ACLs instead of XML ACLs for S3
These are simpler than the full XML API and better supported by
non-AWS S3 implementations, e.g., Ceph, S3Proxy.  Further this makes
the provider more consistent when creating a bucket or object which
only supports setting canned ACLs.
2015-12-19 19:28:03 -08:00
Andrew Gaul c2ba0bef32 Allow updating S3 canned ACLs
These are simpler than the full XML API and better supported by
non-AWS S3 implementations, e.g., Ceph, S3Proxy.
2015-12-19 19:28:03 -08:00
Andrew Gaul 13701bf7ab Do not test marker contents
Some providers like Azure have opaque markers.
2015-12-19 15:24:12 -08:00
Andrew Gaul 4c8f758e3a Loosen restrictions for S3 bucket naming
US Standard and most S3 clones allow more bucket names than DNS
allows.
2015-12-19 15:10:01 -08:00
Andrew Gaul 750c5d5316 JCLOUDS-1042: Remove unneeded ACL swizzling
The previous commit shows that we do not need this.
2015-12-18 23:28:39 -08:00
Andrew Gaul a7c66d5120 JCLOUDS-1042: Test reading from public container
Blob has default access but container is public-read.
2015-12-18 23:28:39 -08:00
Andrew Gaul cc6d45f6d4 Upgrade to testng 6.8.21
Note that this is the last version compatible with Java 6.  Changelog:

https://github.com/cbeust/testng/blob/master/CHANGES.txt
2015-12-18 16:29:34 -08:00
Zack Shoylev 7d73a0392a Adds nova extension names and an example test 2015-12-16 11:35:09 -06:00
Ignasi Barrera 962de83e86 JCLOUDS-1050: Fix BootstrapConfig deserialization in Chef 2015-12-16 16:05:09 +01:00
Zack Shoylev 415a8a6600 Allows extensions to fall back on using an optional name in keystone/openstack 2015-12-14 10:24:59 -06:00
Andrew Gaul b64d05abb9 Allow setting content metadata during copyBlob
Also remove unneeded call to setBlobMetadata which copy sets.
2015-12-13 22:33:08 -08:00
Andrew Gaul 85bc4a6b7e Fix Azure setBlobProperties
Also backfill test.
2015-12-13 22:24:53 -08:00
Andrew Gaul 0634da98a7 Return ETag from AzureBlobClient.copyBlob
This avoids a call to getBlobProperties in AzureBlobStore.copyBlob.
2015-12-13 18:12:57 -08:00
Andrew Gaul bc41e96c49 Use correct package name for CloudFiles tests
Found via error-prone.
2015-12-13 04:27:40 -08:00
Andrew Gaul b47c50b7a5 Correct malformed format strings
Found via error-prone.
2015-12-12 08:53:08 +08:00
Andrew Gaul 28d090139d Make inner classes static when possible
Found via error-prone.
2015-12-12 08:53:07 +08:00
Andrew Gaul 06e379d7a5 Remove @Nullable from methods returning primitives
Found via error-prone.
2015-12-12 08:53:07 +08:00
Graeme-Miller 75e2c6999c Instead of setting up static NAT for all networks, we setup static NAT for default network only 2015-12-11 16:39:20 +01:00
Andrew Gaul 4abd5dd16f Revert "Delete parts when removing Swift multipart objects"
This reverts commit 3c1588527d.
2015-12-08 10:08:22 +08:00
Josef Cacek 0fb1b459a6 JCLOUDS-1044 fix handling NULL JsonTokens in adapters under NullFilteringTypeAdapterFactories class 2015-12-05 22:45:24 +01:00
Josef Cacek 4d899cacae JCLOUDS-1045 remove swift dependency from allblobstore module 2015-12-03 02:18:22 +01:00
Ignasi Barrera 90423d7b9b Fix filesystem api in OSX 2015-11-24 15:38:10 +01:00
Ignasi Barrera 86dadc3a75 JCLOUDS-1038: Fix date deserialization in EC2 apis 2015-11-24 14:53:18 +01:00
Zack Shoylev 02b79106c3 Fixes tenantId when using the keystone admin extension 2015-11-23 16:03:02 -06:00
Zack Shoylev ab25fc7259 putblob fails when file exists 2015-11-23 16:01:46 -06:00
Ka-Hing Cheung ca70d44143 don't remove directory if it's a blob
the filesystem blob store was always removing a/ when a/b was
removed, even if the client explicitly created a/. this patch
fixes that
2015-11-20 17:27:19 -08:00
Andrew Gaul 2dd231b02b Filter out folders when listing a container
These folders represent directories in the filesystem blobstore which
real blobstores do not have.  Their presence complicates providing the
correct marker for paginated listing.  Future commits will remove
folders entirely.
2015-11-20 16:27:53 -08:00
Andrew Gaul f706d9c13a JCLOUDS-835: Atomically put filesystem blobs
Write to a temporary file and rename to the desired destination to
better match object store semantics.
2015-11-20 14:44:02 -08:00
Andrew Gaul 4ce357ba69 Consistently quote ETags in comparisons
Addresses integration test regression in
25f4807df8.
2015-11-19 19:00:34 -08:00
Andrew Gaul 25f4807df8 Tolerate missing or extra quotes in ETags
References andrewgaul/s3proxy#77.
2015-11-19 17:40:48 -08:00
Andrew Gaul 8d87bfc61b Avoid buffering ByteSource when not needed
This allows range requests of large files to work.  Regression from
79fe91bd03.
2015-11-19 15:10:23 -08:00
Zack Shoylev 2efcb2c5a9 Fixes a windows locale bug with the "Everyone" principal 2015-11-19 11:50:15 -06:00
Reijhanniel Jearl Campos 52c6c2b7b5 JCLOUDS-947: Properly configure live tests in ProfitBricks 2015-11-18 22:35:05 +01:00
Andrew Gaul 3c1588527d Delete parts when removing Swift multipart objects
Swift implements multi-part upload with user-visible parts and an
explicit manifest.  When deleting an MPU blob it can delete only the
manifest or both the manifest and parts.  For consistency with other
providers, we now do the latter in the portable abstraction.  Swift
ignores the multipart-manifest=delete parameter for single-part
objects.  Fixes andrewgaul/s3proxy#92.
2015-11-17 23:06:33 -08:00
Andrew Gaul 721a0dbd3c Add support for standard_ia storage class
This enables storage with lower at-rest prices, higher request prices,
and lower availability.  Also rework existing reduced redundancy
parsing into a more generic storage class.  More background on
standard_ia:

https://aws.amazon.com/blogs/aws/aws-storage-update-new-lower-cost-s3-storage-option-glacier-price-reduction/
2015-11-16 14:04:51 -08:00
Ignasi Barrera 575d39e7c5 JCLOUDS-1033: Don't fail if no private key has been provided in DigitalOcean 2015-11-16 22:58:29 +01:00
Andrew Gaul df48540cb7 Enable FindBugs CheckRelaxingNullnessAnnotation
Also fix up warnings.
2015-11-13 17:44:55 -08:00
Andrew Gaul 22c789da9e JCLOUDS-894: Use prefix when listing MPU on Swift
Previously we only used the blob name and not the blob
name/slo/timestamp/part size cookie which yield extra parts when
listing an MPU with parts from a previous MPU.  Listing using the
stricter prefix gives the expected results.
Fixes andrewgaul/s3proxy#91.
2015-11-13 12:15:29 -08:00
Zack Shoylev 41ce90ec36 Fixes tests failing on windows filesystems Makes windows behavior more consistent, especially for deletes 2015-11-11 21:01:13 -06:00
Ignasi Barrera fb63b0ee61 Remove HPCloud from the aggregator poms 2015-11-10 10:17:30 +01:00
Chris Custine 5d82b40d78 JCLOUDS-1026: Remove public HP cloud compute providers
https://issues.apache.org/jira/browse/JCLOUDS-1026
2015-11-09 19:04:59 -07:00
Ignasi Barrera 34f6d6c7f3 Fix aggregator poms 2015-11-08 23:04:21 +01:00
Andrew Gaul 14e566e151 Upgrade to modernizer-maven-plugin 1.4.0
Release notes:

https://github.com/andrewgaul/modernizer-maven-plugin/releases/tag/modernizer-maven-plugin-1.4.0
2015-11-05 17:09:48 -08:00
Ignasi Barrera 667252d9e8 Fix Google Compute Engine live tests 2015-11-03 00:16:40 +01:00
Timur Alperovich 1fc9b0e259 Move QueryParam encoding to a separate class.
The patch implements a QueryValue class, which encodes the underlying
value based on whether the "encoded" flag is set. This class is used
by the RestAnnotationProcessor to propagate the @Encoded value set on
any parameters.

Since the encoding is now handled by the QueryValue instances, we
should no longer call encodeQueryLine() in the URI builder and instead
call buildQueryLine(). The caveat is that we need to make sure all of
the parameters that may need to be encoded are converted to QueryValue
objects. This is done by converting Object instances to QueryValue by
an instance of the TransformObjectToQueryValue when adding any query
parameters to the URI.
2015-11-02 17:11:42 +01:00
Ignasi Barrera 9df30c5a09 JCLOUDS-717: Join Enterprise and OpenSource Chef 2015-10-30 01:02:53 +01:00
Ignasi Barrera 3fbd399f6b Prefer polling the status of the node 2015-10-29 15:55:23 +01:00
Ignasi Barrera c9306c1fc8 Fix a bunch of aws-ec2 live tests 2015-10-29 01:54:09 +01:00
Ignasi Barrera 985d45122b JCLOUDS-987: Create security groups in AWS if subnetId is specified 2015-10-29 01:54:03 +01:00
Ignasi Barrera afd3c322a9 JCLOUDS-1030: Ignore synthetic members in serialization strategies 2015-10-28 23:15:41 +01:00