Commit Graph

2737 Commits

Author SHA1 Message Date
Andrew Gaul ea6b616faa JCLOUDS-1399: Support S3 One Zone-IA storage class
Also improve test coverage of storage classes.
2018-04-04 21:44:28 -07:00
Andrew Gaul 90fac3b683 JCLOUDS-1271: Remove BlobStore.signRemoveBlob
Most providers never supported this functionality and the portable
abstraction should not have included it.
2018-03-09 20:52:12 -08:00
andreaturli d17487b223 fix typo in project.version
- from 2.2.0-SNAPSHOST to 2.2.0-SNAPSHOT
2018-02-21 16:45:43 +01:00
andreaturli a5db565581 Next development version 2.2.0-SNAPSHOST 2018-02-16 11:23:12 +01:00
andreaturli 9273bd56ad Apache jclouds 2.1.0-rc3 release 2018-02-16 11:20:21 +01:00
andreaturli 50ae6828f5 [JCLOUDS-1382] improve usage of Neutron capabilities from Nova
- move Neutron FloatingIP API from extension to feature
- introduce FloatingIpForServer as abstract domain object to represents Nova and Neutron objects
- manage floating ip cache and its invalidation
2018-02-15 18:08:53 +01:00
andreaturli 09936b57fc [JCLOUDS-1377] add support for injectable Neutron Context into Nova
- fix NovaComputeServiceExpectTest
- fix NovaComputeServiceExpectTest
- fix CreateSecurityGroupIfNeededTest
- fix FindSecurityGroupInRegionOrCreateTest
- fix checkstyle
- fix removal from security group cache
- fix listSecurityGroupsForNode
- change both Nova and Neutron listSecurityGroupsForNode to use NovaApi.listSecurityGroupForServer
2018-02-02 16:26:22 +01:00
Ignasi Barrera 3ac9279acb Update pom.xml after promoting Neutron 2018-01-17 08:30:49 +01:00
Ignasi Barrera ef679199a4 Promote OpenStack Neutron 2018-01-17 08:24:48 +01:00
Ignasi Barrera 6fd7c7c38b JCLOUDS-1373: Add eu-west-3 (Paris) AWS region 2018-01-16 14:49:12 +01:00
Ignasi Barrera d50acd5e61 JCLOUDS-1372: Add support for R4 instance types 2018-01-16 12:28:19 +01:00
Ignasi Barrera 4a7b2051f0 Fix build when using Guava 20.0 2018-01-16 11:21:30 +01:00
Ignasi Barrera f0c248f19e Add support for Keystone V3 2018-01-16 09:08:06 +01:00
Ignasi Barrera dd73410d12 Add support for OpenStack Keystone V3
Refactors the Keystone Authentication and Service Catalog classes to a
common model that can be used by V2 and V3 of Keystone. Each version
will have their own Authentication APIs and Service Catalog Suppliers,
and the higher level Keystone Authentication will transparently delegate
to the right API based on the keystone-version property.

OpenStack APIs will just have to define the default keystone-version
property they work with, and declare the generic Keystone Authentication
and Service Catalog modules.

Co-authored-by: Andrea Turli <andreaturli@apache.org>
2018-01-16 09:06:30 +01:00
Andrew Gaul 18eb7f3d38 Handle "file" and "file/" collision
Fixes gaul/s3proxy#240.
2018-01-09 22:26:53 -08:00
Andrew Gaul 5ca4827d1b JCLOUDS-1370: Add CannedAccessPolicy constants
Also use CaseFormat instead of extra logic.
2018-01-09 13:06:47 -08:00
Andrew Gaul 52c92a9eb5 Propagate access denied when creating container
Previously this method only reported whether it created a container or
not and callers could not determine whether there was an error or if
the container already existed.  References gaul/s3proxy#122.
2018-01-09 12:44:24 -08:00
andreaturli 64d84c12b0 [Openstack-Neutron] move SecurityGroupApi from extensions to features 2017-12-12 09:47:11 +01:00
Ignasi Barrera 624367dfbf OAuth filter customization per request 2017-11-30 09:07:20 +01:00
andreaturli ec03b710a7 [JCLOUDS-1360] add support for block device mappings to Nova 2017-11-22 11:29:49 +01:00
Andrew Gaul 9b59d099d1 JCLOUDS-1358: Handle 0-byte input with v4 signature
Previously this failed due to use of mark and reset on the
InputStream.
2017-11-21 11:36:11 -08:00
Andrew Gaul 73c3b6024c Add exception cause 2017-11-20 10:49:55 -08:00
Andrew Gaul a62494a117 JCLOUDS-1110: Inject timeStamp into Atmos provider
This addresses a unit test failure.
2017-11-14 10:51:58 -08:00
Andrew Gaul fda87d7dc7 JCLOUDS-1110: Atmos time-limited signed URLs 2017-11-13 14:44:02 -08:00
Andrew Gaul 3923fb0d71 JCLOUDS-598: Populate additional MD5 field
Follow-on to a6337fdcc4.
2017-11-13 09:21:19 -08:00
Andrew Gaul 2df35de45c Remove nonsense check for non-existent URL
This URL does not resolve and slows integration tests.
2017-11-12 17:51:49 -08:00
Andrew Gaul a6337fdcc4 JCLOUDS-598: Populate Atmos MD5 from wschecksum 2017-11-12 17:51:49 -08:00
Andrew Gaul 50026c8f2d Return Atmos objectID as ETag
Previously Atmos returned null.  Also rework the fix for JCLOUDS-339
which does not reproduce with AT&T Synaptic.  Fixes gaul/s3proxy#247.
2017-11-11 14:20:49 -08:00
Andrew Gaul 8b94febfeb Throw consistent exception on invalid range read
Previously S3, Swift, and local blobstores threw a generic
IllegalArgumentException for this uncommon error.  Instead
consistently throw HttpResponseException.
2017-11-10 21:39:47 -08:00
Andrew Gaul 855ac7776c JCLOUDS-1080: remove spurious parameter from test
Follow on to 52044d23d9.
2017-11-07 11:49:00 -08:00
Andrew Gaul 52044d23d9 JCLOUDS-1080: request JSON only for list objects 2017-11-07 10:58:09 -08:00
Andrew Gaul 9dc85ecaab JCLOUDS-1281: Improve Swift DLO tests
Add DynamicLargeObjectApiLiveTest to Rackspace Cloud Files suite and
use correct regionId so that tests pass.  Also remove duplicative
test.
2017-11-07 10:50:10 -08:00
Archana Chinnaiah eaf3c779dc JCLOUDS-1281: Swift dynamic large objects 2017-11-07 10:50:10 -08:00
Andrew Gaul b2ced53e16 JCLOUDS-1111: Overwrite objects via remove and put
This partially reverts commit
e446b5b8b4.  AT&T Synaptic returns a
bogus error on with x-emc-force-overwrite:

HTTP 400, code=1012, message=There was a mismatch between the object
size and the specified extent size.
2017-11-06 18:05:47 -08:00
Andrew Gaul 46759f8bda Fix a few Coverity defects 2017-11-05 20:12:47 -08:00
Richard Downer 98c289721c Recognise CentOS images on AWS Marketplace
CentOS’ officially-supported AMIs are hosted on the AWS Marketplace.
This adds support for those images, recognising the AMI naming
convention and ensuring the OS metadata is parsed correctly and the
correct SSH login name is used.

There is no change to the default jclouds configuration and the
official CentOS images will not be detected by default. To use these
images, you must alter the ami-query properties to include searching
the “AWS Marketplace”, which has an owner ID of 679593333241. You must
also manually log on to the AWS Marketplace, select your chosen CentOS
image, and “subscribe” to it (you can do this by proceeding as if to
launch an image, but stopping after you have agreed to the subscription
and before launching).
2017-11-03 10:23:37 +01:00
Andrew Gaul c0a7938b69 Address Checkstyle violation 2017-10-30 18:31:40 -07:00
Andrew Gaul af05e24ae9 Implement partial prefix support for Atmos
Atmos only supports listing by directories while other blobstores
allow listing via arbitrary prefixes.  Allow requests which list
directories via both prefix and delimiter = "/" to succeed instead of
failing all requests.  Also change a test which specified recursive to
instead be delimiter = "/".  Fixes gaul/s3proxy#244.
2017-10-30 17:51:35 -07:00
Andrew Gaul 0960ea4969 Add mtime support to Atmos directory listing 2017-10-29 17:40:25 -07:00
Andrew Gaul 4b10bcc054 Enable passing Atmos tests
Also clarify why we skip certain tests with SkipException.
2017-10-29 13:43:10 -07:00
Andrew Gaul 29397b516b Add parent to system metadata
Previously this caused spurious test failures.
2017-10-28 11:06:38 -07:00
Andrew Gaul cf2c077836 Handle empty prefix on Atmos 2017-10-28 11:06:25 -07:00
Ignasi Barrera f7b74d95c9 Exclude tier tests on OSX 2017-10-25 08:30:49 +02:00
Andrew Gaul c901bf3a0f JCLOUDS-1337: Swift putBlob portable storage tiers 2017-10-23 21:31:49 -07:00
Andrew Gaul f513bf7c40 JCLOUDS-1337: Atmos putBlob portable storage tiers 2017-10-23 21:31:48 -07:00
Andrew Gaul b93270aef2 JCLOUDS-1337: only set fs tier when non-null
Fixes regression from 61d4be8746.
2017-10-20 20:51:50 -07:00
Andrew Gaul 5448b92ee8 JCLOUDS-1337: only set fs tier when non-null
Fixes regression from 61d4be8746.
2017-10-17 23:14:09 -07:00
Andrew Gaul 89053d9a8b JCLOUDS-1337: S3 putBlob portable storage tiers
Also promote hacky and limited storage class support from aws-s3
provider to s3 api.
2017-10-12 11:16:48 -07:00
Andrew Gaul 61d4be8746 JCLOUDS-1337: filesystem putBlob portable storage tiers 2017-10-12 11:15:14 -07:00
Andrew Gaul 6cf0a63186 Remove stray Javadoc /** 2017-09-27 09:45:35 -07:00
Andrew Gaul 9628a9064e Upgrade to EasyMock 3.4
Also switch from unmaintained easymockclassextension to easymock.
Release notes:

https://github.com/easymock/easymock/releases
2017-09-17 15:45:46 -07:00
Andrew Gaul 6046fc925c Avoid hard-coding rackspace region in swift test 2017-09-09 13:42:59 -07:00
Andrew Gaul 7704893650 Remove Inject on constructors for abstract classes
This is not meaningful since these classes cannot be instantiated.
Found via error-prone.
2017-09-07 23:37:24 -07:00
Ignasi Barrera 1c1f81c373 JCLOUDS-1258: Allow China endpoints in FormSignerV4 2017-09-08 08:12:50 +02:00
Timur Alperovich 6feacb1ee5 Fixes for jclouds openstack swift tests.
The ContainerApiLiveTest class has tests that mostly assume that a
single test container always exists. They collide on changes to the
state of the container. Some of the tests, though, assume the
container does _not_ exist and attempt to create it and subsequently
delete it. The change in this patch makes their behavior uniform with
respect to that test container: assume it exists and if the test tries
to delete it or mutate it, delete the container and create it at the
end of the test.

The fix in ObjectApi is for the Object GET requests. Currently,
jclouds passes "format=json", which is not a supported query parameter
for object GET.

Lastly, ObjectApiLiveTest that test server-side copy should ignore the
X-Openstack-Request-ID header, as its value will change between the
two requests.
2017-09-07 13:25:50 -07:00
Andrew Gaul 97c77541ec Allow empty delimiters with openstack-swift 2017-09-07 12:28:41 -07:00
Andrew Gaul 3a1710d15c Default Swift signer timeout to 15 minutes
This matches other providers.
2017-09-07 11:53:54 -07:00
Andrew Gaul b424a6c164 Alphabetize modules 2017-09-07 07:27:56 -07:00
Andrew Gaul fb584d844d Address BindingToUnqualifiedCommonType warnings
Found via error-prone.
2017-09-07 07:08:28 -07:00
Andrew Gaul d7d28fe7d7 JCLOUDS-1333: Correct JDK 1.8 method overloading
Newer JDK have a different resolution process, likely due to enhanced
target type inference.  Found via:

mvn test -Dmaven.compile.source=1.8 -Dmaven.compile.target=1.8
2017-09-07 01:41:52 -07:00
Andrew Gaul c20afbeda6 Make auto service optional
This makes dependencies consistent and eliminates warnings of the
form:

$M2_HOME/repository/org/apache/jclouds/driver/jclouds-slf4j/2.1.0-SNAPSHOT/jclouds-slf4j-2.1.0-SNAPSHOT.jar(org/jclouds/logging/slf4j/config/SLF4JLoggingModule.class): warning: Cannot find annotation method 'value()' in type 'AutoService': class file for com.google.auto.service.AutoService not found

Reference:

https://github.com/google/auto/tree/master/service#download
2017-08-30 12:27:08 -07:00
Andrew Gaul bfd54049b7 Make auto service optional
This makes dependencies consistent and eliminates warnings of the
form:

$M2_HOME/repository/org/apache/jclouds/driver/jclouds-slf4j/2.1.0-SNAPSHOT/jclouds-slf4j-2.1.0-SNAPSHOT.jar(org/jclouds/logging/slf4j/config/SLF4JLoggingModule.class): warning: Cannot find annotation method 'value()' in type 'AutoService': class file for com.google.auto.service.AutoService not found

Reference:

https://github.com/google/auto/tree/master/service#download
2017-08-30 12:23:49 -07:00
Andrew Gaul a4e38f7c84 Prefer Guava BaseEncoding over JAXB 2017-08-30 11:26:15 -07:00
Andrew Gaul 6bb9c927bb JCLOUDS-1101: Use correct HttpException
Importing javax.xml.ws.http.HTTPException causes Java 9 issues.
2017-08-30 09:50:58 -07:00
Andrew Gaul df43b36487 Avoid injecting on final fields
This can cause thread visibility issues.  Found via error-prone.
2017-08-28 21:08:59 -07:00
Andrew Gaul 6b8a81335d JCLOUDS-1225: Address Guava 18 Objects changes
Fixed with:

find -name \*.java | xargs sed -i 's/Objects.[Tt]oStringHelper/More&/g'
find -name \*.java | xargs sed -i 's/Objects.firstNonNull/More&/g'
find -name \*.java | xargs sed -i 's/^\(import com.google.common.base.\)\(Objects.*\)/\1More\2\n\1\2/g'
find -name \*.java | xargs java -jar google-java-format-1.3-all-deps.jar -i --fix-imports-only --skip-sorting-imports
2017-08-24 18:31:07 -07:00
Andrew Gaul 517323fbd5 JCLOUDS-1225: Address Guava 18 MoreExecutors changes
Fixed with:

find -name \*.java | xargs sed -i 's/sameThreadExecutor/newDirectExecutorService/g'
2017-08-22 17:14:22 -07:00
Andrew Gaul a493e2ca2f JCLOUDS-1225: Address Guava 18 Objects changes
Fixed with:

find -name \*.java | xargs sed -i 's/Objects.[Tt]oStringHelper/More&/g'
find -name \*.java | xargs sed -i 's/Objects.firstNonNull/More&/g'
find -name \*.java | xargs sed -i 's/^\(import com.google.common.base.\)\(Objects.*\)/\1More\2\n\1\2/g'
find -name \*.java | xargs java -jar google-java-format-1.3-all-deps.jar -i --fix-imports-only --skip-sorting-imports
2017-08-22 17:12:53 -07:00
Chaithanya Ganta e331a000d1 JCLOUDS-1322: Remove Expect header for requests with empty body
Signed-off-by: Chaithanya Ganta <ganta@adobe.com>
2017-08-21 21:05:02 -07:00
Andrea Turli 7c58f9d77d JCLOUDS-1323: use security group names in openstack nova options
- fix NovaComputeServiceExpectTest
2017-08-09 12:25:53 +02:00
Andrea Turli c61fa9d02b fix NovaComputeServiceExpectTest 2017-08-09 11:15:59 +02:00
Andrew Gaul 3e1e9cbab6 Remove duplicate Nullable import 2017-08-06 18:38:52 -07:00
Andrew Gaul 3ed99ad74b Allow lastModified to be null
Follow on to ec932321bd
2017-08-06 17:38:36 -07:00
Andrea Turli aa11765bee [JCLOUDS-1318] fix based on nodeTerminatePredicate
- wait for serer deletion, before deleting the security group
- rename cleanupServer to cleanupResources
- remove keyPairCache
- better usage of tags to remove securityGroups created by jclouds
- remove keyPair after the creation of a group
- remove test for create unique keypair
- openstack nova re-adding support for existing keypair
- fix securityGroupApi check
- fix other unit tests
- remove ServerPredicates as it is now duplicated
- remove TemplateOptions.securityGroupNames as deprecated
- address commits for ApplyNovaTemplateOptionsCreatesNodesWithGroupEncodedIntoNameAndAddToSet
- fix testCreateNodeWhileUserSpecifiesKeyPairAndUserSpecifiedGroups
2017-07-17 11:32:24 +02:00
Andrew Gaul 0bc935dd57 Remove clojure bindings
These have not seen any development in many years.
2017-07-10 11:39:11 -07:00
Andrew Gaul e446b5b8b4 JCLOUDS-1111: Force overwriting Atmos objects
This avoids a racy delete then write in the portable abstraction.
2017-07-08 15:50:04 -07:00
Svetoslav Neykov 624e4cb6af
JCLOUDS-1307: Invalidate SG on removal even if already externally deleted 2017-06-27 13:32:38 +03:00
Svetoslav Neykov 1d4cb6c392 [JCLOUDS-1306] Fix SG cache invalidation when deleting 2017-06-09 12:04:29 +02:00
Geoff Macartney 28c3c33bf0 Temporarily use a custom annotation instead of @SinceApiVersion.
The intention is to use @SinceApiVersion for this purpose, but that
would affect a number of APIs, and we would want to have good test
coverage before merging that change (in
FormSignerUtils#getAnnotatedApiVersion). However, there is some issue
wth certain tests at resent that means we cannot successfully test
all APIs that make use of @SinceApiVersion in order to assure
ourselves that FormSignerUtils will not introduce some problem.

See https://github.com/jclouds/jclouds/pull/1102#issuecomment-302682049
for details.

This annotation is introduced as a temporary measure in order to
decouple the functionality of FormSignerUtils#getAnnotatedApiVersion
from @SinceApiVersion and the tests in question. It can be removed and
replaced by @SinceApiVersion when those tests are fixed.

Designates that a method overrides the {@link ApiVersion} on the class
with a specific value.
2017-05-29 10:24:52 +02:00
Geoff Macartney ce0a0ad213 Add ModifySubnetAttribute 2017-05-29 10:24:52 +02:00
Andrew Gaul 2b49bf3618 JCLOUDS-1299: Handle % in names during Swift list
uriBuilder.appendPath assumes an encoded path.
2017-05-16 17:29:02 -07:00
ui4j a299a6c1fd JCLOUDS-1297: Fix uploadId format for non-English locale
Add Locale.ENGLISH parameter to String.format(...) to generate same uploadId for different locales.
2017-05-15 20:01:40 +03:00
Stephen Tomkinson 89ae3b4fa6 Add logic from JCLOUDS-1261 to the other 2 types of Aws4 signer. 2017-05-10 10:09:22 -07:00
Tomas Tulka 6452960c72 fix for deleting an invalid file
If the file for delete is invalid (typically a wrong filename) an IOException will be thrown.
2017-05-10 01:02:09 -07:00
Stephen Tomkinson 15d27da739 Resolves JCLOUDS-1261 by ensuring non-standard port numbers are in the host header that's used for the AWSv4 auth calculations. 2017-05-10 00:00:14 -07:00
Andrea Turli 1c6c36b81a JCLOUDS-1269; JCLOUDS-1120: ec2 subnet/securityGroups fixes 2017-05-04 17:40:50 +02:00
Aled Sage fe134cf612 Fix logging in nova: don’t assume logback 2017-05-04 16:48:15 +02:00
Andrew Gaul 2e7de9f850 JCLOUDS-1271: Deprecate signRemoveBlob
URL signing should always use authentication parameters instead of
headers yet no provider supports DELETEs with the former.
2017-04-19 18:08:52 -07:00
Andrew Gaul db00f679ef JCLOUDS-255: Consistently use query string auth
Fixes regressions introduced in
7a110b31ba.
2017-04-19 00:54:33 -07:00
Andrew Gaul 7a110b31ba JCLOUDS-255: Support S3 signed URLs with expiry
Mostly code motion from AWSS3BlobRequestSigner to S3BlobRequestSigner
with some additional cleanups.
2017-04-18 18:51:22 -07:00
John McDonnell 24f961eac2 [JCLOUDS-1128] Added page and pagesize fields to ListProjectOptions.
Cloudstack's listProjects request supports additional fields for page
and pagesize.  These were missing but have been added in as part of this
commit.
2017-04-18 15:05:41 +02:00
John McDonnell 723a732521 [JCLOUDS-1130] Cloudstack fixed projectid parameter in listUsageRecords.
Cloudstack's listUsageRecords response doesn't use camelcase so the projectid
parameter was incorrect.  Updated from projectId to projectid.
2017-04-18 15:05:34 +02:00
Andrew Gaul 5a1c95525f JCLOUDS-1262: Correct US-OH region
Follow on to 04ab255d9f.
2017-04-10 13:11:38 -07:00
Andrew Gaul 93a8ce433b Allow null S3 list MPU lastModified 2017-04-10 12:09:02 -07:00
Mat Mannion d41101df59 JCLOUDS-1264: Swift Unicode multipart manifests
This fixes a bug where previously BindManifestToJsonPayload used the
character length as the ContentLength, instead of the byte length,
which caused issues if the JSON contained multi-byte Unicode
characters.
2017-04-04 05:05:53 -07:00
Ignasi Barrera 04ab255d9f JCLOUDS-1262: Add the Ohio region to AWS 2017-03-29 09:17:41 +02:00
Michelle Zhang 96a1c4427e JCLOUDS-1217/JCLOUDS-1258: Add Canada, London and China regions to AWS 2017-03-29 08:53:07 +02:00
Andrew Gaul 01780db4db Remove erroneous static final from BIG_FILE
Also use preferred bigFile capitalization for member.  Fixes
regression from e058973abc.
2017-03-17 17:41:32 -07:00
Andrew Gaul e058973abc Ensure SOME_CONSTANTS are static final
Found via error-prone.
2017-03-17 16:59:41 -07:00
Andrew Gaul 7767acbec9 Correct invalid comparison
Found via error-prone.
2017-03-17 16:42:38 -07:00