Commit Graph

2885 Commits

Author SHA1 Message Date
gurkerl83 fd7fe5c01c Sync OSGi handling with Apache JClouds Project
This project, the aws-lab version of Apache JClouds, share the exact build instructions as the primary Apache JClouds project with all its modules.
Apache JClouds is shifting its strategy in handling OSGi configuration. Instead of using the Maven Bundle Plugin, a wrapper of the BND plugin, the BND plugin gets used directly.
- Remove the OSGi configuration from each module. The configuration gets served to the BND through dedicated configuration / bnd files.
- Onboard bnd-configuration files, one per module.

Ignore bnd files in rat plugin
2020-12-07 09:30:29 +09:00
gurkerl83 9215bfcb70 In the final state of this feature, a rebase on Master was executed. In resolving a merge problem with the Maven project file "JClouds Project," an important instruction got overwritten, to generate test jars for each module. This modification re-adds this ability for all modules. Counter versa, defining this build step repeatedly, e.g., in the api/oauth module, is no longer required. Also, correct a typo, add groupId.
Note: Previously, the maven jar plugin contained a configuration embedded in each module's generated manifest files. The configuration got relocated to the project/bnd.bnd file in a previous commit, and gets handled through the bnd plugin.
2020-10-26 19:58:41 +09:00
gurkerl83 32f6c4d50f Remove the OSGi configuration from each module. The approach of defining OSGi configuration through common properties and serving them to the bundle plugin gets no longer used; instead, OSGi configuration gets defined in each module's dedicated bnd file. 2020-10-26 19:58:41 +09:00
gurkerl83 7a9cd345a6 Onboard bnd-configuration files, one per module 2020-10-26 19:58:41 +09:00
gurkerl83 d82868cc47 Replace embedded and repackaged GSON library
Replace substituted GSON package names with those provided from the vendor.
Reduce OSGi-metadata declaration of core-module because the artificial package org.jclouds.json.gson.internal was removed.
Remove the Gson module its children Gson bundle, and Gson shaded.
Remove duplication conflict and check-style rules due to the removal of the internal Gson module.
Add maven repository where a custom version of the Gson library gets hosted, which exports all packages.

Remove particular repository

Remove the declaration of the repository that serves a custom build GSON version. The build uses GSON in its original form of the vendor, which gets distributed through the standard distribution channel. The identifiers for group, artifact, and version correspond to the latest stable release of GSON.

Integrate GSON library in Clouds Core Bundle

The change contained in the commit puts the GSON library into the classpath of the JClouds core module.
After several tests with Karaf and Karaf JClouds, especially if the Maven identifier matches the original GSON library, there are only a limited number of ways to keep the deployment effort low.

Specifically, Karaf has a set of predefined Maven repositories that can be easily customized. The order in which a particular repository is resolved into the customized GSON library is more difficult. In normal OSGi applications, which do not have such a management function, I imagine this configuration to be more complicated.  Sure, a unique identifier would help, but then we are back to step 1.

Although I honestly don't like to see this kind of approach in a codebase I'm working with, there are not many alternatives to the main aspect of deployment mentioned above.

Maybe the approach can still ease the problem in the short term. In a further mid-term step, however, this problem must be addressed in general.
2020-10-26 19:58:41 +09:00
Andrew Gaul 1cd28c93c4 Remove unintended executable permissions
Fixed via:

find -executable -not -type d -name \*.java -exec chmod -x {} \;
2020-10-19 13:13:34 +09:00
Tamas Cservenak d65047c87b
JCLOUDS-1552: Do not attempt to parse empty payload (#82) 2020-09-21 15:46:25 +02:00
Andrew Gaul 3e25b835c6 JCLOUDS-1333: Fix Guava 21 issues 2020-06-25 19:29:06 +09:00
Sam Ottenhoff 8762fbaf8e JCLOUDS-1473 add INTELLIGENT_TIERING enum 2020-06-25 09:11:33 +09:00
Andrew Gaul 62767a1461 JCLOUDS-1333: JCLOUDS-1334: JCLOUDS-1470: Require Java 8 and Guava 22
This allows compatibility with Guava 29.  Also unwind some older
workarounds.
2020-06-25 08:11:30 +09:00
ikky888 69ca45720d
JCLOUDS-1541: Add Middle East (Bahrain) region to the AWS EC2 and S3 providers list 2020-04-04 10:48:01 +09:00
Colm O hEigeartaigh 99ef891e76 JCLOUDS-1540 - Update Snakeyaml 2020-03-03 17:41:33 +09:00
Ignasi Barrera f5b29c7028 Next development version 2.3.0-SNAPSHOT 2019-10-21 10:32:43 +02:00
Ignasi Barrera 7221844fac Apache jclouds 2.2.0-rc1 release 2019-10-21 10:32:43 +02:00
Ignasi Barrera 644fb4a511 Next development version 2.3.0-SNAPSHOT 2019-10-13 11:00:35 +02:00
Ignasi Barrera 0aef8ca79f Apache jclouds 2.2.0-rc1 release 2019-10-13 11:00:06 +02:00
Ignasi Barrera a2cbdd3385 Etag is not supported in OSX 2019-10-13 10:32:50 +02:00
Roded Bahat a248697c06 JCLOUDS-1509: read AWS response data with the UTF-8 charset explicitly
AWS response data is encoded in UTF-8. Creating a String from said data
using the JVM's default charset results in incorrect encoding on
environments in which the JVM's default charset is not UTF-8.

https://issues.apache.org/jira/browse/JCLOUDS-1509
2019-08-08 15:32:25 +03:00
Ignasi Barrera b8606a10dd
JCLOUDS-1166: Relocate the gson internal packge to be able to keep using it (#35)
* JCLOUDS-1166: Relocate the gson internal packge to be able to keep using it

* Fixes

* Fix import order and shaded jar

* More fixes

* Proper dependency configuration

* Fix typos

* Bring back duplicate exclusions
2019-06-17 17:43:29 +02:00
Andrew Gaul a70ca2d25c Convert a handful of HTTP URLs to HTTPS 2019-05-21 20:52:22 +09:00
Sergi Castro 3e855c26be
Add ARM architecture to ec2 image and the related instance types 2019-05-14 09:00:31 +02:00
Olaf Flebbe 92a3c68838 JCLOUDS-1497: Fix checkstyle-suppressions for jcloud-labs (#27)
* Fix NoWhitespaceBefore Checkstyle Violation

* checkstyle updates: remove suppressionsfilter from checkstyle.xml

* suppressions now done in maven-checkstyle-plugin
2019-05-07 11:49:54 +02:00
Olaf Flebbe d51d6e44bc JCLOUDS-1496: Update maven-compiler-plugin for increased JDK compatibility (#25)
* JCLOUDS-1496: Update maven-compiler-plugin for increased JDK compatibility

* increase maven-compiler-plugin version

* A space change to trigger jenkins again

* Another space change to trigger jenkins again
2019-04-24 16:12:45 +02:00
Olaf Flebbe 2fbf10c9c9 JCLOUDS-1495: maven plugins are not correctly referred to (#24) 2019-04-10 19:54:09 -07:00
Andrew Gaul a1c9ce8217 JCLOUDS-1494: Add S3 Deep Archive tier
Also test Glacier tier now that Amazon allows creating objects with
this storage class.
2019-04-06 18:46:14 +09:00
Dani Estevez 6fda1736ad Adds EU North 1 region (Stockholm) 2019-03-15 12:00:20 -04:00
Dani Estevez 1c9322e41c Adds China North West Region (Ningxia) 2019-03-15 12:00:20 -04:00
Andrew Gaul 8eae27c98d Make interfaces with only statics into classes
This prevents instantiation.  Found via error-prone 2.3.3.
2019-02-23 16:34:11 +09:00
Andrew Gaul 53c47aa0bd Correct uses of checkNotNull
These should provide a descriptive second argument, not the same as
the first argument which is null in the failure case.  This also found
a logic error in CreateVolumeResponseHandler.
2019-02-23 16:33:40 +09:00
Andrew Gaul a4855eb9fd JCLOUDS-1489: expect S3 put archive to succeed 2019-02-07 20:03:14 -08:00
Andrew Gaul 13f32b28c9 Lazily open InputStream during complete MPU
Previously the filesystem provider could exhaust file descriptors by
eagerly opening up to 10,000 parts.  This partially undoes
JCLOUDS-1367.
2019-01-29 21:54:47 -08:00
Andrew Gaul 29eec441e9 JCLOUDS-1371: JCLOUDS-1488: list optimize prefix
Previously getBlobKeysInsideContainer returned all keys and filtered
in LocalBlobStore.  Now getBlobKeysInsideContainer filters via prefix
which can dramatically decrease the number of keys returned,
especially for the filesystem provider.  Further optimizations are
possible for delimiter.
2019-01-29 17:39:51 -08:00
Joe Meiring a36c9dcef0 Fix for FileSystem blob store clearContainer with options 2019-01-04 14:17:00 -08:00
Andrew Gaul f9cebd59f8 Add error handling for missing xattr
Previously this prevented listing inside a directory when using file
systems like HFS and NFS.  References gaul/s3proxy#279.
2019-01-04 11:41:32 -08:00
duc 70f0635ac2 JCLOUDS-1479: populate container cache on create 2019-01-02 19:29:31 -08:00
John McDonnell 25f2bcd8c9 [JCLOUDS-1468] Adding missing options for the ListUsageRecords API call. 2018-12-05 19:46:24 +01:00
kraza 5916059be9 JCLOUDS-1467
Added support for c5d & m5d AWS nodes.

JCLOUDS-1467
Updated the SSD drive information.
2018-12-03 17:22:46 -05:00
Andrew Gaul b7d59e3fe9 Correct precondition
Follow on to 1ae735bb7d.
2018-11-14 14:53:05 -08:00
Andrew Gaul 1ae735bb7d Do not allow options in filesystem clearContainer
This requires some additional logic to clean up empty directories.
Test regression from 22ce5484a4.
2018-11-14 14:42:30 -08:00
Timur Alperovich 896e99df09 Filesystem: Fix the MPU ETags to match S3.
Prior commit introduced a bug in the computation of the MPU ETag value,
where it was concatenating strings, rather than operating on the bytes
of the integer value.
2018-11-02 12:25:17 -07:00
Timur Alperovich 539a9854c1 JCLOUDS-1450: Use S3-style ETags for MPUs.
S3 uses a different ETag for multipart uploads (MPUs) than regular
objects. The ETag consists of the md5 hash of the concatenated ETags of
individual parts followed by the number of parts (separated by "-").

The patch changes the LocalBlobStore's implementation of
CompleteMultipartUpload to set the S3-style ETag before calling
putBlob() and return that ETag to the caller.

To simplify testing, a new protected method with a default NOOP
implementation is added to the BaseBlobIntegrationTest. It allows
providers to further verify MPUs (i.e. ensuring the correct ETag has
been stored alongside the object).
2018-10-24 13:25:00 -07:00
Trent Schmidt 219e2958d7 JCLOUDS-1460: Adding instances to AWS 2018-10-24 14:42:46 -04:00
Oliver Gondža 8b17dfea87 Make NovaErrorHandler logging of sensitive information configurable 2018-10-18 11:32:04 +02:00
Andrew Gaul cc65957997 Error-prone 2.3.2 fixes 2018-10-12 23:28:00 -07:00
David Currie 7ebf12bf38 JCLOUDS-1447: URL encode x-amz-copy-source
The x-amz-copy-source header on S3 CopyObject should be URL encoded (as
a path). This is not universally true of all headers though (for example
the = in x-amz-copy-source-range) therefore introducing a new parameter
on @Headers to indicate whether URL encoding should take place.
2018-09-12 08:01:43 -07:00
Markus Alexander Kuppe 88f44a6d8e JCLOUDS-1339: Support launching an x1 EC2 instance
https://issues.apache.org/jira/browse/JCLOUDS-1339
2018-08-23 20:25:05 +02:00
Vikas Rangarajan cf67233765 JCLOUDS-1436: Add support for aws ec2 m5 instance types
fix: remove duplicate imports
2018-08-01 17:28:09 -04:00
Vikas Rangarajan 982c6bea02 JCLOUDS-1387: Add support for c5 instance types in AWS ec2 2018-08-01 16:53:06 -04:00
Ignasi Barrera 3839432757
Configure Java 7 language level 2018-07-11 09:42:18 -07:00
Jesse Glick 019ccb7431
Pinning down JCLOUDS-1401 fix behavior with a unit test which is able to distinguish URI.query from URI.rawQuery. 2018-07-11 09:35:34 -07:00
Jesse Glick 9a65be789e
Live test demonstrating effectiveness of JCLOUDS-1401 fix. 2018-07-11 09:35:34 -07:00
Jesse Glick 34d272ac80
Allow S3ClientLiveTest (and AWSS3ClientLiveTest) to use a session token. 2018-07-11 09:35:34 -07:00
Jesse Glick f770455ff9
[JCLOUDS-1401] Properly URL-encode the CanonicalQueryString when it contains funny characters. 2018-07-11 09:35:34 -07:00
Lucia Monte 6b4be0d686 JCLOUDS-1427: Fix Elastic IP deserialization when IP has tags 2018-06-20 15:31:13 +02:00
Ignasi Barrera b76a594e81 JCLOUDS-1421: Add default credentials to images created by the ImageExtension
By default, when listing images the ComputeServiceAdapter adds the
default credentials for each image. This is not done when images are
created by the image extension, and NPEs can appear in code that assumes
the default credentials are there, as the field is not nullable.

This change tries to populate the known node credentials for images
created form nodes, and falls back to the default strategy to add the
default credentials to an image if there are not known credentials.
2018-05-22 16:30:26 +02:00
Mat Mannion 29655f3149 JCLOUDS-1263: Add live integration test for manipulating objects through the Swift ObjectApi with unicode characters in their path 2018-05-21 22:19:25 -07:00
Mat Mannion 14621c08d7 JCLOUDS-1263: URL encode object names from list object response before creating URIs with them
This fix calls urlEncode on the path before appending them to the existing path, as the core UriBuilder
class expects them to be URL-encoded.
2018-05-21 22:19:25 -07:00
Jesse Glick 65a5ec8164 [JCLOUDS-1200] Mark nonserializable fields in HttpResponseException and subtypes as transient. 2018-05-21 22:13:04 -07:00
Alix Lourme 053dfd0134 JCLOUDS-1414: OpenStack Keystone V3 - different auth "domains" support 2018-05-10 10:43:02 +02:00
Andrew Gaul 46a9f574e2 Fix some orphaned format strings
These lack values to format with.  Found via error-prone 2.3.0.
2018-04-19 08:43:46 -07:00
Andrew Gaul 7e0a2dbd13 JCLOUDS-1400: Honor Swift signed URL content type 2018-04-12 22:43:43 -07:00
Andrew Gaul f6d47b9c0e JCLOUDS-1333: Fix Java 8 and Guava 21 issues 2018-04-10 08:17:35 -07:00
Andrew Gaul ea6b616faa JCLOUDS-1399: Support S3 One Zone-IA storage class
Also improve test coverage of storage classes.
2018-04-04 21:44:28 -07:00
Andrew Gaul 90fac3b683 JCLOUDS-1271: Remove BlobStore.signRemoveBlob
Most providers never supported this functionality and the portable
abstraction should not have included it.
2018-03-09 20:52:12 -08:00
andreaturli 7fa2ea613e fix typo in project.version
- from 2.2.0-SNAPSHOST to 2.2.0-SNAPSHOT
2018-02-21 16:57:09 +01:00
andreaturli d17487b223 fix typo in project.version
- from 2.2.0-SNAPSHOST to 2.2.0-SNAPSHOT
2018-02-21 16:45:43 +01:00
andreaturli 2305035e89 Next development version 2.2.0-SNAPSHOST 2018-02-16 11:29:15 +01:00
andreaturli a3799e6630 Apache jclouds 2.1.0-rc3 release 2018-02-16 11:28:53 +01:00
andreaturli a5db565581 Next development version 2.2.0-SNAPSHOST 2018-02-16 11:23:12 +01:00
andreaturli 9273bd56ad Apache jclouds 2.1.0-rc3 release 2018-02-16 11:20:21 +01:00
andreaturli 50ae6828f5 [JCLOUDS-1382] improve usage of Neutron capabilities from Nova
- move Neutron FloatingIP API from extension to feature
- introduce FloatingIpForServer as abstract domain object to represents Nova and Neutron objects
- manage floating ip cache and its invalidation
2018-02-15 18:08:53 +01:00
andreaturli 09936b57fc [JCLOUDS-1377] add support for injectable Neutron Context into Nova
- fix NovaComputeServiceExpectTest
- fix NovaComputeServiceExpectTest
- fix CreateSecurityGroupIfNeededTest
- fix FindSecurityGroupInRegionOrCreateTest
- fix checkstyle
- fix removal from security group cache
- fix listSecurityGroupsForNode
- change both Nova and Neutron listSecurityGroupsForNode to use NovaApi.listSecurityGroupForServer
2018-02-02 16:26:22 +01:00
Ignasi Barrera 3ac9279acb Update pom.xml after promoting Neutron 2018-01-17 08:30:49 +01:00
Ignasi Barrera ef679199a4 Promote OpenStack Neutron 2018-01-17 08:24:48 +01:00
Ignasi Barrera 6fd7c7c38b JCLOUDS-1373: Add eu-west-3 (Paris) AWS region 2018-01-16 14:49:12 +01:00
Ignasi Barrera d50acd5e61 JCLOUDS-1372: Add support for R4 instance types 2018-01-16 12:28:19 +01:00
Ignasi Barrera 4a7b2051f0 Fix build when using Guava 20.0 2018-01-16 11:21:30 +01:00
Ignasi Barrera f0c248f19e Add support for Keystone V3 2018-01-16 09:08:06 +01:00
Ignasi Barrera dd73410d12 Add support for OpenStack Keystone V3
Refactors the Keystone Authentication and Service Catalog classes to a
common model that can be used by V2 and V3 of Keystone. Each version
will have their own Authentication APIs and Service Catalog Suppliers,
and the higher level Keystone Authentication will transparently delegate
to the right API based on the keystone-version property.

OpenStack APIs will just have to define the default keystone-version
property they work with, and declare the generic Keystone Authentication
and Service Catalog modules.

Co-authored-by: Andrea Turli <andreaturli@apache.org>
2018-01-16 09:06:30 +01:00
Andrew Gaul 18eb7f3d38 Handle "file" and "file/" collision
Fixes gaul/s3proxy#240.
2018-01-09 22:26:53 -08:00
Andrew Gaul 5ca4827d1b JCLOUDS-1370: Add CannedAccessPolicy constants
Also use CaseFormat instead of extra logic.
2018-01-09 13:06:47 -08:00
Andrew Gaul 52c92a9eb5 Propagate access denied when creating container
Previously this method only reported whether it created a container or
not and callers could not determine whether there was an error or if
the container already existed.  References gaul/s3proxy#122.
2018-01-09 12:44:24 -08:00
andreaturli 64d84c12b0 [Openstack-Neutron] move SecurityGroupApi from extensions to features 2017-12-12 09:47:11 +01:00
Ignasi Barrera 624367dfbf OAuth filter customization per request 2017-11-30 09:07:20 +01:00
andreaturli ec03b710a7 [JCLOUDS-1360] add support for block device mappings to Nova 2017-11-22 11:29:49 +01:00
Andrew Gaul 9b59d099d1 JCLOUDS-1358: Handle 0-byte input with v4 signature
Previously this failed due to use of mark and reset on the
InputStream.
2017-11-21 11:36:11 -08:00
Andrew Gaul 73c3b6024c Add exception cause 2017-11-20 10:49:55 -08:00
Andrew Gaul a62494a117 JCLOUDS-1110: Inject timeStamp into Atmos provider
This addresses a unit test failure.
2017-11-14 10:51:58 -08:00
Andrew Gaul fda87d7dc7 JCLOUDS-1110: Atmos time-limited signed URLs 2017-11-13 14:44:02 -08:00
Andrew Gaul 3923fb0d71 JCLOUDS-598: Populate additional MD5 field
Follow-on to a6337fdcc4.
2017-11-13 09:21:19 -08:00
Andrew Gaul 2df35de45c Remove nonsense check for non-existent URL
This URL does not resolve and slows integration tests.
2017-11-12 17:51:49 -08:00
Andrew Gaul a6337fdcc4 JCLOUDS-598: Populate Atmos MD5 from wschecksum 2017-11-12 17:51:49 -08:00
Andrew Gaul 50026c8f2d Return Atmos objectID as ETag
Previously Atmos returned null.  Also rework the fix for JCLOUDS-339
which does not reproduce with AT&T Synaptic.  Fixes gaul/s3proxy#247.
2017-11-11 14:20:49 -08:00
Andrew Gaul 8b94febfeb Throw consistent exception on invalid range read
Previously S3, Swift, and local blobstores threw a generic
IllegalArgumentException for this uncommon error.  Instead
consistently throw HttpResponseException.
2017-11-10 21:39:47 -08:00
Andrew Gaul 855ac7776c JCLOUDS-1080: remove spurious parameter from test
Follow on to 52044d23d9.
2017-11-07 11:49:00 -08:00
Andrew Gaul 52044d23d9 JCLOUDS-1080: request JSON only for list objects 2017-11-07 10:58:09 -08:00
Andrew Gaul 9dc85ecaab JCLOUDS-1281: Improve Swift DLO tests
Add DynamicLargeObjectApiLiveTest to Rackspace Cloud Files suite and
use correct regionId so that tests pass.  Also remove duplicative
test.
2017-11-07 10:50:10 -08:00
Archana Chinnaiah eaf3c779dc JCLOUDS-1281: Swift dynamic large objects 2017-11-07 10:50:10 -08:00
Andrew Gaul b2ced53e16 JCLOUDS-1111: Overwrite objects via remove and put
This partially reverts commit
e446b5b8b4.  AT&T Synaptic returns a
bogus error on with x-emc-force-overwrite:

HTTP 400, code=1012, message=There was a mismatch between the object
size and the specified extent size.
2017-11-06 18:05:47 -08:00
Andrew Gaul 46759f8bda Fix a few Coverity defects 2017-11-05 20:12:47 -08:00