Commit Graph

10805 Commits

Author SHA1 Message Date
gurkerl83 d82868cc47 Replace embedded and repackaged GSON library
Replace substituted GSON package names with those provided from the vendor.
Reduce OSGi-metadata declaration of core-module because the artificial package org.jclouds.json.gson.internal was removed.
Remove the Gson module its children Gson bundle, and Gson shaded.
Remove duplication conflict and check-style rules due to the removal of the internal Gson module.
Add maven repository where a custom version of the Gson library gets hosted, which exports all packages.

Remove particular repository

Remove the declaration of the repository that serves a custom build GSON version. The build uses GSON in its original form of the vendor, which gets distributed through the standard distribution channel. The identifiers for group, artifact, and version correspond to the latest stable release of GSON.

Integrate GSON library in Clouds Core Bundle

The change contained in the commit puts the GSON library into the classpath of the JClouds core module.
After several tests with Karaf and Karaf JClouds, especially if the Maven identifier matches the original GSON library, there are only a limited number of ways to keep the deployment effort low.

Specifically, Karaf has a set of predefined Maven repositories that can be easily customized. The order in which a particular repository is resolved into the customized GSON library is more difficult. In normal OSGi applications, which do not have such a management function, I imagine this configuration to be more complicated.  Sure, a unique identifier would help, but then we are back to step 1.

Although I honestly don't like to see this kind of approach in a codebase I'm working with, there are not many alternatives to the main aspect of deployment mentioned above.

Maybe the approach can still ease the problem in the short term. In a further mid-term step, however, this problem must be addressed in general.
2020-10-26 19:58:41 +09:00
Andrew Gaul 1cd28c93c4 Remove unintended executable permissions
Fixed via:

find -executable -not -type d -name \*.java -exec chmod -x {} \;
2020-10-19 13:13:34 +09:00
Andrew Gaul 31a3e5b5df JCLOUDS-1498: Upgrade to Guice 4.2.3
Release notes:

https://github.com/google/guice/wiki/Guice423
2020-09-26 15:48:32 +09:00
Tamas Cservenak d65047c87b
JCLOUDS-1552: Do not attempt to parse empty payload (#82) 2020-09-21 15:46:25 +02:00
Andrew Gaul 3ea2cce5f2 Optimize MultiBlobInputStream.read()
Previously this allocated a byte array for every call.
2020-08-01 18:37:17 +09:00
Andrew Gaul 9c21bf2cc2 JCLOUDS-1367: Do not open open InputStream in copyBlob
This avoids an unneeded call to ByteStreams2.toByteArrayAndClose in
getBlob for range requests and elides a large memory allocation.
2020-08-01 18:37:04 +09:00
Andrew Gaul 8de7b696e1 Store transient Blob data with ByteArrayPayload
This avoids a race condition due to sharing the same Closer instance
and unbounded growth of its Closeable Deque.  References
gaul/s3proxy#303.
2020-07-12 05:13:48 +09:00
Jean-Noël Rouvignac 9e4c7a16da
JCLOUDS-1542 follow-up: check whether isAccessible() is already set (#79)
AccessibleObject.canAccess() would be much much better,
but this API can only be used from Java 9 onward.
2020-07-05 23:33:07 +02:00
Jean-Noël Rouvignac 238c975078
JCLOUDS-1542 Java 11 warns of illegal reflective access (#76)
With Java 11, an illegal reflective access is output for the google cloud storage blobstore.

    WARNING: An illegal reflective access operation has occurred
    WARNING: Illegal reflective access by org.jclouds.reflect.Reflection2$1 (file:/.../jclouds-core.jar) to constructor java.lang.String(char[],int,int,java.lang.Void)
    WARNING: Please consider reporting this to the maintainers of org.jclouds.reflect.Reflection2$1
    WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
    WARNING: All illegal access operations will be denied in a future release

Indeed, JClouds calls `setAccessible(true)` on the package protected constructor
     `java.lang.String(char[],int,int,java.lang.Void)`.
Investigating the code, it turns out it is looking for constructors annotated with any of:
* java.beans.ConstructorProperties
* org.jclouds.json.SerializedNames
* com.google.inject.Inject

But `String` being defined in `java.base` module, it is impossible
     that it will be annotated with any of these annotation.

This commit is complementary to JClouds commit db4e4af931 .

 

Reflection2.java:
Do not call `setAccessible(true)` on core java constructors and methods.

 

For reference, here is the stacktrace of this illegal access warning:

    java.lang.String.<init>(String.java:3208)
    java.lang.String.<init>(String.java:251)
    java.util.StringJoiner.compactElts(StringJoiner.java:250)
    java.util.StringJoiner.toString(StringJoiner.java:173)
    jdk.internal.module.IllegalAccessLogger.loudWarning(IllegalAccessLogger.java:339)
    jdk.internal.module.IllegalAccessLogger.log(IllegalAccessLogger.java:288)
    jdk.internal.module.IllegalAccessLogger.log(IllegalAccessLogger.java:261)
    jdk.internal.module.IllegalAccessLogger.logIfOpenedForIllegalAccess(IllegalAccessLogger.java:226)
    java.lang.reflect.AccessibleObject.logIfOpenedForIllegalAccess(AccessibleObject.java:366)
    java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:325)
    java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
    java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:189)
    java.lang.reflect.Constructor.setAccessible(Constructor.java:182)
    org.jclouds.reflect.Reflection2$1.load(Reflection2$1.java:157)
    org.jclouds.reflect.Reflection2$1.load(Reflection2$1.java:153)
    com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache$LoadingValueReference.java:3529)
    com.google.common.cache.LocalCache$Segment.loadSync(LocalCache$Segment.java:2278)
    com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache$Segment.java:2155)
    com.google.common.cache.LocalCache$Segment.get(LocalCache$Segment.java:2045)
    com.google.common.cache.LocalCache.get(LocalCache.java:3953)
    com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
    com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache$LocalLoadingCache.java:4960)
    org.jclouds.reflect.Reflection2.get(Reflection2.java:346)
    org.jclouds.reflect.Reflection2.constructors(Reflection2.java:100)
    org.jclouds.json.internal.NamingStrategies$AnnotationConstructorNamingStrategy.getDeserializer(NamingStrategies$AnnotationConstructorNamingStrategy.java:271)
    org.jclouds.json.internal.DeserializationConstructorAndReflectiveTypeAdapterFactory.create(DeserializationConstructorAndReflectiveTypeAdapterFactory.java:125)
    com.google.gson.Gson.getAdapter(Gson.java:458)
    org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:117)
    org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:166)
    org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:102)
    com.google.gson.Gson.getAdapter(Gson.java:458)
    com.google.gson.Gson.toJson(Gson.java:696)
    com.google.gson.Gson.toJson(Gson.java:683)
    com.google.gson.Gson.toJson(Gson.java:638)
    com.google.gson.Gson.toJson(Gson.java:618)
    org.jclouds.json.internal.GsonWrapper.toJson(GsonWrapper.java:65)
    org.jclouds.oauth.v2.functions.ClaimsToAssertion.apply(ClaimsToAssertion.java:59)
    org.jclouds.oauth.v2.functions.ClaimsToAssertion.apply(ClaimsToAssertion.java:43)
    org.jclouds.rest.internal.RestAnnotationProcessor.getParamValue(RestAnnotationProcessor.java:829)
    org.jclouds.rest.internal.RestAnnotationProcessor.getFormParamKeyValues(RestAnnotationProcessor.java:847)
    org.jclouds.rest.internal.RestAnnotationProcessor.addFormParams(RestAnnotationProcessor.java:435)
    org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:258)
    org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:137)
    org.jclouds.rest.internal.InvokeHttpMethod.toCommand(InvokeHttpMethod.java:189)
    org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:85)
    org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
    org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
    org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
    org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
    com.sun.proxy.$Proxy49.authorize(Unknown Source)
    org.jclouds.oauth.v2.filters.JWTBearerTokenFlow$AuthorizeToken.load(JWTBearerTokenFlow$AuthorizeToken.java:84)
    org.jclouds.oauth.v2.filters.JWTBearerTokenFlow$AuthorizeToken.load(JWTBearerTokenFlow$AuthorizeToken.java:68)
    com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache$LoadingValueReference.java:3529)
    com.google.common.cache.LocalCache$Segment.loadSync(LocalCache$Segment.java:2278)
    com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache$Segment.java:2155)
    com.google.common.cache.LocalCache$Segment.get(LocalCache$Segment.java:2045)
    com.google.common.cache.LocalCache.get(LocalCache.java:3953)
    com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
    com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache$LocalLoadingCache.java:4960)
    com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache$LocalLoadingCache.java:4966)
    org.jclouds.oauth.v2.filters.JWTBearerTokenFlow.filter(JWTBearerTokenFlow.java:99)
    org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:90)
    org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
    org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
    org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
    org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
    com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:86)
    com.sun.proxy.$Proxy54.simpleUpload(Unknown Source)
    org.jclouds.googlecloudstorage.blobstore.GoogleCloudStorageBlobStore.uploadMultipartPart(GoogleCloudStorageBlobStore.java:422)
    org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:415)
    org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:402)
    com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
    com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
    com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor$Worker.java:628)
    java.lang.Thread.run(Thread.java:834)
2020-06-26 19:55:32 +02:00
Andrew Gaul 3e25b835c6 JCLOUDS-1333: Fix Guava 21 issues 2020-06-25 19:29:06 +09:00
Sam Ottenhoff 8762fbaf8e JCLOUDS-1473 add INTELLIGENT_TIERING enum 2020-06-25 09:11:33 +09:00
Andrew Gaul 49a54ea9ca Disable doclint during Jenkins build
Now that we require Java 8 we do not need this workaround.
2020-06-25 08:37:25 +09:00
Andrew Gaul 62767a1461 JCLOUDS-1333: JCLOUDS-1334: JCLOUDS-1470: Require Java 8 and Guava 22
This allows compatibility with Guava 29.  Also unwind some older
workarounds.
2020-06-25 08:11:30 +09:00
Jean-Noël Rouvignac 20b1394f36 JCLOUDS-1491 Jclouds uses a deprecated version of Guava to support Azure storage
DnsNameValidator.java uses a deprecated guava APIs in code that is used
 to support Azure cloud storage. When forcing the use of more recent guava
 versions, the code fails with NoSuchFieldError.

However, CharMatcher.JAVA_LETTER_OR_DIGIT has been removed in guava 26.0,
 and CharMatcher.javaLetterOrDigit() should be used instead since guava
 19.0. Note that CharMatcher.javaLetterOrDigit() was immediately
 deprecated in Guava 26.0, and java.lang.Character.isLetterOrDigit(int)
 should be used instead.

This commit replaces the use of this deprecated API by
 java.lang.Character.isLetterOrDigit(int).
 It is no worse than the previous code.

(If I understand correctly, updating the guava version is a challenge due to
dependencies on Apache Karaf anyway)
2020-06-03 09:14:41 +09:00
Andrew Gaul 6e6f8ebf77 JCLOUDS-912: JCLOUDS-1547: GCS InputStream single-part upload
Previously this provider worked around a RestAnnotationProcessor quirk
by using multi-part uploads for InputStream payloads.  Instead work
around the quirk another way which allows a single-part upload.  This
allows inclusion of the Content-MD5 header during object creation.
Backfill tests with both ByteSource and InputStream inputs.
2020-05-31 17:48:31 +09:00
Andrew Gaul 08a16c95fb JCLOUDS-1546: Support GCS Archive storage class
Also change portable abstraction mapping for Tier.ARCHIVE.
2020-05-09 12:01:41 +09:00
roded d220b245d7 JCLOUDS-1543: remove unused imports from FetchBlobMetadataTest.java (#70)
Co-authored-by: Roded Bahat <roded.bahat@model9.io>
2020-04-22 10:32:54 +02:00
Fritz Elfert 286fe5cba0
Fix toolchain setup on jenkins (#68) 2020-04-22 09:56:36 +02:00
Roded Bahat 5ac92111c4 JCLOUDS-1543: change FetchBlobMetadata to retain original blob order 2020-04-17 19:24:15 +09:00
Fritz Elfert 89e1713c7c
Enable docs in travis build (#69) 2020-04-17 08:26:50 +02:00
Fritz Elfert 413ee30720
Fix javadoc generation on JDK >= 8 (#67) 2020-04-16 13:23:31 +02:00
Timur Alperovich 8d0e9dc962 Hash the content for fs MPU ETag if no xattr.
If there is no extended attribute support in the file system, the blobs
will not have their associated ETags available. In that case, the file
system blob store should rehash the content while producing the combined
blob and return the expected S3-style ETag.
2020-04-04 18:42:59 +09:00
ikky888 69ca45720d
JCLOUDS-1541: Add Middle East (Bahrain) region to the AWS EC2 and S3 providers list 2020-04-04 10:48:01 +09:00
Xavier BOURGOUIN d6702e5ee0 Fix BlobMetadata null size when using ApacheHCHttp module
JClouds is apparently exclusively using the Payload object from the HTTP
response to fill in the size of the BlobMetadata (when calling
blobStore.blobMetadata(...) ) - adapt this driver accordingly otherwise
we systematically get null size BlobMetadata out of it.
2020-03-08 22:15:43 +09:00
Colm O hEigeartaigh 99ef891e76 JCLOUDS-1540 - Update Snakeyaml 2020-03-03 17:41:33 +09:00
majaseremet ca5190636a
JCLOUDS-1533 - Fix upload with SAS token when blob name contains slash (#61) 2020-02-17 15:28:33 +01:00
Ian Springer 3db2939885 JCLOUDS-1538: fix typo in rfc1123SimpleDateFormat (#60) 2020-01-24 09:34:50 -05:00
Colm O hEigeartaigh b96158e6ed JCLOUDS-1532 - Update SSHJ + JSCH (#57) 2019-12-03 17:17:06 +01:00
Markus Alexander Kuppe 11d05ac9bf JCLOUDS-1529: Do not export org.jclouds.json.gson.internal (#56)
Do not export packages matching org.jclouds.json.gson.internal*
because it causes a use constraint violation with OSGi between
jclouds-core and jclouds-gson.

https://issues.apache.org/jira/browse/JCLOUDS-1529
2019-11-28 10:34:16 +01:00
Colm O hEigeartaigh 3a338da682 JCLOUDS-1528 - Use TLS instead of SSL in SSLContext.getInstance (#55) 2019-11-25 14:54:48 +01:00
Colm O hEigeartaigh 18da28af9f Downgrade BouncyCastle to 1.60, which works with Oracle JDK 1.7 (#54) 2019-11-18 14:54:30 +01:00
Colm O hEigeartaigh 30830bfa4e JCLOUDS-1526 - Update BouncyCastle dependency (#53) 2019-11-15 16:03:40 +01:00
Colm O hEigeartaigh 5f08ee2f1a JCLOUDS-1525 - Update xmlbuilder dependency (#52) 2019-11-15 16:03:28 +01:00
Andrew Gaul febc9b73df Fix TOCTOU issue when getting blob during removal
Fixes gaul/s3proxy#318.
2019-11-09 12:08:40 +09:00
Xavier BOURGOUIN 316b74db94 Fix infinite socket timeout when using ApacheHCHttpCommandExecutorServiceModule (#51)
Because the Java type BasicHttpParams wasn't matching the one returned
by provider "newBasicHttpParams" (which is returning a HttpParams type), it was configuring the Apache HTTP client with default settings (including connection and socket timeout), ignoring the ones of JClouds which were must safer in this regard (JCloudsones are currently 60sec for both, while Apache HC ones are -1, so infinite). It also prevented to override those default/infinite timeout settings with JClouds ones.
2019-10-23 11:44:58 +02:00
Ignasi Barrera f5b29c7028 Next development version 2.3.0-SNAPSHOT 2019-10-21 10:32:43 +02:00
Ignasi Barrera 7221844fac Apache jclouds 2.2.0-rc1 release 2019-10-21 10:32:43 +02:00
roded 6d389b0d86 JCLOUDS-1520: change UntrustedSSLContextSupplier to return the same SSLContext (#49)
Using the same SSLContext prevents consistent cache misses on the JVM's KeepAliveCache
when attempting to reuse TLS connections.
2019-10-16 09:48:47 +02:00
Ignasi Barrera 0f93525814
Add TravisCI file for pull requests (#50)
* Add TravisCI file for pull requests

* Fix env var config

* Oracle JDK not available in the default travis env

* Add license header

* Do not use Maven toolchains

* Less verbose build

* Further reduce output
2019-10-14 10:25:06 +02:00
Ignasi Barrera 644fb4a511 Next development version 2.3.0-SNAPSHOT 2019-10-13 11:00:35 +02:00
Ignasi Barrera 0aef8ca79f Apache jclouds 2.2.0-rc1 release 2019-10-13 11:00:06 +02:00
Ignasi Barrera a2cbdd3385 Etag is not supported in OSX 2019-10-13 10:32:50 +02:00
Xavier BOURGOUIN c445547ea8 Fix null content-length header on HEAD requests
https://issues.apache.org/jira/projects/JCLOUDS/issues/JCLOUDS-1505
2019-10-10 09:59:38 -07:00
Andrew Phillips 6ef0937c67 Update README copyright dates 2019-09-19 19:19:16 -04:00
Simone Locci fa1962b223 Add OS disk type in template options (#46)
* Add OS disk type in template options

* Fix review and code style
2019-09-18 16:34:34 +02:00
Simone Locci 6a076fe0c8 Add azure disk sku (#45)
* Add azure disk SKU

* Add tests

* Fix review adding a disk type enum

* Fix review using storage account type enum
2019-09-17 17:48:32 +02:00
Colm O hEigeartaigh 7b1efdc307 JCLOUDS-1512 - Use SecureRandom in Sha512Crypt 2019-08-22 09:14:30 -07:00
Roded Bahat dfeafcc6fb JCLOUDS-1510: ParseSax use charset constant for all String creations
This commit prevents the convertStreamToStringAndParse method from
failing XML validation on environments in which the default charset is
not compatible with UTF-8.
2019-08-19 04:59:51 -07:00
Simone Locci fcf72e5ea5 Add tag support to azure disks 2019-08-09 12:24:49 +02:00
Roded Bahat a248697c06 JCLOUDS-1509: read AWS response data with the UTF-8 charset explicitly
AWS response data is encoded in UTF-8. Creating a String from said data
using the JVM's default charset results in incorrect encoding on
environments in which the JVM's default charset is not UTF-8.

https://issues.apache.org/jira/browse/JCLOUDS-1509
2019-08-08 15:32:25 +03:00