Note: Previously, the maven jar plugin contained a configuration embedded in each module's generated manifest files. The configuration got relocated to the project/bnd.bnd file in a previous commit, and gets handled through the bnd plugin.
In the last commit (last section of squashed commit), the GSON library was integrated into the JClouds core module using maven-bundle plugins include resource instruction. Building OSGi instruction variables from the respective modules show a weakness when resources such as script builder shell scripts are required to be integrated into the bundle but not provide a dedicated variable declaration for the resource section.
The following commit demonstrates a change in strategy in declaration and integration of OSGi metadata.
- Replace old bundle-plugin with newest bnd-plugin (bundle-plugin uses bnd-plugin internally)
- Move OSGi metadata declarations from a maven variable passing strategy into dedicated bnd.bnd files
+ Cleaner pom files, no bundle packaging
+ Intellisense / Autocomplete support for .bnd files in terms of package exports etc.
For demonstration, the overall OSGi adjustments are limited to project, core, script builder, compute, blob store, and load balancer because most custom OSGi metadata is defined here.
Note: Other modules are currently disabled from build because some feedback is needed first.
Make GSON integration work.
To understand the changes, see the core modules' bnd file. GSON internal packages also define a version. Both already exported and new export declarations are fused. The global JClouds core module exports defined the entire set of GSON packages available.
Some minor modifications were made in the module project; replace maven jar plugin with a minified version of the declaration, outsourced in projects bnd file.
Replace substituted GSON package names with those provided from the vendor.
Reduce OSGi-metadata declaration of core-module because the artificial package org.jclouds.json.gson.internal was removed.
Remove the Gson module its children Gson bundle, and Gson shaded.
Remove duplication conflict and check-style rules due to the removal of the internal Gson module.
Add maven repository where a custom version of the Gson library gets hosted, which exports all packages.
Remove particular repository
Remove the declaration of the repository that serves a custom build GSON version. The build uses GSON in its original form of the vendor, which gets distributed through the standard distribution channel. The identifiers for group, artifact, and version correspond to the latest stable release of GSON.
Integrate GSON library in Clouds Core Bundle
The change contained in the commit puts the GSON library into the classpath of the JClouds core module.
After several tests with Karaf and Karaf JClouds, especially if the Maven identifier matches the original GSON library, there are only a limited number of ways to keep the deployment effort low.
Specifically, Karaf has a set of predefined Maven repositories that can be easily customized. The order in which a particular repository is resolved into the customized GSON library is more difficult. In normal OSGi applications, which do not have such a management function, I imagine this configuration to be more complicated. Sure, a unique identifier would help, but then we are back to step 1.
Although I honestly don't like to see this kind of approach in a codebase I'm working with, there are not many alternatives to the main aspect of deployment mentioned above.
Maybe the approach can still ease the problem in the short term. In a further mid-term step, however, this problem must be addressed in general.
With Java 11, an illegal reflective access is output for the google cloud storage blobstore.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jclouds.reflect.Reflection2$1 (file:/.../jclouds-core.jar) to constructor java.lang.String(char[],int,int,java.lang.Void)
WARNING: Please consider reporting this to the maintainers of org.jclouds.reflect.Reflection2$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Indeed, JClouds calls `setAccessible(true)` on the package protected constructor
`java.lang.String(char[],int,int,java.lang.Void)`.
Investigating the code, it turns out it is looking for constructors annotated with any of:
* java.beans.ConstructorProperties
* org.jclouds.json.SerializedNames
* com.google.inject.Inject
But `String` being defined in `java.base` module, it is impossible
that it will be annotated with any of these annotation.
This commit is complementary to JClouds commit db4e4af931 .
Reflection2.java:
Do not call `setAccessible(true)` on core java constructors and methods.
For reference, here is the stacktrace of this illegal access warning:
java.lang.String.<init>(String.java:3208)
java.lang.String.<init>(String.java:251)
java.util.StringJoiner.compactElts(StringJoiner.java:250)
java.util.StringJoiner.toString(StringJoiner.java:173)
jdk.internal.module.IllegalAccessLogger.loudWarning(IllegalAccessLogger.java:339)
jdk.internal.module.IllegalAccessLogger.log(IllegalAccessLogger.java:288)
jdk.internal.module.IllegalAccessLogger.log(IllegalAccessLogger.java:261)
jdk.internal.module.IllegalAccessLogger.logIfOpenedForIllegalAccess(IllegalAccessLogger.java:226)
java.lang.reflect.AccessibleObject.logIfOpenedForIllegalAccess(AccessibleObject.java:366)
java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:325)
java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:189)
java.lang.reflect.Constructor.setAccessible(Constructor.java:182)
org.jclouds.reflect.Reflection2$1.load(Reflection2$1.java:157)
org.jclouds.reflect.Reflection2$1.load(Reflection2$1.java:153)
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache$LoadingValueReference.java:3529)
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache$Segment.java:2278)
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache$Segment.java:2155)
com.google.common.cache.LocalCache$Segment.get(LocalCache$Segment.java:2045)
com.google.common.cache.LocalCache.get(LocalCache.java:3953)
com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache$LocalLoadingCache.java:4960)
org.jclouds.reflect.Reflection2.get(Reflection2.java:346)
org.jclouds.reflect.Reflection2.constructors(Reflection2.java:100)
org.jclouds.json.internal.NamingStrategies$AnnotationConstructorNamingStrategy.getDeserializer(NamingStrategies$AnnotationConstructorNamingStrategy.java:271)
org.jclouds.json.internal.DeserializationConstructorAndReflectiveTypeAdapterFactory.create(DeserializationConstructorAndReflectiveTypeAdapterFactory.java:125)
com.google.gson.Gson.getAdapter(Gson.java:458)
org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:117)
org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:166)
org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:102)
com.google.gson.Gson.getAdapter(Gson.java:458)
com.google.gson.Gson.toJson(Gson.java:696)
com.google.gson.Gson.toJson(Gson.java:683)
com.google.gson.Gson.toJson(Gson.java:638)
com.google.gson.Gson.toJson(Gson.java:618)
org.jclouds.json.internal.GsonWrapper.toJson(GsonWrapper.java:65)
org.jclouds.oauth.v2.functions.ClaimsToAssertion.apply(ClaimsToAssertion.java:59)
org.jclouds.oauth.v2.functions.ClaimsToAssertion.apply(ClaimsToAssertion.java:43)
org.jclouds.rest.internal.RestAnnotationProcessor.getParamValue(RestAnnotationProcessor.java:829)
org.jclouds.rest.internal.RestAnnotationProcessor.getFormParamKeyValues(RestAnnotationProcessor.java:847)
org.jclouds.rest.internal.RestAnnotationProcessor.addFormParams(RestAnnotationProcessor.java:435)
org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:258)
org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:137)
org.jclouds.rest.internal.InvokeHttpMethod.toCommand(InvokeHttpMethod.java:189)
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:85)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
com.sun.proxy.$Proxy49.authorize(Unknown Source)
org.jclouds.oauth.v2.filters.JWTBearerTokenFlow$AuthorizeToken.load(JWTBearerTokenFlow$AuthorizeToken.java:84)
org.jclouds.oauth.v2.filters.JWTBearerTokenFlow$AuthorizeToken.load(JWTBearerTokenFlow$AuthorizeToken.java:68)
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache$LoadingValueReference.java:3529)
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache$Segment.java:2278)
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache$Segment.java:2155)
com.google.common.cache.LocalCache$Segment.get(LocalCache$Segment.java:2045)
com.google.common.cache.LocalCache.get(LocalCache.java:3953)
com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache$LocalLoadingCache.java:4960)
com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache$LocalLoadingCache.java:4966)
org.jclouds.oauth.v2.filters.JWTBearerTokenFlow.filter(JWTBearerTokenFlow.java:99)
org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:90)
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:86)
com.sun.proxy.$Proxy54.simpleUpload(Unknown Source)
org.jclouds.googlecloudstorage.blobstore.GoogleCloudStorageBlobStore.uploadMultipartPart(GoogleCloudStorageBlobStore.java:422)
org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:415)
org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:402)
com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor$Worker.java:628)
java.lang.Thread.run(Thread.java:834)
DnsNameValidator.java uses a deprecated guava APIs in code that is used
to support Azure cloud storage. When forcing the use of more recent guava
versions, the code fails with NoSuchFieldError.
However, CharMatcher.JAVA_LETTER_OR_DIGIT has been removed in guava 26.0,
and CharMatcher.javaLetterOrDigit() should be used instead since guava
19.0. Note that CharMatcher.javaLetterOrDigit() was immediately
deprecated in Guava 26.0, and java.lang.Character.isLetterOrDigit(int)
should be used instead.
This commit replaces the use of this deprecated API by
java.lang.Character.isLetterOrDigit(int).
It is no worse than the previous code.
(If I understand correctly, updating the guava version is a challenge due to
dependencies on Apache Karaf anyway)
Previously this provider worked around a RestAnnotationProcessor quirk
by using multi-part uploads for InputStream payloads. Instead work
around the quirk another way which allows a single-part upload. This
allows inclusion of the Content-MD5 header during object creation.
Backfill tests with both ByteSource and InputStream inputs.
Do not export packages matching org.jclouds.json.gson.internal*
because it causes a use constraint violation with OSGi between
jclouds-core and jclouds-gson.
https://issues.apache.org/jira/browse/JCLOUDS-1529
This commit prevents the convertStreamToStringAndParse method from
failing XML validation on environments in which the default charset is
not compatible with UTF-8.
* JCLOUDS-1166: Relocate the gson internal packge to be able to keep using it
* Fixes
* Fix import order and shaded jar
* More fixes
* Proper dependency configuration
* Fix typos
* Bring back duplicate exclusions
This change improves the performance of writing to sockets with the
default Java URL connection HTTP client, by enlarging the buffer used
for socket writes from an implicit hard-coded 4KB / 8KB buffer to a
configurable 32KB buffer.
The buffer size is now controlled by the following property with the
following default value:
jclouds.output-socket-buffer-size: 32768
The implementation is based on a variant of ByteStreams.copy (written as
ByteStreams2.copy) which accepts the buffer size as an argument, unlike
the original Guava code that uses a hard-coded size.
The change was done directly within the loop that copies the input
stream to the output stream, and not by wrapping a BufferedOutputStream
around the existing output stream, in order to avoid copying the payload
twice.
On some platforms this change can improve both the putBlob throughput
and the total CPU consumption.
These should provide a descriptive second argument, not the same as
the first argument which is null in the failure case. This also found
a logic error in CreateVolumeResponseHandler.
The x-amz-copy-source header on S3 CopyObject should be URL encoded (as
a path). This is not universally true of all headers though (for example
the = in x-amz-copy-source-range) therefore introducing a new parameter
on @Headers to indicate whether URL encoding should take place.
Previously read returned a value between -128 and 127. -1 indicates
end of stream, causing issues for callers. Instead return values
between 0 and 255 as intended.
Reflective creation of SimpleTimeLimiter to allow compatibility with
Guava 23.0. SimpleTimeLimiter.create(ExecutorService) was introduced
in Guava 22.0 to replace the SimpleTimeLimiter(ExecutorService)
constructor, which was deprecated in Guava 22.0 and removed in Guava
23.0.
roded
gurkerl83
Tamas Cservenak
Jean-Noël Rouvignac
Andrew Gaul
Jean-Noël Rouvignac
Ian Springer
Markus Alexander Kuppe
Colm O hEigeartaigh
Ignasi Barrera
roded
Roded Bahat
Olaf Flebbe
Dori Polotsky
David Currie
Mathieu Tortuyaux
Jesse Glick
andreaturli
Tim Peierls
Chaithanya Ganta
Svetoslav Neykov