* implement support for SharedKey signature
This is the recommended signature scheme for Azure, and the only scheme
that is supported by the Azurite emulator.
https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key
* Remove print statement
Co-authored-by: Ignasi Barrera <nacx@apache.org>
* Remove print statement
* simplify logic
---------
Co-authored-by: Ignasi Barrera <nacx@apache.org>
* Remove Reflection access modification
* Make some fields public so reflection can access them with strong
encapsulation
* Ignore test which fails due to differences in File serialization
This fixes temporary file information disclosure vulnerability due to the use
of the vulnerable `File.createTempFile()` method. The vulnerability is fixed by
using the `Files.createTempFile()` method which sets the correct posix permissions.
Weakness: CWE-377: Insecure Temporary File
Severity: Medium
CVSSS: 5.5
Detection: CodeQL & OpenRewrite (https://public.moderne.io/recipes/org.openrewrite.java.security.SecureTempFileCreation)
Reported-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Bug-tracker: https://github.com/JLLeitschuh/security-research/issues/18
Co-authored-by: Moderne <team@moderne.io>
* JCLOUDS-1617: Fix HTTPS support in OkHttpCommandExecutorService
Added support for proxy server type = HTTPS
* Update DelegatingSocketFactory.java
Added java doc
Remove distribution profiles from the JClouds parent, main, and core modules. Each profile defines assembly descriptors, but none of those files are provided by the project anymore.
The maven assembly plugin relies on those descriptors to dictate its execution.
The estimation is that individual maven artifacts, one per module, get deployed instead of a compressed deployment unit.
In that case, it is enough to use mavens deploy command in combination with distribution management configuration pushing individual jars to the artifact repositories.
Increase of OSGi dependencies core and compendium (now cmpn) from 4.2 to 6.0.
Previously it was possible to run JClouds in OSGi environments from version 4.
An essential aspect to use JClouds in an OSGi environment requires so-called feature sets. These can be generated manually or automatically - see JClouds-Karaf project. Since there have been significant changes in the structure and behaviour of Karaf in the meantime, an adaptation is appropriate.
Breaking change - probably not, as no other APIs of core and compendium are used than up to now.
OSGi - V4.2 - Karaf 2.2.x - to 2.2.9 (Current Status - not active)
OSGi - V6.0 - Karaf 4.1.x - 4.2 (Current Status - active)
Remove javax libraries inject and annotation. In JDK 8 those libraries are provided by the JDK. Younger JDK versions > 11 exclude them again to make the JDK leaner. Supporting younger JDK versions means integrating more younger libraries maintained by the Jakarta project.
Note: Previously, the maven jar plugin contained a configuration embedded in each module's generated manifest files. The configuration got relocated to the project/bnd.bnd file in a previous commit, and gets handled through the bnd plugin.
In the last commit (last section of squashed commit), the GSON library was integrated into the JClouds core module using maven-bundle plugins include resource instruction. Building OSGi instruction variables from the respective modules show a weakness when resources such as script builder shell scripts are required to be integrated into the bundle but not provide a dedicated variable declaration for the resource section.
The following commit demonstrates a change in strategy in declaration and integration of OSGi metadata.
- Replace old bundle-plugin with newest bnd-plugin (bundle-plugin uses bnd-plugin internally)
- Move OSGi metadata declarations from a maven variable passing strategy into dedicated bnd.bnd files
+ Cleaner pom files, no bundle packaging
+ Intellisense / Autocomplete support for .bnd files in terms of package exports etc.
For demonstration, the overall OSGi adjustments are limited to project, core, script builder, compute, blob store, and load balancer because most custom OSGi metadata is defined here.
Note: Other modules are currently disabled from build because some feedback is needed first.
Make GSON integration work.
To understand the changes, see the core modules' bnd file. GSON internal packages also define a version. Both already exported and new export declarations are fused. The global JClouds core module exports defined the entire set of GSON packages available.
Some minor modifications were made in the module project; replace maven jar plugin with a minified version of the declaration, outsourced in projects bnd file.
Replace substituted GSON package names with those provided from the vendor.
Reduce OSGi-metadata declaration of core-module because the artificial package org.jclouds.json.gson.internal was removed.
Remove the Gson module its children Gson bundle, and Gson shaded.
Remove duplication conflict and check-style rules due to the removal of the internal Gson module.
Add maven repository where a custom version of the Gson library gets hosted, which exports all packages.
Remove particular repository
Remove the declaration of the repository that serves a custom build GSON version. The build uses GSON in its original form of the vendor, which gets distributed through the standard distribution channel. The identifiers for group, artifact, and version correspond to the latest stable release of GSON.
Integrate GSON library in Clouds Core Bundle
The change contained in the commit puts the GSON library into the classpath of the JClouds core module.
After several tests with Karaf and Karaf JClouds, especially if the Maven identifier matches the original GSON library, there are only a limited number of ways to keep the deployment effort low.
Specifically, Karaf has a set of predefined Maven repositories that can be easily customized. The order in which a particular repository is resolved into the customized GSON library is more difficult. In normal OSGi applications, which do not have such a management function, I imagine this configuration to be more complicated. Sure, a unique identifier would help, but then we are back to step 1.
Although I honestly don't like to see this kind of approach in a codebase I'm working with, there are not many alternatives to the main aspect of deployment mentioned above.
Maybe the approach can still ease the problem in the short term. In a further mid-term step, however, this problem must be addressed in general.
With Java 11, an illegal reflective access is output for the google cloud storage blobstore.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jclouds.reflect.Reflection2$1 (file:/.../jclouds-core.jar) to constructor java.lang.String(char[],int,int,java.lang.Void)
WARNING: Please consider reporting this to the maintainers of org.jclouds.reflect.Reflection2$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Indeed, JClouds calls `setAccessible(true)` on the package protected constructor
`java.lang.String(char[],int,int,java.lang.Void)`.
Investigating the code, it turns out it is looking for constructors annotated with any of:
* java.beans.ConstructorProperties
* org.jclouds.json.SerializedNames
* com.google.inject.Inject
But `String` being defined in `java.base` module, it is impossible
that it will be annotated with any of these annotation.
This commit is complementary to JClouds commit db4e4af931 .
Reflection2.java:
Do not call `setAccessible(true)` on core java constructors and methods.
For reference, here is the stacktrace of this illegal access warning:
java.lang.String.<init>(String.java:3208)
java.lang.String.<init>(String.java:251)
java.util.StringJoiner.compactElts(StringJoiner.java:250)
java.util.StringJoiner.toString(StringJoiner.java:173)
jdk.internal.module.IllegalAccessLogger.loudWarning(IllegalAccessLogger.java:339)
jdk.internal.module.IllegalAccessLogger.log(IllegalAccessLogger.java:288)
jdk.internal.module.IllegalAccessLogger.log(IllegalAccessLogger.java:261)
jdk.internal.module.IllegalAccessLogger.logIfOpenedForIllegalAccess(IllegalAccessLogger.java:226)
java.lang.reflect.AccessibleObject.logIfOpenedForIllegalAccess(AccessibleObject.java:366)
java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:325)
java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280)
java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:189)
java.lang.reflect.Constructor.setAccessible(Constructor.java:182)
org.jclouds.reflect.Reflection2$1.load(Reflection2$1.java:157)
org.jclouds.reflect.Reflection2$1.load(Reflection2$1.java:153)
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache$LoadingValueReference.java:3529)
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache$Segment.java:2278)
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache$Segment.java:2155)
com.google.common.cache.LocalCache$Segment.get(LocalCache$Segment.java:2045)
com.google.common.cache.LocalCache.get(LocalCache.java:3953)
com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache$LocalLoadingCache.java:4960)
org.jclouds.reflect.Reflection2.get(Reflection2.java:346)
org.jclouds.reflect.Reflection2.constructors(Reflection2.java:100)
org.jclouds.json.internal.NamingStrategies$AnnotationConstructorNamingStrategy.getDeserializer(NamingStrategies$AnnotationConstructorNamingStrategy.java:271)
org.jclouds.json.internal.DeserializationConstructorAndReflectiveTypeAdapterFactory.create(DeserializationConstructorAndReflectiveTypeAdapterFactory.java:125)
com.google.gson.Gson.getAdapter(Gson.java:458)
org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:117)
org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:166)
org.jclouds.json.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:102)
com.google.gson.Gson.getAdapter(Gson.java:458)
com.google.gson.Gson.toJson(Gson.java:696)
com.google.gson.Gson.toJson(Gson.java:683)
com.google.gson.Gson.toJson(Gson.java:638)
com.google.gson.Gson.toJson(Gson.java:618)
org.jclouds.json.internal.GsonWrapper.toJson(GsonWrapper.java:65)
org.jclouds.oauth.v2.functions.ClaimsToAssertion.apply(ClaimsToAssertion.java:59)
org.jclouds.oauth.v2.functions.ClaimsToAssertion.apply(ClaimsToAssertion.java:43)
org.jclouds.rest.internal.RestAnnotationProcessor.getParamValue(RestAnnotationProcessor.java:829)
org.jclouds.rest.internal.RestAnnotationProcessor.getFormParamKeyValues(RestAnnotationProcessor.java:847)
org.jclouds.rest.internal.RestAnnotationProcessor.addFormParams(RestAnnotationProcessor.java:435)
org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:258)
org.jclouds.rest.internal.RestAnnotationProcessor.apply(RestAnnotationProcessor.java:137)
org.jclouds.rest.internal.InvokeHttpMethod.toCommand(InvokeHttpMethod.java:189)
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:85)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156)
org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123)
com.sun.proxy.$Proxy49.authorize(Unknown Source)
org.jclouds.oauth.v2.filters.JWTBearerTokenFlow$AuthorizeToken.load(JWTBearerTokenFlow$AuthorizeToken.java:84)
org.jclouds.oauth.v2.filters.JWTBearerTokenFlow$AuthorizeToken.load(JWTBearerTokenFlow$AuthorizeToken.java:68)
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache$LoadingValueReference.java:3529)
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache$Segment.java:2278)
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache$Segment.java:2155)
com.google.common.cache.LocalCache$Segment.get(LocalCache$Segment.java:2045)
com.google.common.cache.LocalCache.get(LocalCache.java:3953)
com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache$LocalLoadingCache.java:4960)
com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache$LocalLoadingCache.java:4966)
org.jclouds.oauth.v2.filters.JWTBearerTokenFlow.filter(JWTBearerTokenFlow.java:99)
org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:90)
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:86)
com.sun.proxy.$Proxy54.simpleUpload(Unknown Source)
org.jclouds.googlecloudstorage.blobstore.GoogleCloudStorageBlobStore.uploadMultipartPart(GoogleCloudStorageBlobStore.java:422)
org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:415)
org.jclouds.blobstore.internal.BaseBlobStore$BlobUploader.call(BaseBlobStore.java:402)
com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor$Worker.java:628)
java.lang.Thread.run(Thread.java:834)
DnsNameValidator.java uses a deprecated guava APIs in code that is used
to support Azure cloud storage. When forcing the use of more recent guava
versions, the code fails with NoSuchFieldError.
However, CharMatcher.JAVA_LETTER_OR_DIGIT has been removed in guava 26.0,
and CharMatcher.javaLetterOrDigit() should be used instead since guava
19.0. Note that CharMatcher.javaLetterOrDigit() was immediately
deprecated in Guava 26.0, and java.lang.Character.isLetterOrDigit(int)
should be used instead.
This commit replaces the use of this deprecated API by
java.lang.Character.isLetterOrDigit(int).
It is no worse than the previous code.
(If I understand correctly, updating the guava version is a challenge due to
dependencies on Apache Karaf anyway)
Previously this provider worked around a RestAnnotationProcessor quirk
by using multi-part uploads for InputStream payloads. Instead work
around the quirk another way which allows a single-part upload. This
allows inclusion of the Content-MD5 header during object creation.
Backfill tests with both ByteSource and InputStream inputs.
Do not export packages matching org.jclouds.json.gson.internal*
because it causes a use constraint violation with OSGi between
jclouds-core and jclouds-gson.
https://issues.apache.org/jira/browse/JCLOUDS-1529
This commit prevents the convertStreamToStringAndParse method from
failing XML validation on environments in which the default charset is
not compatible with UTF-8.
* JCLOUDS-1166: Relocate the gson internal packge to be able to keep using it
* Fixes
* Fix import order and shaded jar
* More fixes
* Proper dependency configuration
* Fix typos
* Bring back duplicate exclusions
Andrew Gaul
Max Smiley
Andrew Gaul
Maxim
Aliaksandr Stsiapanay
Lars Hagen
Jonathan Leitschuh
SATYANAN-ANAND
gurkerl83
korlov42
roded
Tamas Cservenak
Jean-Noël Rouvignac
Jean-Noël Rouvignac
Ian Springer
Markus Alexander Kuppe
Colm O hEigeartaigh
Ignasi Barrera
roded
Roded Bahat
Olaf Flebbe