populateBlobKeysInContainer will no longer recurse when the delimiter
matches "/". This makes listing deep hierarchies with a delimiter
faster. Note that the general LocalBlobStore handling is still
required for the general cases. This requires removing a bogus test
case. References gaul/s3proxy#473.
This reduces the number of stat calls required when prefix is deep in the
filesystem hierarchy. Further optimizations to delimiter are possible.
References gaul/s3proxy#473.
This fixes temporary file information disclosure vulnerability due to the use
of the vulnerable `File.createTempFile()` method. The vulnerability is fixed by
using the `Files.createTempFile()` method which sets the correct posix permissions.
Weakness: CWE-377: Insecure Temporary File
Severity: Medium
CVSSS: 5.5
Detection: CodeQL & OpenRewrite (https://public.moderne.io/recipes/org.openrewrite.java.security.SecureTempFileCreation)
Reported-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Bug-tracker: https://github.com/JLLeitschuh/security-research/issues/18
Co-authored-by: Moderne <team@moderne.io>
Previously s3 always used v2 and aws-s3 always used v4. Now s3
defaults to v2 and can override to v4. Note that this does not change
BlobRequestSigner.
The issue happens if a cloud provider returns lowercase metadata headers, for example: "x-object-meta-apiversion" instead of "X-Object-Meta-ApiVersion"
In that case, BlobStore.blobMetadata(CONTAINER, PATH).getUserMetadata()
incorrectly returns an empty map.
This happens because the code is looking for the exact String "-Meta-" (case-sensitive).
This checkin allows to handle metadata headers of any case, and also adds a unit test for that situation.
Currently, jclouds assumes that the ListBuckets response follows a
specific order: the <Owner> tag, followed by the <Buckets> tag. Within
the <Owner> tag, jclouds further assumes that the <ID> must occur before
the <DisplayName> tag. If the XML body does not adhere to this order,
the parser throws a NullPointerException.
DigitalOcean spaces does not adhere to this order and returns the
<DisplayName> tag before the <ID> tag. The patch changes the parser to
not depend on the order of the tags.
XMLBuilder is a very nice fluent API for building XML documents.
However, it is only used by a tiny portion of jclouds: the s3 api. The
use of the XMLBuilder class requires a dependency to
com.jamesmurty.utils:java-xmlbuilder jar and a transitive
dependency to the net.iharder:base64 jar (superseded by
java.util.Base64 in java 8). They are 18kb each approximately and they
not OSGi compatible. They are not huge, but they represent more API
surface and more things to change when trying to use jclouds in an OSGi
context (they need to be replaced by OSGi compatible bundles like
org.apache.servicemix.bundles.java-xmlbuilder).
This commit replaces the use of XMLBuilder by a direct use of the
javax.xml and org.w3c.dom APIs.
I hope retesting will be minimal, and most of this code is covered by
unit tests.
The JClouds project module upgrades the okhttp server library and related dependencies such as mockwebserver from 2.2.0 to 3.14.9. The vendor switched the groupId declaration from com.squareup.okhttp to com.squareup.okhttp3.
Adjust imports and api calls for newer okhttp vers
- Mock-Webserver v2.2.0 - Vendor, defined dependency bcprov-jdk15on > Previously, the dependency got excluded; otherwise, it conflicted with the existing bouncy castle version used in JClouds.
- Mock-Webserver v3.14.9 - The vendor does not define a bouncy castle dependency anymore; instead, it moved to the okhttp-tls module.
- Introduce the okhttp-tls module for tests in the core module. The following APIs (Docker, Elastic-Stack) and providers Softlayer extend their respective MockTest from BaseMockWebServerTest. The mock base test is located in the test jar of JClouds core module.
- Due to conflicting bouncy castle classes in the classpath, those which get loaded from the okhttp-tls module, and those defined in JClouds bouncy castle module, the bouncy castle dependency of okhttp-tls has to get skipped for the two APIs and providers mentioned.
Side note:
The JClouds GAE driver module also requires the new okhttp-tls dependency because of the following chain of inheritance.
Different from the situation above, the bouncy castle classes of the okhttp-tls got not excluded.
GaeHttpCommandExecutorServiceIntegrationTest -> BaseHttpCommandExecutorServiceIntegrationTest -> BaseMockWebServerTest
The reason for this is unknown to me.
* animal sniffer should be on java18, just like `<jdk.version>`
* Only use XMLBuilder's elem() and text() methods to have similar looking code
* Remove unnecessary call to XMLBuilder's up() because the returned value is never used
* Simplify code
* Deduplicate code
* Make the code more explicit by returning the rootBuilder
This project, the aws-lab version of Apache JClouds, share the exact build instructions as the primary Apache JClouds project with all its modules.
Apache JClouds is shifting its strategy in handling OSGi configuration. Instead of using the Maven Bundle Plugin, a wrapper of the BND plugin, the BND plugin gets used directly.
- Remove the OSGi configuration from each module. The configuration gets served to the BND through dedicated configuration / bnd files.
- Onboard bnd-configuration files, one per module.
Ignore bnd files in rat plugin
Note: Previously, the maven jar plugin contained a configuration embedded in each module's generated manifest files. The configuration got relocated to the project/bnd.bnd file in a previous commit, and gets handled through the bnd plugin.
Replace substituted GSON package names with those provided from the vendor.
Reduce OSGi-metadata declaration of core-module because the artificial package org.jclouds.json.gson.internal was removed.
Remove the Gson module its children Gson bundle, and Gson shaded.
Remove duplication conflict and check-style rules due to the removal of the internal Gson module.
Add maven repository where a custom version of the Gson library gets hosted, which exports all packages.
Remove particular repository
Remove the declaration of the repository that serves a custom build GSON version. The build uses GSON in its original form of the vendor, which gets distributed through the standard distribution channel. The identifiers for group, artifact, and version correspond to the latest stable release of GSON.
Integrate GSON library in Clouds Core Bundle
The change contained in the commit puts the GSON library into the classpath of the JClouds core module.
After several tests with Karaf and Karaf JClouds, especially if the Maven identifier matches the original GSON library, there are only a limited number of ways to keep the deployment effort low.
Specifically, Karaf has a set of predefined Maven repositories that can be easily customized. The order in which a particular repository is resolved into the customized GSON library is more difficult. In normal OSGi applications, which do not have such a management function, I imagine this configuration to be more complicated. Sure, a unique identifier would help, but then we are back to step 1.
Although I honestly don't like to see this kind of approach in a codebase I'm working with, there are not many alternatives to the main aspect of deployment mentioned above.
Maybe the approach can still ease the problem in the short term. In a further mid-term step, however, this problem must be addressed in general.
