10850 Commits

Author SHA1 Message Date
Andrew Gaul
b5e4e1d0fd JCLOUDS-1627: Upgrade to Jakarta packages
This resolves an issue with newer Guice versions.
2024-02-23 18:04:48 +09:00
Andrew Gaul
6bb738195f Add a test for uploading large objects
This is disabled by default.  Also rework upload helper to avoid using
an unbounded amount of memory.
2024-02-21 21:57:51 +09:00
Andrew Gaul
ed160d30c5 Copy Swift logging configuration to GCS
This avoids logging everything at trace level.
2024-02-21 21:56:29 +09:00
Jan Vermeulen
47f34770c9 JCLOUDS-1606: JCLOUDS-1608: Fix MPU off-by-one
Previously GCS could not upload large objects due to its 32 part
limit.
2024-02-21 21:35:16 +09:00
dependabot[bot]
a983eaced9 Bump testng from 7.5 to 7.5.1 in /project
Bumps [testng](https://github.com/cbeust/testng) from 7.5 to 7.5.1.
- [Release notes](https://github.com/cbeust/testng/releases)
- [Changelog](https://github.com/testng-team/testng/blob/master/CHANGES.txt)
- [Commits](https://github.com/cbeust/testng/compare/7.5...7.5.1)

---
updated-dependencies:
- dependency-name: org.testng:testng
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-20 20:35:55 +09:00
Andrew Gaul
62abf39575 Correct testName
testng 7.5.1 previously failed with this.  References #171.
2024-02-20 20:20:28 +09:00
davidsenk
41d842d449
JCLOUDS-1519: Fix the authorization error with b2 application keys 2024-02-19 11:05:35 +09:00
Andrew Gaul
2a773c2a72 Upgrade netty to 3.10.6.Final 2024-02-18 21:32:12 +09:00
dependabot[bot]
8af11edaff Bump ch.qos.logback:logback-core from 1.2.9 to 1.2.13 in /project
Bumps [ch.qos.logback:logback-core](https://github.com/qos-ch/logback) from 1.2.9 to 1.2.13.
- [Commits](https://github.com/qos-ch/logback/compare/v_1.2.9...v_1.2.13)

---
updated-dependencies:
- dependency-name: ch.qos.logback:logback-core
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-18 21:26:42 +09:00
dependabot[bot]
20bbb21a51 Bump bcprov-ext-jdk18on from 1.71 to 1.74 in /drivers/bouncycastle
Bumps [bcprov-ext-jdk18on](https://github.com/bcgit/bc-java) from 1.71 to 1.74.
- [Changelog](https://github.com/bcgit/bc-java/blob/main/docs/releasenotes.html)
- [Commits](https://github.com/bcgit/bc-java/commits)

---
updated-dependencies:
- dependency-name: org.bouncycastle:bcprov-ext-jdk18on
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-18 21:08:03 +09:00
Mickael LANOE
6455fd8518 Parse out of order ListBucket response elements
Currently, jclouds assumes that the <ID> must occur before the
<DisplayName> tag. If the XML body does not adhere to this order, the
parser throws a NullPointerException.

Scaleway is a cloud provider that does not adhere to this order and
returns the <DisplayName> tag before the <ID> tag. The patch changes the
parser to not depend on the order of the tags.

See commit 04feb8f8482e48b6c26166edb3d6fd5a607d858b for a similar fix
2023-12-10 00:21:47 +09:00
jixinchi
9b01dbc983 no need to check contents contains md because it is a set 2023-11-14 21:48:21 +09:00
jixinchi
311a4102da fix typo and add unit test 2023-11-14 21:48:21 +09:00
jixinchi
ef09dbb6ad do not remove md in DelimeterFilter then add it by commonPrefixes; just keep it 2023-11-14 21:48:21 +09:00
Valentin
47e3022519
FilesystemStorageStrategyImpl.getBlob() for a folder return blob as type Blob (#187)
Co-authored-by: Valentin Popov <popov@mailarchiva.ru>
2023-11-11 16:37:00 -05:00
Lars Hagen
a2628f9cbf
implement support for SharedKey signature (#186)
* implement support for SharedKey signature

This is the recommended signature scheme for Azure, and the only scheme
that is supported by the Azurite emulator.

https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key

* Remove print statement

Co-authored-by: Ignasi Barrera <nacx@apache.org>

* Remove print statement

* simplify logic

---------

Co-authored-by: Ignasi Barrera <nacx@apache.org>
2023-10-20 11:19:06 +02:00
jixinchi
eb1181daf5
check file exists in getContainerAccess (#185) 2023-10-12 18:18:30 +02:00
Edo Demirbilek
611b4c4a11
Adding access key auth support for openstack V3 (#183)
* Adding access key auth support for openstack V3 from an earlier commit + adding test

* added new line at the end to fix checkstyle warning
2023-09-04 23:49:45 +02:00
Ignasi Barrera
6b49389610
Fix checkstyle violation in google-cloud-storage (#184) 2023-09-04 23:49:17 +02:00
Andrew Gaul
1252f1951f Replace US_CENTRAL2 with US_CENTRAL1 in tests
US_CENTRAL2 has been removed.
2023-07-29 16:36:59 +09:00
Andrew Gaul
20ebc67fc8 Update google-cloud-storage regions 2023-07-29 16:36:59 +09:00
dependabot[bot]
ef580d4aa7 Bump okio from 1.17.2 to 3.4.0 in /drivers/okhttp
Bumps [okio](https://github.com/square/okio) from 1.17.2 to 3.4.0.
- [Changelog](https://github.com/square/okio/blob/master/CHANGELOG.md)
- [Commits](https://github.com/square/okio/compare/okio-parent-1.17.2...parent-3.4.0)

---
updated-dependencies:
- dependency-name: com.squareup.okio:okio
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-29 16:16:26 +09:00
Andrew Gaul
132604d71a Add missing jclouds.blobstore.httpstream.url
Lacking this caused integration test failures with newer testng.
2023-07-29 15:30:28 +09:00
Andrew Gaul
8075bbe50a JCLOUDS-1618: Upgrade to gson 2.10.1 2023-06-15 19:05:39 -07:00
Andrew Gaul
2275637fb8 Remove bnd
This blocks upgrading gson.  References #176.
2023-06-15 18:43:55 -07:00
dependabot[bot]
a00454bdd1 Bump guava from 31.1-jre to 32.0.0-jre in /project
Bumps [guava](https://github.com/google/guava) from 31.1-jre to 32.0.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-14 20:19:30 -07:00
Andrew Gaul
e7894e5a77 Use a class with a File member
Using gson on a File directly breaks Java strong encapsulation.
2023-06-14 20:17:28 -07:00
Andrew Gaul
db69360b5f Revert "Revert "Upgrade to snakeyaml 2.0""
This reverts commit cf4a926afef2887d818c41d65a91a4a9c95a1c27.  The
current CI failure is unrelated to this dependency.
2023-06-14 20:16:28 -07:00
Andrew Gaul
39050e8fad Java 9/16 strong encapsulation fixes
* Remove Reflection access modification
* Make some fields public so reflection can access them with strong
  encapsulation
* Ignore test which fails due to differences in File serialization
2023-06-11 23:09:18 -07:00
Andrew Gaul
c8ca330857 Remove unneeded -XX:MaxPermSize=256m
Newer JDK remove this flag.
2023-06-06 03:50:06 -07:00
Andrew Gaul
318138e6b0 Upgrade to surefire 3.1.0
Release notes:

https://github.com/apache/maven-surefire/releases/tag/surefire-3.0.0
https://github.com/apache/maven-surefire/releases/tag/surefire-3.1.0
2023-06-06 03:24:44 -07:00
Andrew Gaul
8a3c5ae992 Upgrade to EasyMock 5.1.0
This resolves a Java 17 compatibility issue.  Release notes:

https://github.com/easymock/easymock/releases/tag/easymock-5.0.0
https://github.com/easymock/easymock/releases/tag/easymock-5.0.1
https://github.com/easymock/easymock/releases/tag/easymock-5.1.0
2023-06-05 04:25:59 -07:00
Andrew Gaul
cf4a926afe Revert "Upgrade to snakeyaml 2.0"
This reverts commit f82e94c3374d7ced6d5a6a7d0238fe85a954ac3d.  This
break CI due to an unexpected Java version mismatch.
2023-05-10 21:30:27 +09:00
Andrew Gaul
f82e94c337 Upgrade to snakeyaml 2.0
This has some API changes.  Closes #169.
2023-04-16 16:43:36 +09:00
Ignasi Barrera
788f75f937 Replace TravisCI with GitHUb Actions 2023-02-08 08:33:39 +09:00
Andrew Gaul
b7f28f1e6a Lazily open parts during LocalBlobStore complete MPU
This removes a previous workaround for opening too many
FileInputStream and exhausting rlimits.
2023-01-29 17:56:15 +09:00
Andrew Gaul
12de6ef74d Atomically replace objects in filesystem putBlob
Fixes gaul/s3proxy#490.  This fixes a regression from
41ce90ec360a46cb7b03f7cb8b66e01a113aa5b6.  Continue to support
Windows logic although it is unclear if this is necessary.
2023-01-24 14:44:53 +09:00
Andrew Gaul
62632c9db6 JCLOUDS-1371: Optimize filesystem delimiter
populateBlobKeysInContainer will no longer recurse when the delimiter
matches "/".  This makes listing deep hierarchies with a delimiter
faster.  Note that the general LocalBlobStore handling is still
required for the general cases.  This requires removing a bogus test
case.  References gaul/s3proxy#473.
2023-01-24 14:44:34 +09:00
Andrew Gaul
e478dd5452 JCLOUDS-1371: JCLOUDS-1488: optimize fs prefix
This reduces the number of stat calls required when prefix is deep in the
filesystem hierarchy.  Further optimizations to delimiter are possible.
References gaul/s3proxy#473.
2023-01-22 18:31:09 +09:00
Andrew Gaul
7e50ae630d Upgrade to Guice 5.1.0
Release notes:

https://github.com/google/guice/wiki/Guice510
2022-12-17 07:41:44 +09:00
dependabot[bot]
b754e177fe Bump snakeyaml from 1.31 to 1.32 in /apis/byon
---
updated-dependencies:
- dependency-name: org.yaml:snakeyaml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-15 12:40:10 +09:00
Andrew Gaul
552eb95c44 Fix Files name collision
Regression from b282b5cbfef760be026660522e78d1bba81988ac.
2022-11-21 07:49:18 +09:00
Jonathan Leitschuh
b282b5cbfe vuln-fix: Temporary File Information Disclosure
This fixes temporary file information disclosure vulnerability due to the use
of the vulnerable `File.createTempFile()` method. The vulnerability is fixed by
using the `Files.createTempFile()` method which sets the correct posix permissions.

Weakness: CWE-377: Insecure Temporary File
Severity: Medium
CVSSS: 5.5
Detection: CodeQL & OpenRewrite (https://public.moderne.io/recipes/org.openrewrite.java.security.SecureTempFileCreation)

Reported-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>
Signed-off-by: Jonathan Leitschuh <Jonathan.Leitschuh@gmail.com>

Bug-tracker: https://github.com/JLLeitschuh/security-research/issues/18


Co-authored-by: Moderne <team@moderne.io>
2022-11-20 21:31:45 +09:00
Hervé Boutemy
25bcb7961e activate Reproducible Builds 2022-11-14 07:54:03 +09:00
Christian.Jung
b2a2025b32 This fixes problem JCLOUDS 1615. 2022-10-14 22:11:19 +09:00
SATYANAN-ANAND
d913a56037
JCLOUDS-1617: Fix HTTPS support in OkHttpCommandExecutorService (#153)
* JCLOUDS-1617: Fix HTTPS support in OkHttpCommandExecutorService

Added support for  proxy server type = HTTPS

* Update DelegatingSocketFactory.java

Added java doc
2022-09-15 17:50:36 +02:00
dependabot[bot]
b098cceaf9
Bump snakeyaml from 1.26 to 1.31 in /apis/byon (#154)
Bumps [snakeyaml](https://bitbucket.org/snakeyaml/snakeyaml) from 1.26 to 1.31.
- [Commits](https://bitbucket.org/snakeyaml/snakeyaml/branches/compare/snakeyaml-1.31..snakeyaml-1.26)

---
updated-dependencies:
- dependency-name: org.yaml:snakeyaml
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-15 13:16:09 +02:00
Andrew Gaul
57a9e7b7cc Deep copy Blob in LocalBlobStore.getBlob
ByteSourcePayload.openStream is not thread safe and lack of
synchronization can throw ArrayIndexOutOfBoundsExceptions.  Instead
deep copy the underlying Payload.  Fixes gaul/s3proxy#303.
2022-08-04 21:17:44 +09:00
Andrew Gaul
5067897ff5 Upgrade to jax-rs 2.1.1
Also address constant TODO.
2022-08-04 07:38:54 +09:00
SATYANAN-ANAND
88f0c341cf JCLOUDS-1616: Proxy credentials validation is missing in OkHttpCommandExecutorService API request
Added support for credentials validation
2022-08-03 21:26:49 +09:00