Commit Graph

10860 Commits

Author SHA1 Message Date
Andrew Gaul 7a438ceebd JCLOUDS-1637: Replace java.xml.bind uses 2024-04-24 20:21:13 +09:00
Andrew Gaul da1bc06f9e JCLOUDS-1635: Add COOL and COLD to Tier
The former replaces INFREQUENT.  References gaul/s3proxy#625.
2024-04-20 17:31:52 +09:00
Andrew Gaul e1f34bbfa7 JCLOUDS-1634: Add AzureBlob COLD access tier
References gaul/s3proxy#625.
2024-04-14 17:51:21 +09:00
Maksim_Hadalau 4435765f55 add license header 2024-04-14 17:37:51 +09:00
Maksim_Hadalau 0688553087 JCLOUDS-1631: fix AWSRequestAuthorizeSignatureV4 when prefix contains special chars 2024-04-14 17:37:51 +09:00
Maxim bc43572d65
JCLOUDS-1630: Handle URI template properly with opened curve bracket (#199)
Co-authored-by: Maksim_Hadalau <Maksim_Hadalau@epam.com>
2024-03-20 15:19:46 +01:00
Aliaksandr Stsiapanay b379c17156 JCLOUDS-1632: GCP BlobStore fails to put a blob if blob name contains non ASCII characters 2024-03-20 22:40:11 +09:00
Andrew Gaul 4c7fb2c8b9 JCLOUDS-1626: Close stream in MultiBlobInputStream
Otherwise the inner FileInputStream will leak if the caller only reads
part of the stream before closing the outer MultiBlobInputStream.
2024-03-03 13:07:24 +09:00
Andrew Gaul 4f3955799b JCLOUDS-1628: Remove Netty driver
3.x has multiple security vulnerabilities but upgrading to 4.x is API
incompatible.  Remove due to lack of known users.
2024-02-26 19:04:05 +09:00
Andrew Gaul 107741f786 JCLOUDS-1629: Upgrade to Guice 7.0.0
This also changes from javax to jakarta annotations.
2024-02-26 15:27:29 +09:00
Andrew Gaul b5e4e1d0fd JCLOUDS-1627: Upgrade to Jakarta packages
This resolves an issue with newer Guice versions.
2024-02-23 18:04:48 +09:00
Andrew Gaul 6bb738195f Add a test for uploading large objects
This is disabled by default.  Also rework upload helper to avoid using
an unbounded amount of memory.
2024-02-21 21:57:51 +09:00
Andrew Gaul ed160d30c5 Copy Swift logging configuration to GCS
This avoids logging everything at trace level.
2024-02-21 21:56:29 +09:00
Jan Vermeulen 47f34770c9 JCLOUDS-1606: JCLOUDS-1608: Fix MPU off-by-one
Previously GCS could not upload large objects due to its 32 part
limit.
2024-02-21 21:35:16 +09:00
dependabot[bot] a983eaced9 Bump testng from 7.5 to 7.5.1 in /project
Bumps [testng](https://github.com/cbeust/testng) from 7.5 to 7.5.1.
- [Release notes](https://github.com/cbeust/testng/releases)
- [Changelog](https://github.com/testng-team/testng/blob/master/CHANGES.txt)
- [Commits](https://github.com/cbeust/testng/compare/7.5...7.5.1)

---
updated-dependencies:
- dependency-name: org.testng:testng
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-20 20:35:55 +09:00
Andrew Gaul 62abf39575 Correct testName
testng 7.5.1 previously failed with this.  References #171.
2024-02-20 20:20:28 +09:00
davidsenk 41d842d449
JCLOUDS-1519: Fix the authorization error with b2 application keys 2024-02-19 11:05:35 +09:00
Andrew Gaul 2a773c2a72 Upgrade netty to 3.10.6.Final 2024-02-18 21:32:12 +09:00
dependabot[bot] 8af11edaff Bump ch.qos.logback:logback-core from 1.2.9 to 1.2.13 in /project
Bumps [ch.qos.logback:logback-core](https://github.com/qos-ch/logback) from 1.2.9 to 1.2.13.
- [Commits](https://github.com/qos-ch/logback/compare/v_1.2.9...v_1.2.13)

---
updated-dependencies:
- dependency-name: ch.qos.logback:logback-core
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-18 21:26:42 +09:00
dependabot[bot] 20bbb21a51 Bump bcprov-ext-jdk18on from 1.71 to 1.74 in /drivers/bouncycastle
Bumps [bcprov-ext-jdk18on](https://github.com/bcgit/bc-java) from 1.71 to 1.74.
- [Changelog](https://github.com/bcgit/bc-java/blob/main/docs/releasenotes.html)
- [Commits](https://github.com/bcgit/bc-java/commits)

---
updated-dependencies:
- dependency-name: org.bouncycastle:bcprov-ext-jdk18on
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-18 21:08:03 +09:00
Mickael LANOE 6455fd8518 Parse out of order ListBucket response elements
Currently, jclouds assumes that the <ID> must occur before the
<DisplayName> tag. If the XML body does not adhere to this order, the
parser throws a NullPointerException.

Scaleway is a cloud provider that does not adhere to this order and
returns the <DisplayName> tag before the <ID> tag. The patch changes the
parser to not depend on the order of the tags.

See commit 04feb8f848 for a similar fix
2023-12-10 00:21:47 +09:00
jixinchi 9b01dbc983 no need to check contents contains md because it is a set 2023-11-14 21:48:21 +09:00
jixinchi 311a4102da fix typo and add unit test 2023-11-14 21:48:21 +09:00
jixinchi ef09dbb6ad do not remove md in DelimeterFilter then add it by commonPrefixes; just keep it 2023-11-14 21:48:21 +09:00
Valentin 47e3022519
FilesystemStorageStrategyImpl.getBlob() for a folder return blob as type Blob (#187)
Co-authored-by: Valentin Popov <popov@mailarchiva.ru>
2023-11-11 16:37:00 -05:00
Lars Hagen a2628f9cbf
implement support for SharedKey signature (#186)
* implement support for SharedKey signature

This is the recommended signature scheme for Azure, and the only scheme
that is supported by the Azurite emulator.

https://learn.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key

* Remove print statement

Co-authored-by: Ignasi Barrera <nacx@apache.org>

* Remove print statement

* simplify logic

---------

Co-authored-by: Ignasi Barrera <nacx@apache.org>
2023-10-20 11:19:06 +02:00
jixinchi eb1181daf5
check file exists in getContainerAccess (#185) 2023-10-12 18:18:30 +02:00
Edo Demirbilek 611b4c4a11
Adding access key auth support for openstack V3 (#183)
* Adding access key auth support for openstack V3 from an earlier commit + adding test

* added new line at the end to fix checkstyle warning
2023-09-04 23:49:45 +02:00
Ignasi Barrera 6b49389610
Fix checkstyle violation in google-cloud-storage (#184) 2023-09-04 23:49:17 +02:00
Andrew Gaul 1252f1951f Replace US_CENTRAL2 with US_CENTRAL1 in tests
US_CENTRAL2 has been removed.
2023-07-29 16:36:59 +09:00
Andrew Gaul 20ebc67fc8 Update google-cloud-storage regions 2023-07-29 16:36:59 +09:00
dependabot[bot] ef580d4aa7 Bump okio from 1.17.2 to 3.4.0 in /drivers/okhttp
Bumps [okio](https://github.com/square/okio) from 1.17.2 to 3.4.0.
- [Changelog](https://github.com/square/okio/blob/master/CHANGELOG.md)
- [Commits](https://github.com/square/okio/compare/okio-parent-1.17.2...parent-3.4.0)

---
updated-dependencies:
- dependency-name: com.squareup.okio:okio
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-29 16:16:26 +09:00
Andrew Gaul 132604d71a Add missing jclouds.blobstore.httpstream.url
Lacking this caused integration test failures with newer testng.
2023-07-29 15:30:28 +09:00
Andrew Gaul 8075bbe50a JCLOUDS-1618: Upgrade to gson 2.10.1 2023-06-15 19:05:39 -07:00
Andrew Gaul 2275637fb8 Remove bnd
This blocks upgrading gson.  References #176.
2023-06-15 18:43:55 -07:00
dependabot[bot] a00454bdd1 Bump guava from 31.1-jre to 32.0.0-jre in /project
Bumps [guava](https://github.com/google/guava) from 31.1-jre to 32.0.0-jre.
- [Release notes](https://github.com/google/guava/releases)
- [Commits](https://github.com/google/guava/commits)

---
updated-dependencies:
- dependency-name: com.google.guava:guava
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-14 20:19:30 -07:00
Andrew Gaul e7894e5a77 Use a class with a File member
Using gson on a File directly breaks Java strong encapsulation.
2023-06-14 20:17:28 -07:00
Andrew Gaul db69360b5f Revert "Revert "Upgrade to snakeyaml 2.0""
This reverts commit cf4a926afe.  The
current CI failure is unrelated to this dependency.
2023-06-14 20:16:28 -07:00
Andrew Gaul 39050e8fad Java 9/16 strong encapsulation fixes
* Remove Reflection access modification
* Make some fields public so reflection can access them with strong
  encapsulation
* Ignore test which fails due to differences in File serialization
2023-06-11 23:09:18 -07:00
Andrew Gaul c8ca330857 Remove unneeded -XX:MaxPermSize=256m
Newer JDK remove this flag.
2023-06-06 03:50:06 -07:00
Andrew Gaul 318138e6b0 Upgrade to surefire 3.1.0
Release notes:

https://github.com/apache/maven-surefire/releases/tag/surefire-3.0.0
https://github.com/apache/maven-surefire/releases/tag/surefire-3.1.0
2023-06-06 03:24:44 -07:00
Andrew Gaul 8a3c5ae992 Upgrade to EasyMock 5.1.0
This resolves a Java 17 compatibility issue.  Release notes:

https://github.com/easymock/easymock/releases/tag/easymock-5.0.0
https://github.com/easymock/easymock/releases/tag/easymock-5.0.1
https://github.com/easymock/easymock/releases/tag/easymock-5.1.0
2023-06-05 04:25:59 -07:00
Andrew Gaul cf4a926afe Revert "Upgrade to snakeyaml 2.0"
This reverts commit f82e94c337.  This
break CI due to an unexpected Java version mismatch.
2023-05-10 21:30:27 +09:00
Andrew Gaul f82e94c337 Upgrade to snakeyaml 2.0
This has some API changes.  Closes #169.
2023-04-16 16:43:36 +09:00
Ignasi Barrera 788f75f937 Replace TravisCI with GitHUb Actions 2023-02-08 08:33:39 +09:00
Andrew Gaul b7f28f1e6a Lazily open parts during LocalBlobStore complete MPU
This removes a previous workaround for opening too many
FileInputStream and exhausting rlimits.
2023-01-29 17:56:15 +09:00
Andrew Gaul 12de6ef74d Atomically replace objects in filesystem putBlob
Fixes gaul/s3proxy#490.  This fixes a regression from
41ce90ec36.  Continue to support
Windows logic although it is unclear if this is necessary.
2023-01-24 14:44:53 +09:00
Andrew Gaul 62632c9db6 JCLOUDS-1371: Optimize filesystem delimiter
populateBlobKeysInContainer will no longer recurse when the delimiter
matches "/".  This makes listing deep hierarchies with a delimiter
faster.  Note that the general LocalBlobStore handling is still
required for the general cases.  This requires removing a bogus test
case.  References gaul/s3proxy#473.
2023-01-24 14:44:34 +09:00
Andrew Gaul e478dd5452 JCLOUDS-1371: JCLOUDS-1488: optimize fs prefix
This reduces the number of stat calls required when prefix is deep in the
filesystem hierarchy.  Further optimizations to delimiter are possible.
References gaul/s3proxy#473.
2023-01-22 18:31:09 +09:00
Andrew Gaul 7e50ae630d Upgrade to Guice 5.1.0
Release notes:

https://github.com/google/guice/wiki/Guice510
2022-12-17 07:41:44 +09:00