28e05dc1da
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests.
...
Contributed by Steve Loughran
2017-10-03 20:06:45 +01:00
a1afc6aa59
HADOOP-13345 S3Guard: Improved Consistency for S3A.
...
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
(cherry-picked from/based on commit 621b43e254
2017-09-29 11:29:22 +01:00
27a41848ac
HADOOP-14890 Move up to AWS SDK 1.11.199.
...
Contributed by Steve Loughran.
2017-09-26 20:11:10 +01:00
6e20ae954c
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter.
...
Contributed by Yonger.
(cherry picked from commit 83449abb40
2017-09-07 19:58:40 +01:00
4c34c03b27
HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.
2017-09-06 01:03:18 -07:00
ced547d5f0
HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws.
...
Contributed by Steve Loughran
2017-08-04 11:07:34 +01:00
2d69952925
HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran
...
Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc
(cherry picked from commit 72993b33b7
2017-06-29 17:08:51 -07:00
44d43a8848
HADOOP-14190. Add more on S3 regions to the s3a documentation.
...
Contributed by Steve Loughran
(cherry picked from commit ee243e5289
2017-06-28 10:26:05 +01:00
ea402c364f
HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu
...
(cherry picked from commit bc4dfe9c9c
2017-06-27 13:20:03 -07:00
867026d94e
HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu
...
(cherry picked from commit 6aeda55bb8
2017-06-05 11:31:13 -07:00
f72f194e24
HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.
...
(cherry picked from commit 6c6a7a5962
2017-05-28 22:29:03 -07:00
2ac5aab8d7
HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries.
...
Contributed by Steve Loughran.
2017-05-18 15:38:46 +01:00
d1da5ba3af
HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus.
...
Contributed by Steve Moist.
2017-04-25 17:42:30 +01:00
3b7bb7b94b
HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran
2017-04-20 17:07:57 -07:00
1ae6be46b4
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs.
...
Contributed by Steve Loughran
2017-04-19 10:19:43 +01:00
508389ee54
HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu
...
(cherry picked from commit 507845074970b90330db86bc1ff0124def550bb5)
2017-04-12 14:30:54 -07:00
1bfb382296
HADOOP-14301. Deprecate SharedInstanceProfileCredentialsProvider in branch-2. Contributed by Mingliang Liu.
2017-04-12 10:07:00 -07:00
8c21b2a259
HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka
...
(cherry picked from commit 5faa949b78
2017-04-03 11:07:56 -07:00
bb187e145a
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 0d053eeb30
2017-03-30 18:13:01 +09:00
b24b416db5
HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu
...
(cherry picked from commit 2e30aa72e0
2017-03-23 11:37:14 -07:00
20878d052c
HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject().
...
Contributed by Yuanbo Liu
2017-03-23 12:52:22 +00:00
b4481fef41
HADOOP-14204 S3A multipart commit failing,
...
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".
Contributed by Steve Loughran
(cherry picked from commit 595501c9dd
2017-03-21 13:14:43 +00:00
90e3283a0a
HADOOP-14059. typo in s3a rename(self, subdir) error message. Contributed by Steve Loughran.
2017-03-20 11:29:30 -07:00
ed0d426a88
HADOOP-14170. FileSystemContractBaseTest is not cleaning up test directory clearly. Contributed by Mingliang Liu
2017-03-14 15:00:59 -07:00
d2d0872f4d
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri.
...
(cherry picked from commit f597f4c43e
2017-03-07 15:16:18 +09:00
51fefbe38b
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry.
...
Contributed by Steve Loughran
2017-03-02 21:41:49 +00:00
29fe5af017
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures.
...
Contributed by Steve Loughran.
2017-02-25 15:31:26 +00:00
c6d7da2112
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory.
...
(cherry picked from commit ff87ca8441
2017-02-24 10:40:58 +00:00
db955f0844
HADOOP-14102. Relax error message assertion in S3A test ITestS3AEncryptionSSEC. Contributed by Mingliang Liu
...
(cherry picked from commit 0013090fb4
2017-02-22 14:38:57 -08:00
171b18693a
HADOOP-14099 Split S3 testing documentation out into its own file. Contributed by Steve Loughran.
2017-02-22 11:41:24 +00:00
6843659689
HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory.
2017-02-21 18:29:30 +00:00
274c02d2fa
HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan
2017-02-20 16:21:00 +00:00
8a88e8ea57
HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge
...
(cherry picked from commit b1c1f05b1d
2017-02-18 18:17:54 +00:00
b23e93d465
HADOOP-14040. Use shaded aws-sdk uber-JAR 1.11.86. Contributed by Steve Loughran and Sean Mackrory
...
(cherry picked from commit 658702efff
2017-02-16 16:59:31 -08:00
29e1007172
HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran
...
(cherry picked from commit bdad8b7b97
2017-02-16 16:42:56 -08:00
25ac54a52e
HADOOP-14058. Fix NativeS3FileSystemContractBaseTest#testDirWithDifferentMarkersWorks. Contributed by Yiqun Lin.
...
(cherry picked from commit b9f8491252
2017-02-15 01:46:59 +09:00
6d62d0ea87
HADOOP-13204. Add support for SSE-KMS and SSE-C in s3a filesystem. (Steve Moist via lei)
2017-02-11 15:02:19 -08:00
ec2f5b18ab
HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran
2017-01-11 17:24:25 +00:00
2dbd86bcf4
HADOOP-13871. ITestS3AInputStreamPerformance.testTimeToOpenAndReadWholeFileBlocks performance awful. Contributed by Steve Loughran
...
(cherry picked from commit c6a3923245
2016-12-12 15:06:39 -08:00
c37e814fae
HADOOP-13855. Fix a couple of the s3a statistic names to be consistent with the rest. Contributed by Steve Loughran
...
(cherry picked from commit 6b6bc4705d2b608e810ae7d1912eac49bfd4c6ca)
2016-12-02 13:48:38 -08:00
a4e0539636
HADOOP-13857. S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs. Contributed by Steve Loughran
...
(cherry picked from commit 2ff84a0040
2016-12-02 13:39:23 -08:00
39b74b6f9e
HADOOP-13830. Intermittent failure of ITestS3NContractRootDir#testRecursiveRootListing: "Can not create a Path from an empty string". Contributed by Steve Loughran
...
(cherry picked from commit 3fd844b99f
2016-11-30 13:04:44 -08:00
1b831d8e01
HADOOP-13823. s3a rename: fail if dest file exists. Contributed by Steve Loughran
...
(cherry picked from commit d60a60be8a
2016-11-28 16:35:18 -08:00
9701ab48e8
HADOOP-12804. Read Proxy Password from Credential Providers in S3 FileSystem. Contributed by Larry McCay.
2016-11-25 13:34:11 +00:00
f3fff5c2e1
HADOOP-13801 regression: ITestS3AMiniYarnCluster failing. Contributed by Steve Loughran
2016-11-23 21:37:04 +00:00
cae4a8c05d
HADOOP-13050. Upgrade to AWS SDK 10.11+. Contributed by Chris Nauroth and Steve Loughran
2016-11-22 14:05:58 -08:00
c4ccafdaff
HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi.
2016-10-31 20:52:49 +00:00
082d69ee66
HADOOP-13502. Split fs.contract.is-blobstore flag into more descriptive flags for use by contract tests. Contributed by Chris Nauroth.
...
(cherry picked from commit 1f8490a5ba
2016-10-26 08:57:29 -07:00
67e01f7218
HADOOP-13614. Purge some superfluous/obsolete S3 FS tests that are slowing test runs down. Contributed by Steve Loughran.
...
(cherry picked from commit 9cad3e2350
2016-10-26 08:27:33 -07:00
05d772e297
HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth.
...
(cherry picked from commit 309a43925c
2016-10-25 09:03:11 -07:00
Steve Loughran
John Zhuge
Mingliang Liu
Chris Nauroth
Akira Ajisaka
Arpit Agarwal
Lei Xu