188 Commits

Author	SHA1	Message	Date
Steve Loughran	1093a73689	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. ... Contributed by Aaron Fabbri	2018-01-18 13:13:58 +00:00
Steve Loughran	f274fe33ea	Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits." ... This reverts commit 35ad9b1dd2.	2018-01-18 12:35:57 +00:00
Aaron Fabbri	268ab4e027	HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran.	2018-01-17 00:05:24 -08:00
Steve Loughran	a0c71dcc33	HADOOP-15079. ITestS3AFileOperationCost#testFakeDirectoryDeletion failing ... after OutputCommitter patch. Contributed by Steve Loughran	2018-01-15 11:33:32 +00:00
Mukul Kumar Singh	2e1e9017aa	HADOOP-15172. Fix the javadoc warning in WriteOperationHelper.java ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-01-15 15:45:51 +09:00
Steve Loughran	1a09da7400	HADOOP-15163. Fix S3ACommitter documentation ... Contributed by Alessandro Andrioni. (cherry picked from commit 100e8a1ae1d930dde084af7d1281e491c7f124ec)	2018-01-10 15:37:07 +00:00
Steve Loughran	b62a5ece95	HADOOP-15161. s3a: Stream and common statistics missing from metrics ... Contributed by Sean Mackrory	2018-01-09 18:46:52 +00:00
Steve Loughran	c8ff0cc304	HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. ... Contributed by Steve Loughran	2017-12-21 14:58:58 +00:00
Steve Loughran	ef450df443	HADOOP-15113. NPE in S3A getFileStatus: null instrumentation on using closed instance. ... Contributed by Steve Loughran.	2017-12-21 14:15:53 +00:00
Steve Loughran	1ba491ff90	HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive	2017-12-20 18:25:33 +00:00
Steve Loughran	35ad9b1dd2	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. ... Contributed by Aaron Fabbri	2017-12-18 21:19:06 +00:00
Kai Zheng	f86c81d923	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-14 11:14:52 +08:00
Kai Zheng	28792b6b7f	Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu" ... This reverts commit 479d6a5792262c977025c26fd4960574b0db6847	2017-12-14 11:05:11 +08:00
Kai Zheng	73b86979d6	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-06 12:01:36 +08:00
Aaron Fabbri	6555af81a2	HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.	2017-12-05 11:06:32 -08:00
Steve Loughran	3150c019ae	HADOOP-15071 S3a troubleshooting docs to add a couple more failure modes. ... Contributed by Steve Loughran	2017-12-05 15:05:41 +00:00
Rohith Sharma K S	3cd75845da	MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.	2017-11-27 22:01:00 +05:30
Steve Loughran	de8b6ca5ef	HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. ... Contributed by Steve Loughran and Ryan Blue.	2017-11-22 15:28:12 +00:00
Akira Ajisaka	6903cf096e	HADOOP-13514. Upgrade maven surefire plugin to 2.20.1 ... Signed-off-by: Allen Wittenauer <aw@apache.org>	2017-11-19 12:39:37 -08:00
Aaron Fabbri	47011d7dd3	HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran	2017-09-25 15:59:38 -07:00
Aaron Fabbri	49467165a5	HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.	2017-09-14 14:10:48 -07:00
Steve Loughran	5bbca80428	HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A. ... Contributed by Aaron Fabbri	2017-09-08 12:07:02 +01:00
Steve Loughran	83449abb40	HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. ... Contributed by Yonger.	2017-09-07 19:55:56 +01:00
John Zhuge	50506e90a8	HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.	2017-09-05 23:26:57 -07:00
Andrew Wang	0d419c984f	Preparing for 3.1.0 development	2017-09-01 11:53:48 -07:00
Steve Loughran	621b43e254	HADOOP-13345 HS3Guard: Improved Consistency for S3A. ... Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu, Sean Mackrory, Steve Loughran and others.	2017-09-01 14:13:41 +01:00
Steve Loughran	7fc324aabd	HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. ... Contributed by Steve Loughran (cherry picked from commit ced547d5f0)	2017-08-04 11:09:08 +01:00
Andrew Wang	af2773f609	Updating version for 3.0.0-beta1 development	2017-06-29 17:57:40 -07:00
Mingliang Liu	72993b33b7	HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran ... Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc	2017-06-29 17:07:52 -07:00
Steve Loughran	ee243e5289	HADOOP-14190. Add more on S3 regions to the s3a documentation. ... Contributed by Steve Loughran	2017-06-28 10:22:13 +01:00
Mingliang Liu	bc4dfe9c9c	HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu	2017-06-27 13:19:14 -07:00
Mingliang Liu	f214a9961f	HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran	2017-06-15 11:16:16 -07:00
Mingliang Liu	6aeda55bb8	HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu	2017-06-05 11:26:56 -07:00
John Zhuge	6c6a7a5962	HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.	2017-05-28 22:25:00 -07:00
Andrew Wang	16ad896d5c	Update maven version for 3.0.0-alpha4 development	2017-05-26 14:09:44 -07:00
Akira Ajisaka	6a52b5e144	HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.	2017-05-25 15:36:44 +09:00
Mingliang Liu	6672810eea	HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran	2017-05-19 11:51:43 -07:00
Steve Loughran	ba70225cf6	HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries. ... Contributed by Steve Loughran. (cherry picked from commit 2ac5aab8d7)	2017-05-18 15:44:39 +01:00
Akira Ajisaka	0d5c8ed8e0	HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.	2017-05-11 16:37:32 -05:00
Steve Loughran	5f934f8386	HADOOP-14305 S3A SSE tests won't run in parallel: Bad request in directory GetFileStatus. ... Contributed by Steve Moist.	2017-04-24 20:33:19 +01:00
Mingliang Liu	667966c13c	HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran	2017-04-20 17:13:36 -07:00
Steve Loughran	6b015d00c9	HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. ... Contributed by Steve Loughran	2017-04-19 10:21:44 +01:00
Mingliang Liu	b053fdc547	HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu	2017-04-12 14:30:34 -07:00
Chris Nauroth	b8305e6d06	HADOOP-14248. Retire SharedInstanceProfileCredentialsProvider in trunk. Contributed by Mingliang Liu.	2017-04-12 10:02:13 -07:00
Mingliang Liu	5faa949b78	HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka	2017-04-03 11:07:14 -07:00
Akira Ajisaka	0d053eeb30	HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula.	2017-03-30 18:11:50 +09:00
Andrew Wang	4966a6e26e	HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.	2017-03-29 14:37:21 -07:00
Andrew Wang	52b00600df	HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.	2017-03-24 11:44:46 -07:00
Mingliang Liu	2e30aa72e0	HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu	2017-03-23 11:33:29 -07:00
Steve Loughran	a5a4867f3b	HADOOP-14120 needless S3AFileSystem.setOptionalPutRequestParameters in S3ABlockOutputStream putObject(). ... Contributed by Yuanbo Liu (cherry picked from commit 20878d052c)	2017-03-23 12:54:54 +00:00