199 Commits

Author	SHA1	Message	Date
Sean Mackrory	7ce6b41509	HADOOP-15332. Fix typos in hadoop-aws markdown docs. Contributed by Gabor Bota.	2018-03-20 21:12:20 -07:00
Steve Loughran	1976e0066e	HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories. ... Contributed by Steve Loughran.	2018-03-15 18:05:14 +00:00
Chris Douglas	45d1b0fdcc	HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer	2018-03-12 20:05:39 -07:00
Steve Loughran	dd05871b8b	HADOOP-15297. Make S3A etag => checksum feature optional. ... Contributed by Steve Loughran.	2018-03-12 14:01:42 +00:00
Steve Loughran	e0307e53e2	HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. ... Contributed by Anis Elleuch. (cherry picked from commit 1dedc68f9d)	2018-03-07 18:51:10 +00:00
Steve Loughran	8110d6a0d5	HADOOP-13761. S3Guard: implement retries for DDB failures and throttling; translate exceptions. ... Contributed by Aaron Fabbri.	2018-03-05 14:06:20 +00:00
fang zhenyi	4d4dde5112	HADOOP-15223. Replace Collections.EMPTY* with empty* when available ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-02-18 22:19:39 +09:00
Steve Loughran	7ac88244c5	HADOOP-14507. Extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key. ... Contributed by Steve Loughran.	2018-02-16 16:37:06 +00:00
Steve Loughran	9a013b255f	HADOOP-15176. Enhance IAM Assumed Role support in S3A client. ... Contributed by Steve Loughran (cherry picked from commit 96c047fbb9)	2018-02-15 15:57:10 +00:00
Steve Loughran	b27ab7dd81	HADOOP-15076. Enhance S3A troubleshooting documents and add a performance document. ... Contributed by Steve Loughran. (cherry picked from commit c761e658f6)	2018-02-15 14:57:56 +00:00
Wangda Tan	60f9e60b3b	Preparing for 3.2.0 development ... Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938	2018-02-11 11:17:38 +08:00
Steve Loughran	1093a73689	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. ... Contributed by Aaron Fabbri	2018-01-18 13:13:58 +00:00
Steve Loughran	f274fe33ea	Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits." ... This reverts commit 35ad9b1dd2.	2018-01-18 12:35:57 +00:00
Aaron Fabbri	268ab4e027	HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran.	2018-01-17 00:05:24 -08:00
Steve Loughran	a0c71dcc33	HADOOP-15079. ITestS3AFileOperationCost#testFakeDirectoryDeletion failing ... after OutputCommitter patch. Contributed by Steve Loughran	2018-01-15 11:33:32 +00:00
Mukul Kumar Singh	2e1e9017aa	HADOOP-15172. Fix the javadoc warning in WriteOperationHelper.java ... Signed-off-by: Akira Ajisaka <aajisaka@apache.org>	2018-01-15 15:45:51 +09:00
Steve Loughran	1a09da7400	HADOOP-15163. Fix S3ACommitter documentation ... Contributed by Alessandro Andrioni. (cherry picked from commit 100e8a1ae1d930dde084af7d1281e491c7f124ec)	2018-01-10 15:37:07 +00:00
Steve Loughran	b62a5ece95	HADOOP-15161. s3a: Stream and common statistics missing from metrics ... Contributed by Sean Mackrory	2018-01-09 18:46:52 +00:00
Steve Loughran	c8ff0cc304	HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls. ... Contributed by Steve Loughran	2017-12-21 14:58:58 +00:00
Steve Loughran	ef450df443	HADOOP-15113. NPE in S3A getFileStatus: null instrumentation on using closed instance. ... Contributed by Steve Loughran.	2017-12-21 14:15:53 +00:00
Steve Loughran	1ba491ff90	HADOOP-14965. S3a input stream "normal" fadvise mode to be adaptive	2017-12-20 18:25:33 +00:00
Steve Loughran	35ad9b1dd2	HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits. ... Contributed by Aaron Fabbri	2017-12-18 21:19:06 +00:00
Kai Zheng	f86c81d923	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-14 11:14:52 +08:00
Kai Zheng	28792b6b7f	Revert "HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu" ... This reverts commit 479d6a5792262c977025c26fd4960574b0db6847	2017-12-14 11:05:11 +08:00
Kai Zheng	73b86979d6	HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu	2017-12-06 12:01:36 +08:00
Aaron Fabbri	6555af81a2	HADOOP-14475 Metrics of S3A don't print out when enabled. Contributed by Younger and Sean Mackrory.	2017-12-05 11:06:32 -08:00
Steve Loughran	3150c019ae	HADOOP-15071 S3a troubleshooting docs to add a couple more failure modes. ... Contributed by Steve Loughran	2017-12-05 15:05:41 +00:00
Rohith Sharma K S	3cd75845da	MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.	2017-11-27 22:01:00 +05:30
Steve Loughran	de8b6ca5ef	HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints. ... Contributed by Steve Loughran and Ryan Blue.	2017-11-22 15:28:12 +00:00
Akira Ajisaka	6903cf096e	HADOOP-13514. Upgrade maven surefire plugin to 2.20.1 ... Signed-off-by: Allen Wittenauer <aw@apache.org>	2017-11-19 12:39:37 -08:00
Aaron Fabbri	47011d7dd3	HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran	2017-09-25 15:59:38 -07:00
Aaron Fabbri	49467165a5	HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran.	2017-09-14 14:10:48 -07:00
Steve Loughran	5bbca80428	HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A. ... Contributed by Aaron Fabbri	2017-09-08 12:07:02 +01:00
Steve Loughran	83449abb40	HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. ... Contributed by Yonger.	2017-09-07 19:55:56 +01:00
John Zhuge	50506e90a8	HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge.	2017-09-05 23:26:57 -07:00
Andrew Wang	0d419c984f	Preparing for 3.1.0 development	2017-09-01 11:53:48 -07:00
Steve Loughran	621b43e254	HADOOP-13345 HS3Guard: Improved Consistency for S3A. ... Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu, Sean Mackrory, Steve Loughran and others.	2017-09-01 14:13:41 +01:00
Steve Loughran	7fc324aabd	HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. ... Contributed by Steve Loughran (cherry picked from commit ced547d5f0)	2017-08-04 11:09:08 +01:00
Andrew Wang	af2773f609	Updating version for 3.0.0-beta1 development	2017-06-29 17:57:40 -07:00
Mingliang Liu	72993b33b7	HADOOP-14596. AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error. Contributed by Steve Loughran ... Change-Id: I49173bf6163796903d64594a8ca8a4bd26ad2bfc	2017-06-29 17:07:52 -07:00
Steve Loughran	ee243e5289	HADOOP-14190. Add more on S3 regions to the s3a documentation. ... Contributed by Steve Loughran	2017-06-28 10:22:13 +01:00
Mingliang Liu	bc4dfe9c9c	HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu	2017-06-27 13:19:14 -07:00
Mingliang Liu	f214a9961f	HADOOP-14494. ITestJets3tNativeS3FileSystemContract tests NPEs in teardown if store undefined. Contributed by Steve Loughran	2017-06-15 11:16:16 -07:00
Mingliang Liu	6aeda55bb8	HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu	2017-06-05 11:26:56 -07:00
John Zhuge	6c6a7a5962	HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge.	2017-05-28 22:25:00 -07:00
Andrew Wang	16ad896d5c	Update maven version for 3.0.0-alpha4 development	2017-05-26 14:09:44 -07:00
Akira Ajisaka	6a52b5e144	HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu.	2017-05-25 15:36:44 +09:00
Mingliang Liu	6672810eea	HADOOP-14432. S3A copyFromLocalFile to be robust, tested. Contributed by Steve Loughran	2017-05-19 11:51:43 -07:00
Steve Loughran	ba70225cf6	HADOOP-11572. s3a delete() operation fails during a concurrent delete of child entries. ... Contributed by Steve Loughran. (cherry picked from commit 2ac5aab8d7)	2017-05-18 15:44:39 +01:00
Akira Ajisaka	0d5c8ed8e0	HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.	2017-05-11 16:37:32 -05:00