Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,033 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

779 Commits

Author SHA1 Message Date
Chris Douglas 77ad73501c HADOOP-14991. Add missing figures to Resource Estimator tool. Contributed by Rui Li 
(cherry picked from commit 139cc752d2)
 2017-10-27 22:00:44 -07:00
Subru Krishnan 82c9b3bbb5 HADOOP-14840. Tool to estimate resource requirements of an application pipeline based on prior executions. (Rui Li via Subru). 
(cherry picked from commit 625039ef20)
 2017-10-25 16:01:13 -07:00
Wangda Tan 881263260c YARN-7318. Fix shell check warnings of SLS. (Gergely Novák via wangda) 
(cherry picked from commit 281d83604d)
 2017-10-20 16:02:04 -07:00
Steve Loughran 2bacb429a1 HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. 
Contributed by Andras Bokor.

(cherry picked from commit f36cbc8475)
 2017-10-20 22:29:22 +01:00
Haibo Chen 175c4472d5 YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo Chen) 
(cherry picked from commit 46eb1033a8)
 2017-10-17 16:04:49 -07:00
Haibo Chen c021c8a19f YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen) 
(cherry picked from commit c071aad5da)
 2017-10-05 15:51:08 -07:00
Yufei Gu f982ffe336 YARN-7211. AMSimulator in SLS does't work due to responseId refactoring.(Contributed by Botong Huang via Yufei Gu) 
(cherry picked from commit 79e37dc6db)
 2017-10-03 15:12:09 -07:00
Akira Ajisaka b658ae002f
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 
(cherry picked from commit c9b525bd94)
 2017-10-04 07:02:31 +09:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Aaron Fabbri 60b966cb45
HADOOP-14220 Enhance S3GuardTool with bucket-info and set-capacity commands, tests. Contributed by Steve Loughran 2017-09-25 16:01:15 -07:00
Jason Lowe 0eb0298027 MAPREDUCE-6967. gridmix/SleepReducer should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit b133dc5700)
 2017-09-22 15:30:20 -05:00
Steve Loughran 9f6b08f840
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure 
Contributed by Steve Loughran

(cherry picked from commit 2d2d97fa7d)
 2017-09-15 17:04:43 +01:00
Aaron Fabbri 62e8a5ceed
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran. 2017-09-14 16:12:29 -07:00
Kai Zheng b873b1395e HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:22:00 +08:00
John Zhuge bb107f5d28 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 12:57:58 -07:00
Steve Loughran 70c95d15e3
HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A. 
Contributed by Aaron Fabbri

(cherry picked from commit 5bbca80428)
 2017-09-08 12:08:36 +01:00
Steve Loughran 5757600b8d
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. 
Contributed by Yonger.

(cherry picked from commit 83449abb40)
 2017-09-07 19:58:03 +01:00
Steve Loughran e5bb78d3d9
HADOOP-14520. WASB: Block compaction for Azure Block Blobs. 
Contributed by Georgi Chalakov

(cherry picked from commit 13eda50003)
 2017-09-07 18:38:24 +01:00
John Zhuge 3dd5444b4d HADOOP-14103. Sort out hadoop-aws contract-test-options.xml. Contributed by John Zhuge. 
(cherry picked from commit 50506e90a8)
 2017-09-06 11:33:43 -07:00
Xiaoyu Yao fd951e3a37 HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin. 
(cherry picked from commit 7ba61235dba87bc292134194044d54d9a12a4d7c)
 2017-09-05 23:35:39 -07:00
Steve Loughran 727e79a78e HADOOP-14820 Wasb mkdirs security checks inconsistent with HDFS. 
Contributed by Sivaguru Sankaridurg

(cherry picked from commit 792eff9ea7)
 2017-09-05 14:21:16 -07:00
Allen Wittenauer a3fee475f7
HADOOP-14414. Calling maven-site-plugin directly for docs profile is unnecessary (Andras Bokor via aw) 2017-09-01 08:38:30 -07:00
Steve Loughran 621b43e254
HADOOP-13345 HS3Guard: Improved Consistency for S3A. 
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
 2017-09-01 14:13:41 +01:00
John Zhuge 7ecc6dbed6 HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 2017-08-31 21:13:22 -07:00
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms 
Signed-off-by: Chris Douglas <cdouglas@apache.org>
 2017-08-30 21:47:24 -07:00
Haibo Chen d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Yufei Gu 26fafc3597 YARN-7077. TestAMSimulator and TestNMSimulator fail. (Contributed by Akira Ajisaka via Yufei Gu) 2017-08-29 16:33:51 -07:00
Steve Loughran 9374f38820
HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory 
Contributed by Esfandiar Manii and Thomas Marquardt.
 2017-08-29 19:03:44 +01:00
Steve Loughran 021974f4cb
HADOOP-14802. Add support for using container saskeys for all accesses. 
Contributed by Sivaguru Sankaridurg
 2017-08-29 19:02:43 +01:00
Andrew Wang f29a0fc288 HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou. 2017-08-25 14:14:23 -07:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Steve Loughran 27ab5f7385
HADOOP-14787. AliyunOSS: Implement the `createNonRecursive` operator. 
Contributed by Genmao Yu
 2017-08-22 11:55:48 +01:00
Kai Zheng 267e19a09f HADOOP-14194. Aliyun OSS should not use empty endpoint as default. Contributed by Genmao Yu 2017-08-21 13:36:28 +08:00
Arun Suresh 8410d862d3 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 2017-08-20 07:54:09 -07:00
Steve Loughran c6b4e656b7
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. 
Contributed by Thomas Marquardt
 2017-08-18 14:13:40 +01:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Chris Douglas 645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
John Zhuge 7769e96149 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 2017-08-13 00:22:34 -07:00
Allen Wittenauer 4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Steve Loughran 54356b1e83
HADOOP-14183. Remove service loader config file for wasb fs. 
Contributed by Esfandiar Manii.
 2017-08-10 16:46:33 +01:00
Steve Loughran f4e1aa0508
HADOOP-14715. TestWasbRemoteCallHelper failing. 
Contributed by Esfandiar Manii.
 2017-08-08 23:37:47 +01:00
Chris Douglas 55a181f845 HADOOP-14730. Support protobuf FileStatus in AdlFileSystem. 2017-08-07 21:31:28 -07:00
Steve Loughran d91b7a8451
HADOOP-14722. Azure: BlockBlobInputStream position incorrect after seek. 
Contributed by Thomas Marquardt
 2017-08-06 20:19:23 +01:00
Steve Loughran 7fc324aabd
HADOOP-14126. Remove jackson, joda and other transient aws SDK dependencies from hadoop-aws. 
Contributed by Steve Loughran

(cherry picked from commit ced547d5f0)
 2017-08-04 11:09:08 +01:00
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Steve Loughran 778d4edd9a
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded. 
Contributed by Thomas Marquardt.
 2017-08-01 21:36:28 +01:00
John Zhuge f735ad1b67 HADOOP-14678. AdlFilesystem#initialize swallows exception when getting user name. Contributed by John Zhuge. 2017-07-28 09:46:54 -07:00
Jitendra Pandey a92bf39e23 HADOOP-14680. Azure: IndexOutOfBoundsException in BlockBlobInputStream. Contributed by Thomas Marquardt. 2017-07-25 16:26:48 -07:00
Jitendra Pandey f2921e51f0 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-24 23:01:01 -07:00
Ravi Prakash 94ca52ae9e HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 2017-07-24 16:01:45 -07:00