18875 Commits

Author SHA1 Message Date
Robert Kanter
27e2b4b364 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter)
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
2018-07-18 16:07:48 -07:00
Eric Yang
d82edec3c0 YARN-8538. Fixed memory leaks in container-executor and test cases.
Contributed by Billie Rinaldi
2018-07-18 13:44:49 -04:00
Yiqun Lin
228508a0ee HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma.
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
2018-07-17 10:47:07 +08:00
Konstantin V Shvachko
53c7d82d53 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal.
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
2018-07-16 18:29:43 -07:00
Wei-Chiu Chuang
34f1dd03ee HADOOP-15598. DataChecksum calculate checksum is contented on hashtable synchronization. Contributed by Prasanth Jayachandran.
(cherry picked from commit 0c7a578927032d5d1ef3469283d7d1fb7dee2a56)
2018-07-16 11:33:43 -07:00
Wangda Tan
44beab0b63 YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda)
Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9
(cherry picked from commit 752dcce5f4cf0f6ebcb40a61f622f1a885c4bda7)
2018-07-16 11:04:08 -07:00
Wei-Chiu Chuang
ac9155d6cb HDFS-13524. Occasional "All datanodes are bad" error in TestLargeBlock#testLargeBlockSize. Contributed by Siyao Meng.
(cherry picked from commit 88b2794244d19b6432253eb649a375e5bcdcf964)
2018-07-16 10:52:56 -07:00
Inigo Goiri
4898edf4f7 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun.
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
2018-07-16 09:47:00 -07:00
Eric E Payne
9a79e893f7 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
2018-07-16 16:32:05 +00:00
bibinchundatt
677bbdcdc4 YARN-8434. Update federation documentation of Nodemanager configurations. Contributed by Bibin A Chundatt.
(cherry picked from commit 4523cc5637bc3558aa5796150b358ca8471773bb)
2018-07-15 13:57:14 +05:30
Jason Lowe
7cbb9597c4 YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan
(cherry picked from commit 17118f446c2387aa796849da8b69a845d9d307d3)
2018-07-13 10:06:38 -05:00
Xiao Chen
d5d987550a HDFS-12837. Intermittent failure in TestReencryptionWithKMS.
(cherry picked from commit b37074be5ab35c238e18bb9c3b89db6d7f8d0986)
2018-07-11 21:03:45 -07:00
Akira Ajisaka
fe256a98ff
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota.
(cherry picked from commit 418cc7f3aeabedc57c94aa9d4c4248c1476ac90e)
2018-07-11 14:47:21 -04:00
Wei-Chiu Chuang
f9fa3cb157 HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng.
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
2018-07-11 10:04:07 -07:00
Yiqun Lin
99e98bf19a HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma.
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
2018-07-11 22:15:07 +08:00
Steve Loughran
caf38532f3
HADOOP-15541. [s3a] Shouldn't try to drain stream before aborting
connection in case of timeout. Contributed by Sean Mackrory.

(cherry picked from commit d503f65b6689b19278ec2a0cf9da5a8762539de8)
2018-07-11 14:55:11 +01:00
bibinchundatt
2aaad4000a YARN-8491. TestServiceCLI#testEnableFastLaunch fail when umask is 077. Contributed by K G Bakthavachalam.
(cherry picked from commit 52e1bc8539ce769f47743d8b2d318a54c3887ba0)
2018-07-11 16:24:37 +05:30
Sunil G
9b4ead92c8 YARN-8512. ATSv2 entities are not published to HBase from second attempt onwards. Contributed by Rohith Sharma K S.
(cherry picked from commit 7f1d3d0e9dbe328fae0d43421665e0b6907b33fe)
2018-07-11 12:27:12 +05:30
Sunil G
6f10491e64 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe.
(cherry picked from commit 705e2c1f7cba51496b0d019ecedffbe5fb55c28b)
2018-07-10 20:12:47 +05:30
Steve Loughran
d54241e9c9
HADOOP-15384. distcp numListstatusThreads option doesn't get to -delete scan.
Contributed by Steve Loughran.

(cherry picked from commit ca8b80bf59c0570bb9172208d3a6c993a6854514)
2018-07-10 10:50:40 +01:00
Wangda Tan
8ad82ea5b5 YARN-8506. Make GetApplicationsRequestPBImpl thread safe. (wangda)
Change-Id: I2c006965375823c83036e7f45f7163d13c0bdf90
2018-07-09 11:35:15 -07:00
Andrew Wang
242b5acdb3 HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi.
(cherry picked from commit eecb5baaaaa54599aeae758abd4007e55e5b531f)
2018-07-09 15:17:34 +02:00
Yiqun Lin
e994c4f8aa HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng.
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
2018-07-09 15:07:47 +08:00
Rohith Sharma K S
785ed3f8ae YARN-8302. ATS v2 should handle HBase connection issue properly. Contributed by Billie Rinaldi.
(cherry picked from commit ba683204498c97654be4727ab9e128c433a45498)
2018-07-06 15:20:39 -07:00
Wei-Chiu Chuang
f5f4d0b7e7 HDFS-13121. NPE when request file descriptors when SC read. Contributed by Zsolt Venczel.
(cherry picked from commit 0247cb6318507afe06816e337a19f396afc53efa)
2018-07-06 15:01:10 -07:00
Xiaoyu Yao
ffe2019887 HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli.
(cherry picked from commit 629f7008b531bad469903e6a1f6f8cb4be691f36)
2018-07-06 11:56:51 -07:00
Yiqun Lin
48addcdbbe HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar.
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
2018-07-04 15:30:17 +08:00
Sunil G
844538d669 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan.
(cherry picked from commit 51654a3962bcd0482c0e1dd32765e9533e3f9158)
2018-07-03 10:00:05 -07:00
Junping Du
3d6ba2dd4e yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu.
(cherry picked from commit d9ba6f3656e8dc97d2813181e27d12e52dca4328)
2018-07-03 22:36:24 +08:00
Andrew Wang
3b266abdad HDFS-13712. BlockReaderRemote.read() logging improvement. Contributed by Gergo Repas.
(cherry picked from commit 344f324710522ffb27852c1a673c4f7d3d6eac4b)
2018-07-03 11:07:57 +02:00
Bharat Viswanadham
0d7b811ee6 HDFS-13715:diskbalancer does not work if one of the blockpools are empty on a Federated cluster. Contributed by Bharat Viswanadham
(cherry picked from commit 59a3038bc3d7913dca3de971026bc32cef536a2d)
2018-07-02 21:48:09 -07:00
Inigo Goiri
177ccbc43f HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak.
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
2018-07-02 16:37:12 -07:00
Sunil G
8ffc191f61 YARN-8415. TimelineWebServices.getEntity should throw ForbiddenException instead of 404 when ACL checks fail. Contributed by Suma Shivaprasad.
(cherry picked from commit fa9ef15ecd6dc30fb260e1c342a2b51505d39b6b)
2018-07-02 15:35:45 -07:00
Shane Kumpf
2f264764a4 YARN-8485. Priviledged container app launch is failing intermittently. Contributed by Eric Yang
(cherry picked from commit 53e267fa7232add3c21174382d91b2607aa6becf)
2018-07-02 16:23:23 -06:00
Eric Yang
0aa7272199 YARN-8465. Fixed docker container status for node manager restart.
Contributed by Shane Kumpf
2018-07-02 13:40:02 -04:00
Wei-Chiu Chuang
2ce892fcf5 HDFS-13635. Incorrect message when block is not found. Contributed by Gabor Bota.
(cherry picked from commit fef20a446f7bf9f29e0f0ee690987fb6fc78a031)
2018-07-02 09:43:44 -07:00
Andrew Wang
9f2d57697a HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon.
(cherry picked from commit f51da9c4d1423c2ac92eb4f40e973264e7e968cc)
2018-07-02 18:31:27 +02:00
Andrew Wang
2d2639beaa HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon.
(cherry picked from commit 5d748bd056a32f2c6922514cd0c5b31d866a9919)
2018-07-02 12:11:11 +02:00
Andrew Wang
0268b954e0 HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon.
(cherry picked from commit 6ba99741086170b83c38d3e7e715d9e8046a1e00)
2018-07-02 12:02:48 +02:00
Nanda kumar
e1cd5732a6 HADOOP-15574: Suppress build error if there are no docs after excluding private annotations. Contributed by Takanobu Asanuma.
(cherry picked from commit d40121845e0e1ace6b349180ced53292353dbb7c)
2018-07-01 17:16:48 +05:30
Eric E Payne
012baf4313 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan.
(cherry picked from commit d36f6b9e93e4c30d24d0e837cb00bd24ffa8f274)
2018-06-29 18:22:51 +00:00
Jason Lowe
dbb0a5f51f YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang
(cherry picked from commit 100470140d86eede0fa240a9aa93226f274ee4f5)
2018-06-29 13:11:41 -05:00
Rohith Sharma K S
00406f583b YARN-8469. [UI2] URL needs to be trimmed to handle index.html redirection while accessing via knox. Contributed by Sunil Govindan.
(cherry picked from commit a820738e347854aed7cdaa7758d9c0830df624e3)
2018-06-29 10:39:30 -07:00
Sunil G
e7f2c9886a YARN-8455. Add basic ACL check for all ATS v2 REST APIs. Contributed by Rohith Sharma K S.
(cherry picked from commit 469b29c0817b7bf1902c9195c4f8d031a909e1c9)
2018-06-29 10:03:53 -07:00
Sunil G
122cb81b65 YARN-8379. Improve balancing resources in already satisfied queues by using Capacity Scheduler preemption. Contributed by Zian Chen.
(cherry picked from commit 291194302cc1a875d6d94ea93cf1184a3f1fc2cc)
2018-06-28 10:24:03 -07:00
Eric Yang
3ce3571a34 YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService.
Contributed by Chandni Singh
2018-06-28 13:11:56 -04:00
Bharat Viswanadham
8fa530746f HDFS-13705:The native ISA-L library loading failure should be made warning rather than an error message. Contributed by Shashikant Banerjee.
(cherry picked from commit d3fa83a44b01c85f39bfb4deaf2972912ac61ca3)
2018-06-28 09:26:58 -07:00
Sunil G
3c68d4ce45 YARN-8401. [UI2] new ui is not accessible with out internet connection. Contributed by Bibin A Chundatt.
(cherry picked from commit fbaff369e9b9022723a7b2c6f25e71122a8f8a15)
2018-06-27 10:35:50 -07:00
Wangda Tan
ec37e02bc9 YARN-8464. Async scheduling thread could be interrupted when there are no NodeManagers in cluster. (Sunil G via wangda)
Change-Id: I4f5f856373378685713e77752ba6cf0988a66065
(cherry picked from commit bedc4fe0799cf3b161100acc521fc62a97793427)
2018-06-26 19:34:15 -07:00
Wangda Tan
8f226f4f1d YARN-8423. GPU does not get released even though the application gets killed. (Sunil G via wangda)
Change-Id: I570db7d60f8c6c21762dd618a9207d1107c486a0
(cherry picked from commit ada8f63d0b3739d245300461387b0516dc92ccf9)
2018-06-26 19:34:10 -07:00