Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,555 Commits 326 Branches 371 Tags
Commit Graph

18555 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Anu Engineer
2ebd553072 HDFS-13727. Log full stack trace if DiskBalancer exits with an unhandled exception. 
Contributed by Gabor Bota.

(cherry picked from commit 64e739e344ac474046d4f4ecf0865dd92be13762)
 2018-07-27 09:43:44 -07:00
Eric Yang
d6627cda9d YARN-8429. Improve diagnostic message when artifact is not set properly. 
Contributed by Gour Saha

(cherry picked from commit 8d3c068e59fdddd18e3f8260713fee83c458aa1d)
 2018-07-27 11:29:43 -04:00
Eric Yang
a869bd970e HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds. 
Contributed by Gabor Bota

(cherry picked from commit 77721f39e26b630352a1f4087524a3fbd21ff06e)
 2018-07-26 18:47:58 -04:00
Eric Yang
177f6045ac YARN-8545. Return allocated resource to RM for failed container. 
Contributed by Chandni Singh

(cherry picked from commit 40fad32824d2f8f960c779d78357e62103453da0)
 2018-07-26 18:25:41 -04:00
Eric Yang
8e3807afe0 YARN-8330. Improved publishing ALLOCATED events to ATS. 
Contributed by Suma Shivaprasad

(cherry picked from commit f93ecf5c1e0b3db27424963814fc01ec43eb76e0)
 2018-07-25 18:51:42 -04:00
Wei-Chiu Chuang
964f3454d1 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
 2018-07-25 11:05:18 -07:00
Eric E Payne
830ef12af8 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 16:30:30 +00:00
bibinchundatt
d2212c20c5 YARN-8577. Fix the broken anchor in SLS site-doc. Contributed by Weiwei Yang. 
(cherry picked from commit 3d3158cea4580eb2e3b2af635c3a7d30f4dbb873)
 2018-07-25 19:00:12 +05:30
bibinchundatt
8e65057eb1 YARN-8541. RM startup failure on recovery after user deletion. Contributed by Bibin A Chundatt. 2018-07-25 15:54:32 +05:30
Weiwei Yang
b89624a943 YARN-8546. Resource leak caused by a reserved container being released more than once under async scheduling. Contributed by Tao Yang. 
(Cherry-picked from commit 5be9f4a5d05c9cb99348719fe35626b1de3055db)
 2018-07-25 17:53:40 +08:00
Gera Shegalov
1396fa2a27 HADOOP-15612. Improve exception when tfile fails to load LzoCodec. (gera) 
(cherry picked from commit 6bec03cfc8bdcf6aa3df9c22231ab959ba31f2f5)
 2018-07-24 23:05:34 -07:00
Xiao Chen
40c06b389a HADOOP-15609. Retry KMS calls when SSLHandshakeException occurs. Contributed by Kitti Nanasi. 
(cherry picked from commit 81d59506e539673edde12e19c0df5c2edd9d02ad)
 2018-07-24 21:46:21 -07:00
Daniel Templeton
00c476abd8 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
(cherry picked from commit 849c45db187224095b13fe297a4d7377fbb9d2cd)
 2018-07-24 16:12:43 -07:00
Haibo Chen
7e7792dd7b YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 
(cherry picked from commit 9d3c39e9dd88b8f32223c01328581bb68507d415)
 2018-07-24 12:50:43 -07:00
Sunil G
4488fd8295 YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 
(cherry picked from commit 35ce6eb1f526ce3db7e015fb1761eee15604100c)
 2018-07-24 22:21:15 +05:30
bibinchundatt
a684a2efb8 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 
(cherry picked from commit ff7c2eda34c2c40ad71b50df6462a661bd213fbd)
 2018-07-24 16:30:31 +05:30
bibinchundatt
0710107f8d YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 
(cherry picked from commit 84612788339392fcda1aef0e27c43f5c6b2a19e5)
 2018-07-24 13:11:31 +05:30
Yiqun Lin
5aca0588ea HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 17a87977f29ced49724f561a68565217c8cb4e94)
 2018-07-24 11:19:21 +08:00
Eric Yang
23b8546a80 YARN-8380. Support bind propagation options for mounts in docker runtime. 
Contributed by Billie Rinaldi

(cherry picked from commit 8688a0c7f88f2adf1a7fce695e06f3dd1f745080)
 2018-07-23 20:13:41 -04:00
Eric Yang
e665c0a9dd YARN-8360. Improve YARN service restart policy and node manager auto restart policy. 
Contributed by Suma Shivaprasad

(cherry picked from commit 84d7bf1eeff6b9418361afa4aa713e5e6f771365)
 2018-07-23 12:59:37 -04:00
Eric Yang
4f2a129f2e YARN-8301. Added YARN service upgrade instructions. 
Contributed by Chandni Singh

(cherry picked from commit 10014a4d88f239d3c072e51bc0739cba1fca9406)
 2018-07-20 19:48:19 -04:00
Weiwei Yang
004e1f248e YARN-8528. Final states in ContainerAllocation might be modified externally causing unexpected allocation results. Contributed by Xintong Song. 2018-07-20 22:43:47 +08:00
Yiqun Lin
823d576a66 HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 
(cherry picked from commit 7b25fb949bf6f02df997beeca7df46c9e84c8d96)
 2018-07-20 17:31:13 +08:00
Wangda Tan
23624c9248 MAPREDUCE-7118. Distributed cache conflicts breaks backwards compatability. (Jason Lowe via wangda) 
Change-Id: I89ab4852b4ad305fec19812e8931c59d96581376
(cherry picked from commit b3b4d4ccb53fdf8dacc66e912822b34f8b3bf215)
 2018-07-19 14:26:40 -07:00
Steve Loughran
749fff577e
HADOOP-15547/ WASB: improve listStatus performance. 
Contributed by Thomas Marquardt.
 2018-07-19 12:29:21 -07:00
Eric Yang
76b8beb289 YARN-8501. Reduce complexity of RMWebServices getApps method. 
Contributed by Szilard Nemeth

(cherry picked from commit 5836e0a46bf9793e0a61bb8ec46536f4a67d38d7)
 2018-07-19 12:32:55 -04:00
Kihwal Lee
a147098c4f HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 
(cherry picked from commit ccf2db7fc2688d262df3309007cb12a4dfedc179)
 2018-07-19 11:19:19 -05:00
Eric Yang
a607c02f15 HADOOP-15610. Fixed pylint version for Hadoop docker image. 
Contributed by Jack Bearden

(cherry picked from commit ba1ab08fdae96ad7c9c4f4bf8672abd741b7f758)
 2018-07-18 20:11:39 -04:00
Robert Kanter
dfa71428ea YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 
(cherry picked from commit 1bc106a738a6ce4f7ed025d556bb44c1ede022e3)
 2018-07-18 16:07:48 -07:00
Robert Kanter
1c7d916347 Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 0838fe833738e04f5e6f6408e97866d77bebbf30)
 2018-07-18 16:07:48 -07:00
Robert Kanter
27e2b4b364 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
 2018-07-18 16:07:48 -07:00
Eric Yang
d82edec3c0 YARN-8538. Fixed memory leaks in container-executor and test cases. 
Contributed by Billie Rinaldi
 2018-07-18 13:44:49 -04:00
Yiqun Lin
228508a0ee HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
 2018-07-17 10:47:07 +08:00
Konstantin V Shvachko
53c7d82d53 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:29:43 -07:00
Wei-Chiu Chuang
34f1dd03ee HADOOP-15598. DataChecksum calculate checksum is contented on hashtable synchronization. Contributed by Prasanth Jayachandran. 
(cherry picked from commit 0c7a578927032d5d1ef3469283d7d1fb7dee2a56)
 2018-07-16 11:33:43 -07:00
Wangda Tan
44beab0b63 YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda) 
Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9
(cherry picked from commit 752dcce5f4cf0f6ebcb40a61f622f1a885c4bda7)
 2018-07-16 11:04:08 -07:00
Wei-Chiu Chuang
ac9155d6cb HDFS-13524. Occasional "All datanodes are bad" error in TestLargeBlock#testLargeBlockSize. Contributed by Siyao Meng. 
(cherry picked from commit 88b2794244d19b6432253eb649a375e5bcdcf964)
 2018-07-16 10:52:56 -07:00
Inigo Goiri
4898edf4f7 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
 2018-07-16 09:47:00 -07:00
Eric E Payne
9a79e893f7 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
 2018-07-16 16:32:05 +00:00
bibinchundatt
677bbdcdc4 YARN-8434. Update federation documentation of Nodemanager configurations. Contributed by Bibin A Chundatt. 
(cherry picked from commit 4523cc5637bc3558aa5796150b358ca8471773bb)
 2018-07-15 13:57:14 +05:30
Jason Lowe
7cbb9597c4 YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 
(cherry picked from commit 17118f446c2387aa796849da8b69a845d9d307d3)
 2018-07-13 10:06:38 -05:00
Xiao Chen
d5d987550a HDFS-12837. Intermittent failure in TestReencryptionWithKMS. 
(cherry picked from commit b37074be5ab35c238e18bb9c3b89db6d7f8d0986)
 2018-07-11 21:03:45 -07:00
Akira Ajisaka
fe256a98ff
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 
(cherry picked from commit 418cc7f3aeabedc57c94aa9d4c4248c1476ac90e)
 2018-07-11 14:47:21 -04:00
Wei-Chiu Chuang
f9fa3cb157 HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
 2018-07-11 10:04:07 -07:00
Yiqun Lin
99e98bf19a HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
 2018-07-11 22:15:07 +08:00
Steve Loughran
caf38532f3
HADOOP-15541. [s3a] Shouldn't try to drain stream before aborting 
connection in case of timeout. Contributed by Sean Mackrory.

(cherry picked from commit d503f65b6689b19278ec2a0cf9da5a8762539de8)
 2018-07-11 14:55:11 +01:00
bibinchundatt
2aaad4000a YARN-8491. TestServiceCLI#testEnableFastLaunch fail when umask is 077. Contributed by K G Bakthavachalam. 
(cherry picked from commit 52e1bc8539ce769f47743d8b2d318a54c3887ba0)
 2018-07-11 16:24:37 +05:30
Sunil G
9b4ead92c8 YARN-8512. ATSv2 entities are not published to HBase from second attempt onwards. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7f1d3d0e9dbe328fae0d43421665e0b6907b33fe)
 2018-07-11 12:27:12 +05:30
Sunil G
6f10491e64 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 
(cherry picked from commit 705e2c1f7cba51496b0d019ecedffbe5fb55c28b)
 2018-07-10 20:12:47 +05:30
Steve Loughran
d54241e9c9
HADOOP-15384. distcp numListstatusThreads option doesn't get to -delete scan. 
Contributed by Steve Loughran.

(cherry picked from commit ca8b80bf59c0570bb9172208d3a6c993a6854514)
 2018-07-10 10:50:40 +01:00