Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,715 Commits 326 Branches 371 Tags
Commit Graph

3075 Commits

Author SHA1 Message Date
Haibo Chen
49590e7c6b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed504ee81d4b74d340c1815bdbb3c9b14)
 2018-08-24 13:27:54 -05:00
Jason Lowe
79ebbec4c8 YARN-8649. NPE in localizer hearbeat processing if a container is killed while localizing. Contributed by lujie 
(cherry picked from commit 585ebd873a55bedd2a364d256837f08ada8ba032)
 2018-08-23 09:40:42 -05:00
Rohith Sharma K S
7278566f27 YARN-8129. Improve error message for invalid value in fields attribute. Contributed by Abhishek Modi. 
(cherry picked from commit d3fef7a5c5b83d27e87b5e49928254a7d1b935e5)
 2018-08-21 12:11:32 +05:30
Rohith Sharma K S
675aa2bbc0 YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff605262aaf3dbd926258afcadc86c72c0)
 2018-08-18 11:06:14 +05:30
Haibo Chen
8118b14db8 YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit d1274c3b71549cb000868500c293cafd880b3713)
 2018-08-18 11:06:02 +05:30
Jason Lowe
c72674ee3c YARN-8640. Restore previous state in container-executor after failure. Contributed by Jim Brennan 
(cherry picked from commit d1d129aa9deecebf42261947fcb0b2ca46dacad5)
 2018-08-14 10:29:58 -05:00
Jonathan Hung
0420ca5a6f YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-10 15:22:11 -07:00
Jason Lowe
b0a364171d YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 
(cherry picked from commit cd04e954d2db27f0a15b7d1c492b7cdb656a51db)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/TestContainer.java
 2018-08-09 10:31:11 -05:00
Sunil G
a3675f382a YARN-8397. Potential thread leak in ActivitiesManager. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6310c0d17d6422a595f856a55b4f1fb82be43739)
 2018-08-01 08:36:12 +05:30
Jonathan Hung
0d6e1a2aab YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 18:06:25 -07:00
Haibo Chen
a4fc0279fc YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 2018-07-30 09:26:00 -07:00
bibinchundatt
9a3b006685 YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2018-07-30 21:31:47 +05:30
Eric E Payne
299dffc72d YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 16:49:03 +00:00
Sunil G
1d8fce0d2f YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 
(cherry picked from commit 35ce6eb1f526ce3db7e015fb1761eee15604100c)
 2018-07-24 22:21:44 +05:30
bibinchundatt
1f713d6c66 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 
(cherry picked from commit ff7c2eda34c2c40ad71b50df6462a661bd213fbd)
 2018-07-24 16:32:21 +05:30
Robert Kanter
6e0db6fe1a YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 
(cherry picked from commit 1bc106a738a6ce4f7ed025d556bb44c1ede022e3)
 2018-07-18 16:10:57 -07:00
Robert Kanter
c1dc4ca2c6 Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 0838fe833738e04f5e6f6408e97866d77bebbf30)
 2018-07-18 16:10:57 -07:00
Robert Kanter
d61d84279f Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 
(cherry picked from commit 351cf87c92872d90f62c476f85ae4d02e485769c)
 2018-07-18 16:10:57 -07:00
Eric E Payne
5738bd8a10 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
 2018-07-16 16:53:07 +00:00
Jason Lowe
1ae35834a2 YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 
(cherry picked from commit 17118f446c2387aa796849da8b69a845d9d307d3)
 2018-07-13 10:08:21 -05:00
Sunil G
ce7c6762f4 YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 
(cherry picked from commit 705e2c1f7cba51496b0d019ecedffbe5fb55c28b)
 2018-07-10 20:13:15 +05:30
Junping Du
0a6942d58c yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 
(cherry picked from commit d9ba6f3656e8dc97d2813181e27d12e52dca4328)
(cherry picked from commit 3d6ba2dd4e4f1c42522c837ca96ee5d13f1491a4)
 2018-07-09 16:22:58 +08:00
Jason Lowe
90631e6dbf YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 
(cherry picked from commit 100470140d86eede0fa240a9aa93226f274ee4f5)
 2018-06-29 13:14:20 -05:00
Weiwei Yang
3151e95d27 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 2018-06-25 09:46:32 +08:00
Rohith Sharma K S
6542e31b78 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 
(cherry picked from commit 9119b3cf8f883aa2d5df534afc0c50249fed03c6)
 2018-06-14 13:51:05 +05:30
Sunil G
d9b00cdd6b YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b932e0ee69ba61f5796388c175d79195a)
 2018-06-13 16:10:24 +05:30
Weiwei Yang
ef105abb70 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 
(Cherry picked from commit 29024a62038c297f11e8992601f2522ffffc7da7)
 2018-06-13 13:57:35 +08:00
Inigo Goiri
6a31dc9927 HADOOP-15529. ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 6e756e8a620e4d6dc3192986679060c52063489b)
 2018-06-12 10:25:36 -07:00
Rohith Sharma K S
96e7b7e8ae YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:25:22 +05:30
Inigo Goiri
26ed145763 YARN-8370. Some Node Manager tests fail on Windows due to improper path/file separator. Contributed by Anbang Hu. 
(cherry picked from commit 2b2f672022547e8c19658213ac5a4090bf5b6c72)
 2018-06-11 19:26:58 -07:00
Inigo Goiri
500057c40c YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:38 -07:00
Robert Kanter
d5e6d0d5f4 YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a44390bc1a01113dce4be4e629637c3e8a)
 2018-06-04 15:42:46 -07:00
Miklos Szegedi
a3afd69051 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 11:07:08 -07:00
Yongjun Zhang
f0de11ba98 Preparing for 3.0.4 development 2018-05-29 23:40:26 -07:00
Jason Lowe
d5708bbcdc YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:15:07 -05:00
Sunil G
521ada1a11 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669baf660df3b70d58987d6e68aec54d6fa)
 2018-05-28 16:34:29 +05:30
Inigo Goiri
7c5a5f31dc YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit e99e5bf104e9664bc1b43a2639d87355d47a77e2)
 2018-05-23 14:16:37 -07:00
Wangda Tan
a5a9c8cf0f YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0f0ad1c877a062b19091e3e15a33ec4)
 2018-05-22 10:34:15 -05:00
Eric E Payne
1b6c662546 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdeef62945b592d5761666ad026b629c0b)
 2018-05-21 20:31:39 +00:00
Weiwei Yang
c48dce09a2 YARN-8278. DistributedScheduling is not working in HA. Contributed by Bibin A Chundatt. 
(Cherry picked from commit 2bb647bb91439e82cf7298e963bb5f7f80bbc3cb)
 2018-05-15 18:55:31 +08:00
Haibo Chen
237078c7d6 YARN-8130 Race condition when container events are published for KILLED applications. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 2d00a0c71b5dde31e2cf8fcb96d9d541d41fb879)
 2018-05-15 11:50:47 +05:30
Vrushali C
67255cd000 YARN-8247 Incorrect HTTP status code returned by ATSv2 for non-whitelisted users. Contributed by Rohith Sharma K S 
(cherry picked from commit 3c95ca4f21dcfcaabdd0694e7d005a45baba953f)
 2018-05-14 10:32:33 +05:30
Jason Lowe
e628206c26 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 
(cherry picked from commit dc912994a1bcb511dfda32a0649cef0c9bdc47d3)
 2018-05-11 14:17:19 -05:00
Weiwei Yang
4e46dc764f YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 2018-05-11 11:04:50 +08:00
bibinchundatt
c198550cc7 YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a5266c8b8351f338f5fc8087a203c68cac)
 2018-05-10 09:32:45 +05:30
Rohith Sharma K S
f13ec73c29 YARN-8253. HTTPS Ats v2 api call fails with 'bad HTTP parsed'. Contributed by Charan Hebri. 
(cherry picked from commit 7450583721757b8af2945ebd9be1a9efed11444c)
 2018-05-08 12:32:25 +05:30
Weiwei Yang
a0b7abf278 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f676237290d18ddbbc9aec369267692)
 2018-05-07 13:59:50 +08:00
Rohith Sharma K S
317deaafc4 YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 
(cherry picked from commit ee2ce923a922bfc3e89ad6f0f6a25e776fe91ffb)
 2018-05-03 18:21:11 +05:30
Weiwei Yang
c2ed611885 YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 
(Cherry picked from commit 251f528814c4a4647cac0af6effb9a73135db180)
 2018-05-02 18:07:34 +08:00
Yiqun Lin
01e924a2ce
YARN-6385. Fix checkstyle warnings in TestFileSystemApplicationHistoryStore 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3265b55119d39ecbda6d75be04a9a1bf59c631f1)
 2018-05-02 18:16:09 +09:00