Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,609 Commits 326 Branches 371 Tags
Commit Graph

1833 Commits

Author SHA1 Message Date
Giovanni Matteo Fumarola
8e6807ef4a YARN-8673. [AMRMProxy] More robust responseId resync after an YarnRM master slave switch. Contributed by Botong Huang. 2018-08-21 13:09:33 -07:00
Rohith Sharma K S
c68d1d49ca YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff605262aaf3dbd926258afcadc86c72c0)
 2018-08-18 11:04:20 +05:30
Jonathan Hung
7abffe4529 YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-10 15:34:21 -07:00
Rohith Sharma K S
21e416ad27 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 2018-08-01 22:25:53 +05:30
Eric E Payne
3aa17bd737 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6e9bb935c3e6ae4da81d70b621781de)
 2018-07-25 17:06:47 +00:00
Eric E Payne
8ee439d791 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3ff90f72392b7a319e4346344db34e03)
 2018-07-16 17:01:38 +00:00
Inigo Goiri
82874e7895 YARN-8412. Move ResourceRequest.clone logic everywhere into a proper API. Contributed by Botong Huang. 2018-06-21 18:25:30 -07:00
Sunil G
96a6798c1d YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b932e0ee69ba61f5796388c175d79195a)
 2018-06-13 16:10:57 +05:30
Rohith Sharma K S
0af3bea05d YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:30:02 +05:30
Robert Kanter
f97bd6bb7f YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 2018-06-04 15:59:27 -07:00
Sunil G
d47a525163 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 2018-06-02 08:30:39 +05:30
Wangda Tan
911852e932 YARN-8068. Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ (Sunil G via wangda) 
Change-Id: I7910bd1064a1b4dbbe2084080c060822ea6f3b48
(cherry picked from commit 9eef19b2ad78b8464da252d0e23c08675898b9d8)
 2018-05-24 13:05:23 -05:00
Wangda Tan
777743beb6 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0f0ad1c877a062b19091e3e15a33ec4)
 2018-05-22 10:34:43 -05:00
Arun Suresh
113e2d6801 YARN-7900. [AMRMProxy] AMRMClientRelayer for stateful FederationInterceptor. (Botong Huang via asuresh) 2018-05-21 11:26:32 -07:00
Sunil G
19cf706711 YARN-8249. Few REST api's in RMWebServices are missing static user check. Contributed by Sunil G. 2018-05-16 12:18:25 +05:30
Robert Kanter
312462532e HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37a76edbe02d556fcfb2e637f45afc21)
(cherry picked from commit 74e5b4b437a0005676d51e770f7dd5110893e356)
 2018-05-07 12:39:57 -07:00
Rohith Sharma K S
250ea47985 YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit ef3ecc308dbea41c6a88bd4d16739c7bbc10cdda)
 2018-04-27 23:00:19 +05:30
Sunil G
ab2b4290a0 YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 
(cherry picked from commit 1634de0fc1430d86b7688d16259a81462fba482f)
 2018-04-27 11:08:27 +05:30
Jason Lowe
06f3f453c2 YARN-7786. NullPointerException while launching ApplicationMaster. Contributed by lujie 2018-04-20 13:31:19 -05:00
Sunil G
733248cec7 YARN-6827. [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7d06806dfdeb3252ac0defe23e8c468eabfa8b5e)
 2018-04-20 12:12:26 +05:30
Inigo Goiri
93fc813329 YARN-8164. Fix a potential NPE in AbstractSchedulerPlanFollower. Contributed by lujie. 
(cherry picked from commit f1461b2487d6c7a0b87e3799a8fdb5ade40ad898)
 2018-04-16 17:32:50 -07:00
Inigo Goiri
f37f680a4e YARN-8165. Incorrect queue name logging in AbstractContainerAllocator. Contributed by Weiwei Yan. 
(cherry picked from commit dd5e18c4aecba56f140c3cc11affc2cb5e61c79d)
 2018-04-16 17:07:38 -07:00
Eric E Payne
7393020cb3 YARN-8147. TestClientRMService#testGetApplications sporadically fails. Contributed by Jason Lowe 
(cherry picked from commit 18844599aef42f79d2af4500aa2eee472dda95cb)
 2018-04-12 18:19:12 +00:00
Eric E Payne
b0dfb18daa YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb893cf0e4aa2d3ecc65440668a1fc8d7)
 2018-04-12 16:38:50 +00:00
Weiwei Yang
a48deb1552 YARN-7527. Over-allocate node resource in async-scheduling mode of CapacityScheduler. Contributed by Tao Yang. 2018-04-12 10:12:46 +08:00
Weiwei Yang
2b2e2ac5f4 YARN-6629. NPE occurred when container allocation proposal is applied but its resource requests are removed before. Contributed by Tao Yang. 2018-04-11 20:21:05 +08:00
Vrushali C
fe4c7ee94c YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 
(cherry picked from commit 345e7624d58a058a1bad666bd1e5ce4b346a9056)
 2018-04-11 09:51:42 +05:30
Sunil G
f7e5e45b72 YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 
(cherry picked from commit bdd2a184d78379d99c802a43ebec7d2cef0bbaf7)
 2018-02-21 15:26:20 +05:30
Jason Lowe
51ad923f51 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 
(cherry picked from commit 85c611ad7dfa87846f708ba0131b4895a054752e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java
 2018-02-19 15:44:56 -06:00
Haibo Chen
c25c082961 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea9d4e38d7f8f9fd48035e342bb22f1c)
 2018-02-05 10:51:03 -08:00
Jinjiang Ling
987a8972ac
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535aeb151c3f283cc41111b15d66990e5)
 2018-01-30 00:13:04 +09:00
Jason Lowe
4d7bbee0a1 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:56:36 -06:00
Sunil G
bed683d300 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968d2897ae006c5ef29eba74023f3483e)
 2018-01-25 14:34:31 +05:30
Rohith Sharma K S
8985fccbbc YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 
(cherry picked from commit d09058b2fd18803d12f0835fdf78aef5e0b99c90)
 2018-01-16 09:01:02 +05:30
Sunil G
e7a8d6bd1e YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:56:08 +05:30
Robert Kanter
ff67c68d3c YARN-7622. Allow fair-scheduler configuration on HDFS (gphillips via rkanter) 2018-01-10 15:03:33 -08:00
bibinchundatt
73478e3bbe YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang. 
(cherry picked from commit b26e30ab1655a8cdf369862a5512db5c2a88ec46)
 2018-01-10 10:31:46 +05:30
Jason Lowe
c892789896 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2018-01-09 10:59:33 -06:00
Jason Lowe
f15c7e86e8 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit d2c2d8c913dfc12585394a1168d4a92934a4cfb4)
 2018-01-09 10:01:06 -06:00
Ray Chiang
18c3982851 YARN-4227. Ignore expired containers from removed nodes in FairScheduler. (Wilfred Spiegelenburg via rchiang) 
(cherry picked from commit bc2d67d6c10619716ef7acce263f3269a86c3150)
 2018-01-08 16:23:29 -08:00
Rohith Sharma K S
37a8224101 YARN-7699. queueUsagePercentage is coming as INF for getApp REST api call. Contributed by Sunil G. 
(cherry picked from commit 281350674c70d65233363849a8b60e9b7dc965cf)
 2018-01-08 15:52:41 +05:30
Sunil G
cac8a8616e YARN-7619. Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user. Contributed by Eric Payne. 
(cherry picked from commit a79abbc03e75a5d2982de12587bd3b45169cc884)
 2018-01-05 14:51:29 +05:30
Rohith Sharma K S
e46142a8c2 YARN-7692. Skip validating priority acls while recovering applications. Contributed by Sunil G. 
(cherry picked from commit c9bf813c9a6c018d14f2bef49ba086ec0e60c761)
 2018-01-03 18:29:38 +05:30
Wangda Tan
c27a5ba65c YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I67e086d6d35c3166ec0f6305490ea6527bd6d799
(cherry picked from commit 41796ea60ab9dbd6a41dd8fd3c6ff50990bba2f5)
 2017-12-08 21:50:41 -08:00
Robert Kanter
54b5da83ec YARN-5594. Handle old RMDelegationToken format when recovering RM (rkanter) 
(cherry picked from commit d8863fc16fa3cbcdda5b99f79386c43e4fae5917)
 2017-12-04 13:18:04 -08:00
Robert Kanter
edf9fd51fb YARN-4813. TestRMWebServicesDelegationTokenAuthentication.testDoAs fails intermittently (grepas via rkanter) 
(cherry picked from commit c83fe4491731c994a4867759d80db31d9c1cab60)
 2017-12-01 12:21:18 -08:00
Wangda Tan
94f7b0f99f YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
 2017-11-23 20:06:16 -08:00
bibinchundatt
5c2986e58d YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 2017-11-20 21:31:50 +05:30
Sunil G
a081d91480 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:42:52 +05:30
Subru Krishnan
0282efab14 YARN-7476. Fix miscellaneous issues in ATSv2 after merge to branch-2. (Varun Saxena via Subru). 2017-11-12 09:19:30 -08:00