Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,072 Commits 326 Branches 371 Tags
Commit Graph

2148 Commits

Author SHA1 Message Date
Rohith Sharma K S
923c833da4 YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit c05b5d424b000bab766f57e88a07f2b4e9a56647)
 2018-05-24 14:22:43 +05:30
Wangda Tan
6aaf33e50c YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
(cherry picked from commit bd15d2396ef0c24fb6b60c6393d16b37651b828e)
 2018-05-22 13:39:15 -07:00
Eric E Payne
5b2bbff141 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdeef62945b592d5761666ad026b629c0b)
 2018-05-21 20:19:44 +00:00
Weiwei Yang
b4213bfd90 YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 
(Cherry picked from commit 9db9cd95bd0348070a286e69e7965c03c9bd39d6)
 2018-05-11 10:55:05 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4c0a6b67f3 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 
(cherry picked from commit d76fbbc9b82e720d7d5188f9ae2f56a8d78f3a98)
 2018-05-10 19:07:23 -07:00
Robert Kanter
0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c43644b4ad22d5385c22cad8ed573c0b1ba)
 2018-05-10 09:52:11 -07:00
bibinchundatt
b29479c93a YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a5266c8b8351f338f5fc8087a203c68cac)
 2018-05-10 09:29:43 +05:30
Rohith Sharma K S
e3a7dbc4ec YARN-8239. [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI. Contributed by Sunil G. 
(cherry picked from commit eb7b256cc2c74e1b461e45444179584668d34ac6)
 2018-05-09 12:33:49 +05:30
Wangda Tan
0873cfcb56 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0f0ad1c877a062b19091e3e15a33ec4)
 2018-05-08 11:35:11 -07:00
Weiwei Yang
68dd01efc1 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f676237290d18ddbbc9aec369267692)
 2018-05-07 11:41:01 +08:00
Rohith Sharma K S
4c93cd228c YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 
(cherry picked from commit ee2ce923a922bfc3e89ad6f0f6a25e776fe91ffb)
 2018-05-03 18:20:21 +05:30
Weiwei Yang
6fce88765a YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 2018-05-02 18:02:24 +08:00
Weiwei Yang
f34f4e9c11 YARN-8212. Pending backlog for async allocation threads should be configurable. Contributed by Tao Yang. 
(cherry picked from commit 2d319e37937c1e20c6a7dc4477ef88defd1f8464)
 2018-05-01 09:57:15 +08:00
Konstantinos Karanasos
9d2967098d YARN-8195. Fix constraint cardinality check in the presence of multiple target allocation tags. Contributed by Weiwei Yang. 
(cherry picked from commit 9b0955545174abe16fd81240db30f175145ee89b)
 2018-04-30 11:55:26 -07:00
Wangda Tan
090a8a274e YARN-8005. Add unit tests for queue priority with dominant resource calculator. (Zian Chen via wangda) 
Change-Id: I17a645f20869a1e5d86fa7a325c93fec908b91dc
(cherry picked from commit 92c5331423afee172e843198f4824469eb92e89b)
 2018-04-27 13:13:52 -07:00
Rohith Sharma K S
b3fd464e55 YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit ef3ecc308dbea41c6a88bd4d16739c7bbc10cdda)
 2018-04-27 22:59:16 +05:30
Sunil G
28e63f3d88 YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 
(cherry picked from commit 71220d218db59cab0b03bbba427e5e9ef5b3003c)
 2018-04-27 10:42:59 +05:30
Sunil G
e2901d97ba YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 
(cherry picked from commit 1634de0fc1430d86b7688d16259a81462fba482f)
 2018-04-27 10:28:18 +05:30
Wangda Tan
af286319fd YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 
Change-Id: Ia83dd2499ee9000b9e09ae5a932f21a13c0ddee6
(cherry picked from commit af986b442b7be0f2fad1241ca9e267a15c20bf43)
 2018-04-25 22:16:07 -07:00
Sunil G
44d28692a8 YARN-8188. RM Nodes UI data table index for sorting column need to be corrected post Application tags display. Contributed by Weiwei Yang. 
(cherry picked from commit 626690612cd0957316628376744a8be62f891665)
 2018-04-25 14:58:07 +05:30
Wangda Tan
3043a93d46 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
(cherry picked from commit bb3c504764f807fccba7f28298a12e2296f284cb)
 2018-04-24 17:44:58 -07:00
Robert Kanter
ea7ad50499 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37a76edbe02d556fcfb2e637f45afc21)
 2018-04-23 16:25:41 -07:00
Konstantinos Karanasos
18c86a3fb6 YARN-8111. Simplify PlacementConstraints API by removing allocationTagToIntraApp. Contributed by Weiwei Yang. 
(cherry picked from commit 28e2244390c990877dc2ee2733cf9b8d2c75128e)
 2018-04-20 12:25:08 -07:00
Jason Lowe
71b0d52987 YARN-7786. NullPointerException while launching ApplicationMaster. Contributed by lujie 
(cherry picked from commit 766544c0b008da9e78bcea6285b2c478653df75a)
 2018-04-20 13:23:46 -05:00
Sunil G
162db6e826 YARN-6827. [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7d06806dfdeb3252ac0defe23e8c468eabfa8b5e)
 2018-04-20 00:06:44 +05:30
Shashikant Banerjee
a13b3edfd8
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247da77d03c6015de4f18be75924fcb22)
 2018-04-19 08:36:32 +09:00
Rohith Sharma K S
fa007d986b YARN-8145. yarn rmadmin -getGroups doesn't return updated groups for user. Contributed by Sunil G. 
(cherry picked from commit 241d985467c4a318fa22a3f3007ee1827c94389c)
 2018-04-18 13:40:32 +05:30
Inigo Goiri
8f300c9ce1 YARN-8165. Incorrect queue name logging in AbstractContainerAllocator. Contributed by Weiwei Yan. 
(cherry picked from commit dd5e18c4aecba56f140c3cc11affc2cb5e61c79d)
 2018-04-17 15:28:13 +05:30
Wangda Tan
6ee62e6b1c YARN-8138. Add unit test to validate queue priority preemption works under node partition. (Zian Chen via wangda) 
Change-Id: Ibebfab98a714c12c2dc643b6d7b9754a7f813632
 2018-04-14 11:04:49 -07:00
Wangda Tan
994c7d66e0 YARN-8091. Revisit checkUserAccessToQueue RM REST API. (wangda) 
Change-Id: I5fab3fe229c34e967487b7327c7b3c8ddf7cb795
 2018-04-13 10:19:10 -07:00
Wangda Tan
7031a853f4 YARN-8048. Support auto-spawning of admin configured services during bootstrap of RM (Rohith Sharma K S via wangda) 
Change-Id: I2d8d61ccad55e1118009294d7e17822df3cd0fd5
 2018-04-13 10:18:48 -07:00
Sunil G
21717db6a0 YARN-8028. Support authorizeUserAccessToQueue in RMWebServices. Contributed by Wangda Tan. 2018-04-13 10:18:12 -07:00
Eric E Payne
9693d8b99d YARN-8147. TestClientRMService#testGetApplications sporadically fails. Contributed by Jason Lowe 
(cherry picked from commit 18844599aef42f79d2af4500aa2eee472dda95cb)
 2018-04-12 17:58:41 +00:00
Eric E Payne
f955d52f39 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb893cf0e4aa2d3ecc65440668a1fc8d7)
 2018-04-12 16:11:57 +00:00
Weiwei Yang
c2036af196 YARN-8127. Resource leak when async scheduling is enabled. Contributed by Tao Yang. 
(cherry picked from commit 7eb783e2634d8c11fb646f1f2fdf597336325312)
 2018-04-11 17:21:47 +08:00
Vrushali C
67cd7acf99 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 
(cherry picked from commit 345e7624d58a058a1bad666bd1e5ce4b346a9056)
 2018-04-11 09:41:04 +05:30
Wangda Tan
25bf1d3ac6 Updated branch-3.1 to track 3.1.1 
Change-Id: Idd55c79921b8acacc8a8a3258450e476f3abb706
 2018-04-09 10:58:50 -07:00
Sunil G
8311fcc75b YARN-7574. Add support for Node Labels on Auto Created Leaf Queue Template. Contributed by Suma Shivaprasad. 
(cherry picked from commit 821b0de4c59156d4a65112de03ba3e7e1c88e309)
 2018-04-09 21:17:56 +05:30
Konstantinos Karanasos
fa464c12ad YARN-8013. Support application tags when defining application namespaces for placement constraints. Contributed by Weiwei Yang. 
(cherry picked from commit 7853ec8d2fb8731b7f7c28fd87491a0a2d47967e)
 2018-04-04 10:53:33 -07:00
Wangda Tan
78832eca8a YARN-8002. Support NOT_SELF and ALL namespace types for allocation tag. (Weiwei Yang via wangda) 
Change-Id: I63b4e4192a95bf7ded98c54e46a2871c72869700
 2018-04-04 09:42:49 +08:00
Sunil G
73cd67d7e3 YARN-8085. ResourceProfilesManager should be set in RMActiveServiceContext. Contributed by Tao Yang. 
(cherry picked from commit 7a59d60e0c50e8264e07f6b1dadaeedba676c8c2)
 2018-03-29 21:41:52 +05:30
Wangda Tan
a2b4daab55 YARN-6629. NPE occurred when container allocation proposal is applied but its resource requests are removed before. (Tao Yang via wangda) 
Change-Id: I805880f90b3f6798ec96ed8e8e75755f390a9ad5
(cherry picked from commit 47f711eebca315804c80012eea5f31275ac25518)
 2018-03-28 11:22:26 -07:00
Wangda Tan
dc99611878 YARN-8072. RM log is getting flooded with MemoryPlacementConstraintManager info logs. (Zian Chen via wangda) 
Change-Id: I2567f4669ad868e13c4301fe44c4bde85bb92c37
(cherry picked from commit 24bc5e0e4d919fddae983fc13d4ba4e4e5cb9533)
 2018-03-26 11:27:40 -07:00
Wangda Tan
62b9537332 YARN-8068. Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ (Sunil G via wangda) 
Change-Id: I7910bd1064a1b4dbbe2084080c060822ea6f3b48
(cherry picked from commit 9eef19b2ad78b8464da252d0e23c08675898b9d8)
 2018-03-26 11:27:32 -07:00
Wangda Tan
54da6a0d78 YARN-8062. yarn rmadmin -getGroups returns group from which the user has been removed. (Sunil G via wangda) 
Change-Id: I80ed63846502bf7751b890b6c6c6a7c0679e2b4a
(cherry picked from commit 5d381570f83022b411a8740d58486a7f68ab2af6)
 2018-03-26 11:27:21 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
0bd99621c5 Preparing for 3.1.0 development 2018-03-19 15:16:54 -07:00
Weiwei Yang
154cfb2b62 YARN-7636. Re-reservation count may overflow when cluster resource exhausted for a long time. contributed by Tao Yang. 2018-03-16 18:57:31 +08:00
Wangda Tan
4bf622043f YARN-7952. RM should be able to recover log aggregation status after restart/fail-over. (Xuan Gong via wangda) 
Change-Id: I725c9afe64831eda0aa6b0bebdbc79d2dd165707
 2018-03-15 13:26:45 -07:00
Wangda Tan
b167d60763 YARN-7657. Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue. (Suma Shivaprasad via wangda) 
Change-Id: I32d566e8727840e43c0d66e39a77edef017e3a83
 2018-03-13 17:55:51 -07:00
Yufei Gu
ff31d8aefa YARN-8024. LOG in class MaxRunningAppsEnforcer is initialized with a faulty class. Contributed by Sen Zhao. 2018-03-12 16:35:26 -07:00