Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,515 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2067 Commits

Author SHA1 Message Date
Sunil G 844538d669 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan. 
(cherry picked from commit 51654a3962)
 2018-07-03 10:00:05 -07:00
Junping Du 3d6ba2dd4e yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 
(cherry picked from commit d9ba6f3656)
 2018-07-03 22:36:24 +08:00
Sunil G 122cb81b65 YARN-8379. Improve balancing resources in already satisfied queues by using Capacity Scheduler preemption. Contributed by Zian Chen. 
(cherry picked from commit 291194302c)
 2018-06-28 10:24:03 -07:00
Eric Yang 3ce3571a34 YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. 
Contributed by Chandni Singh
 2018-06-28 13:11:56 -04:00
Wangda Tan ec37e02bc9 YARN-8464. Async scheduling thread could be interrupted when there are no NodeManagers in cluster. (Sunil G via wangda) 
Change-Id: I4f5f856373378685713e77752ba6cf0988a66065
(cherry picked from commit bedc4fe079)
 2018-06-26 19:34:15 -07:00
Eric Yang f6aa6a6981 YARN-8108. Added option to disable loading existing filters to prevent 
security filter from initialize twice.
            Contributed by Sunil Govindan

(cherry picked from commit b69ba0f330)
 2018-06-26 17:45:13 -04:00
Weiwei Yang bfdbc9dea2 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 
(Cherry picked from commit 440140cea6)
 2018-06-25 09:28:57 +08:00
Rohith Sharma K S 1aa6c9407c YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 
(cherry picked from commit 9119b3cf8f)
 2018-06-14 13:51:13 +05:30
Sunil G ed20c59d36 YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b93)
 2018-06-13 16:09:52 +05:30
Weiwei Yang 4488ad5297 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 2018-06-13 13:54:09 +08:00
Rohith Sharma K S af0309d793 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace0)
 2018-06-12 17:23:22 +05:30
Robert Kanter 93d6ed859e YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a443)
 2018-06-04 15:32:47 -07:00
Wangda Tan 0be93a7973 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
(cherry picked from commit ff583d3fa3)
 2018-06-01 15:53:26 -07:00
Wangda Tan 1cee85d8c4 YARN-7962. Race Condition When Stopping DelegationTokenRenewer causes RM crash during failover. (BELUGA BEHR via wangda) 
Change-Id: I617e2645f60a57080058ad5f06af860fb3f682c8
(cherry picked from commit 931f78718f)
 2018-06-01 15:47:51 -07:00
Weiwei Yang ec4240a7fa YARN-8367. Fix NPE in SingleConstraintAppPlacementAllocator when placement constraint in SchedulingRequest is null. Contributed by Weiwei Yang. 
(Cherry picked from commit 6468071f13)
 2018-05-31 21:04:41 +08:00
Wangda Tan 3eb1cb18c7 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669)
 2018-05-29 09:28:34 -07:00
Sunil G 7f9caf8986 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669ba)
 2018-05-28 16:33:48 +05:30
Wangda Tan 4d41cb1696 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
(cherry picked from commit f24c842d52)
 2018-05-25 22:04:54 -07:00
Eric E Payne 053937f436 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 
(cherry picked from commit 8d5509c681)
 2018-05-25 16:27:09 +00:00
Rohith Sharma K S 923c833da4 YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit c05b5d424b)
 2018-05-24 14:22:43 +05:30
Wangda Tan 6aaf33e50c YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
(cherry picked from commit bd15d2396e)
 2018-05-22 13:39:15 -07:00
Eric E Payne 5b2bbff141 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdee)
 2018-05-21 20:19:44 +00:00
Weiwei Yang b4213bfd90 YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 
(Cherry picked from commit 9db9cd95bd)
 2018-05-11 10:55:05 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 4c0a6b67f3 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 
(cherry picked from commit d76fbbc9b8)
 2018-05-10 19:07:23 -07:00
Robert Kanter 0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c4364)
 2018-05-10 09:52:11 -07:00
bibinchundatt b29479c93a YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a526)
 2018-05-10 09:29:43 +05:30
Rohith Sharma K S e3a7dbc4ec YARN-8239. [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI. Contributed by Sunil G. 
(cherry picked from commit eb7b256cc2)
 2018-05-09 12:33:49 +05:30
Wangda Tan 0873cfcb56 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0)
 2018-05-08 11:35:11 -07:00
Weiwei Yang 68dd01efc1 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f)
 2018-05-07 11:41:01 +08:00
Rohith Sharma K S 4c93cd228c YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 
(cherry picked from commit ee2ce923a9)
 2018-05-03 18:20:21 +05:30
Weiwei Yang 6fce88765a YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 2018-05-02 18:02:24 +08:00
Weiwei Yang f34f4e9c11 YARN-8212. Pending backlog for async allocation threads should be configurable. Contributed by Tao Yang. 
(cherry picked from commit 2d319e3793)
 2018-05-01 09:57:15 +08:00
Konstantinos Karanasos 9d2967098d YARN-8195. Fix constraint cardinality check in the presence of multiple target allocation tags. Contributed by Weiwei Yang. 
(cherry picked from commit 9b09555451)
 2018-04-30 11:55:26 -07:00
Wangda Tan 090a8a274e YARN-8005. Add unit tests for queue priority with dominant resource calculator. (Zian Chen via wangda) 
Change-Id: I17a645f20869a1e5d86fa7a325c93fec908b91dc
(cherry picked from commit 92c5331423)
 2018-04-27 13:13:52 -07:00
Rohith Sharma K S b3fd464e55 YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit ef3ecc308d)
 2018-04-27 22:59:16 +05:30
Sunil G 28e63f3d88 YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 
(cherry picked from commit 71220d218d)
 2018-04-27 10:42:59 +05:30
Sunil G e2901d97ba YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 
(cherry picked from commit 1634de0fc1)
 2018-04-27 10:28:18 +05:30
Wangda Tan af286319fd YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 
Change-Id: Ia83dd2499ee9000b9e09ae5a932f21a13c0ddee6
(cherry picked from commit af986b442b7be0f2fad1241ca9e267a15c20bf43)
 2018-04-25 22:16:07 -07:00
Sunil G 44d28692a8 YARN-8188. RM Nodes UI data table index for sorting column need to be corrected post Application tags display. Contributed by Weiwei Yang. 
(cherry picked from commit 626690612cd0957316628376744a8be62f891665)
 2018-04-25 14:58:07 +05:30
Wangda Tan 3043a93d46 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
(cherry picked from commit bb3c504764f807fccba7f28298a12e2296f284cb)
 2018-04-24 17:44:58 -07:00
Robert Kanter ea7ad50499 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37)
 2018-04-23 16:25:41 -07:00
Konstantinos Karanasos 18c86a3fb6 YARN-8111. Simplify PlacementConstraints API by removing allocationTagToIntraApp. Contributed by Weiwei Yang. 
(cherry picked from commit 28e2244390)
 2018-04-20 12:25:08 -07:00
Jason Lowe 71b0d52987 YARN-7786. NullPointerException while launching ApplicationMaster. Contributed by lujie 
(cherry picked from commit 766544c0b0)
 2018-04-20 13:23:46 -05:00
Sunil G 162db6e826 YARN-6827. [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7d06806dfd)
 2018-04-20 00:06:44 +05:30
Shashikant Banerjee a13b3edfd8
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247)
 2018-04-19 08:36:32 +09:00
Rohith Sharma K S fa007d986b YARN-8145. yarn rmadmin -getGroups doesn't return updated groups for user. Contributed by Sunil G. 
(cherry picked from commit 241d985467)
 2018-04-18 13:40:32 +05:30
Inigo Goiri 8f300c9ce1 YARN-8165. Incorrect queue name logging in AbstractContainerAllocator. Contributed by Weiwei Yan. 
(cherry picked from commit dd5e18c4ae)
 2018-04-17 15:28:13 +05:30
Wangda Tan 6ee62e6b1c YARN-8138. Add unit test to validate queue priority preemption works under node partition. (Zian Chen via wangda) 
Change-Id: Ibebfab98a714c12c2dc643b6d7b9754a7f813632
 2018-04-14 11:04:49 -07:00
Wangda Tan 994c7d66e0 YARN-8091. Revisit checkUserAccessToQueue RM REST API. (wangda) 
Change-Id: I5fab3fe229c34e967487b7327c7b3c8ddf7cb795
 2018-04-13 10:19:10 -07:00
Wangda Tan 7031a853f4 YARN-8048. Support auto-spawning of admin configured services during bootstrap of RM (Rohith Sharma K S via wangda) 
Change-Id: I2d8d61ccad55e1118009294d7e17822df3cd0fd5
 2018-04-13 10:18:48 -07:00