Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,006 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1972 Commits

Author SHA1 Message Date
Weiwei Yang 373e32480b YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23)
(cherry picked from commit 70efe253f3)
 2019-08-05 10:02:33 -07:00
Wangda Tan d18ec19ed8 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07)
(cherry picked from commit 46baafedf1)
 2019-08-05 10:02:33 -07:00
Eric E Payne c04ccf296d YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 2019-07-30 20:00:32 +00:00
Erik Krogen 849e1f018e HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-25 10:07:04 -07:00
Jonathan Hung 3b346f50de YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:54:38 -07:00
Ahmed Hussein 9d63fc7ed6 YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2019-05-29 14:19:20 -05:00
Weiwei Yang 5d85a34b1f YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu. 
(cherry picked from commit 8c1bba375b)
 2019-04-10 23:05:12 +08:00
Weiwei Yang 0e167fde55 YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang. 2019-04-08 13:44:33 +08:00
Sunil G ec0bed1008 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 6ffe6ea899)
 2019-02-07 19:09:19 +05:30
Eric E Payne 50ef25464f YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb)
 2019-01-29 18:12:36 +00:00
Jonathan Hung bce8f28bec YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e21)
(cherry picked from commit bf760e7e81)
(cherry picked from commit 80a08c29adc9d07974bbe8087abcec750d5f7916)
 2019-01-25 13:51:21 -08:00
Haibo Chen 3368f141b2 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef)
(cherry picked from commit 3a764250262e745a6762dd6551b0d0a6af39e891)
 2019-01-24 15:58:27 -08:00
Weiwei Yang 0e3d5f3064 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751)
 2019-01-22 11:15:03 +08:00
Weiwei Yang 009f5b55ad YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed Wilfred Spiegelenburg. 
(cherry picked from commit 91e9c9f96e)
 2019-01-08 14:09:49 +08:00
Jonathan Hung a0291a015c YARN-9175. Null resources check in ResourceInfo for branch-3.0 2019-01-03 15:58:10 -05:00
Weiwei Yang c4e39e3a59 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f96)
 2019-01-04 02:00:59 +08:00
Jonathan Hung 80ef3e7581 YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
(cherry picked from commit dca69d178dba21c41fd1293187f29143f7e81e19)
 2018-12-07 10:46:07 -08:00
Jonathan Hung 15057a7e67 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 
(cherry picked from commit c3d10aa6eddadd0e00edfb87cc77c9483ec8fb13)
 2018-11-30 10:17:26 -08:00
bibinchundatt b42c2679ad YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8)
 2018-11-29 22:08:48 +05:30
Weiwei Yang 067e39f781 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f03)
 2018-11-18 23:25:31 +08:00
Akira Ajisaka 50ce9fd4e2
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-10 18:02:57 +09:00
Weiwei Yang be2a0a5e23 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659)
 2018-11-07 22:54:34 +08:00
Jason Lowe f3e6682ba7 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c726)
 2018-11-06 08:59:47 -06:00
Weiwei Yang ddb349ceb3 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722)
 2018-11-06 14:54:33 +08:00
Weiwei Yang c3a0f07db0 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7e)
 2018-11-06 13:43:24 +08:00
Jonathan Hung 68ec0fa0a8 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b)
 2018-11-01 14:31:57 -07:00
Jason Lowe a9bc3e3708 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9c)
 2018-10-23 12:57:49 -05:00
Weiwei Yang 37af9580c8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea)
 2018-10-19 10:03:39 +08:00
Wangda Tan b67dc5e1d1 YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) 
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
(cherry picked from commit 74665e3a7d)
 2018-10-11 16:00:10 +00:00
Weiwei Yang af85ce6ae4 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489)
 2018-10-10 09:56:16 +08:00
Eric E Payne b412bb2240 YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 
(cherry picked from commit 8598b498bc)
 2018-09-28 15:48:51 +00:00
Jason Lowe 17583e690a YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e)
 2018-09-26 16:54:29 -07:00
Weiwei Yang 4edcde055b Revert "YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang." 
This reverts commit f56e36935d.
 2018-09-19 19:47:06 +08:00
Weiwei Yang f56e36935d YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang. 
(cherry picked from commit 0712537e79)
 2018-09-19 19:41:07 +08:00
Weiwei Yang ef17728c58 YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi. 
(cherry picked from commit f1a893fdbc)
 2018-09-14 16:43:28 +08:00
Eric E Payne 48dc8de28b YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang. 
(cherry picked from commit 987d8191ad)
 2018-09-10 20:18:14 +00:00
Haibo Chen 49590e7c6b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed5)
 2018-08-24 13:27:54 -05:00
Rohith Sharma K S 675aa2bbc0 YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff60)
 2018-08-18 11:06:14 +05:30
Jonathan Hung 0420ca5a6f YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-10 15:22:11 -07:00
Sunil G a3675f382a YARN-8397. Potential thread leak in ActivitiesManager. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6310c0d17d)
 2018-08-01 08:36:12 +05:30
Jonathan Hung 0d6e1a2aab YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 18:06:25 -07:00
Eric E Payne 299dffc72d YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 
(cherry picked from commit 9485c9aee6)
 2018-07-25 16:49:03 +00:00
Sunil G 1d8fce0d2f YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 
(cherry picked from commit 35ce6eb1f5)
 2018-07-24 22:21:44 +05:30
bibinchundatt 1f713d6c66 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 
(cherry picked from commit ff7c2eda34)
 2018-07-24 16:32:21 +05:30
Eric E Payne 5738bd8a10 YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 
(cherry picked from commit 937ef39b3f)
 2018-07-16 16:53:07 +00:00
Junping Du 0a6942d58c yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 
(cherry picked from commit d9ba6f3656)
(cherry picked from commit 3d6ba2dd4e)
 2018-07-09 16:22:58 +08:00
Weiwei Yang 3151e95d27 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 2018-06-25 09:46:32 +08:00
Rohith Sharma K S 6542e31b78 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 
(cherry picked from commit 9119b3cf8f)
 2018-06-14 13:51:05 +05:30
Sunil G d9b00cdd6b YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b93)
 2018-06-13 16:10:24 +05:30
Weiwei Yang ef105abb70 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 
(Cherry picked from commit 29024a6203)
 2018-06-13 13:57:35 +08:00