Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,501 Commits 326 Branches 371 Tags
Commit Graph

2210 Commits

Author SHA1 Message Date
bibinchundatt
63ed16e076 Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph" 
This reverts commit cf1944eb6e18e00ac18346b92d5f6373de1c1765.
 2019-03-04 17:01:40 +05:30
Weiwei Yang
7575e3090d YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph. 2019-02-28 09:32:07 +08:00
Weiwei Yang
7fa5373ec4 YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie. 
(cherry picked from commit 8c30114b0055bdb44608d4a6d1fa838a04821ff6)
 2019-02-27 17:35:09 +08:00
Weiwei Yang
bdde6a612e YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang. 
(cherry picked from commit 3e1739d5890b739b0045fa5f17139e4de2f9a4ab)
 2019-02-26 00:18:24 +08:00
Weiwei Yang
cdce1c17a0 YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph. 
(cherry picked from commit 9cd5c5447f008b38be43b21ca596d8a36ad0cd55)
 2019-02-24 22:48:55 +08:00
Weiwei Yang
604a915bab YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie. 
(cherry picked from commit 9c88695bcda0ffe4c7f49d643c649dfa1dce9bde)
 2019-02-24 22:21:53 +08:00
bibinchundatt
3e1bd53a37 YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph 2019-02-23 07:59:51 +05:30
bibinchundatt
cf1944eb6e YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph 2019-02-22 20:51:47 +05:30
Weiwei Yang
c2ef443359 YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph. 2019-02-21 18:06:26 +08:00
Weiwei Yang
fbd03543d8 YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph. 
(cherry picked from commit 0a1637c750a5dbe67f1a9b2969ac1b2d4785a11c)
 2019-02-11 15:56:34 +08:00
Weiwei Yang
b64e9df949 YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie. 
(cherry picked from commit 28ad20a711735243bc10b10e33866dc525f415eb)
 2019-02-04 14:00:30 +05:30
Sunil G
99876a5ab8 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 604b2489a9e168a0fd702343f6f8844df9e86d17)
 2019-02-04 12:49:06 +05:30
Eric E Payne
4052b7ee60 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 17:52:54 +00:00
Jonathan Hung
bf760e7e81 YARN-9222. Print launchTime in ApplicationSummary 
(cherry picked from commit 6cace58e212d3ee0aec988926a5a17c9cc58e645)
 2019-01-25 13:23:37 -08:00
Weiwei Yang
bc6374f282 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 2019-01-23 18:10:28 +08:00
Weiwei Yang
8ad7711605 YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph. 
(cherry picked from commit 721d5c2a5f43b29c9cc4439980f21854d9e8ae27)
 2019-01-23 18:07:45 +08:00
Weiwei Yang
9114489566 YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang. 
(cherry picked from commit d43df31751bcadab77d42b31e3e1dd5748b471b5)
 2019-01-22 10:46:37 +08:00
Weiwei Yang
ac2f4b64f9 YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang. 
(cherry picked from commit abde1e1f58d5b699e4b8e460cff68e154738169b)
 2019-01-21 21:20:01 +08:00
Wangda Tan
fe7cb2d84a YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda) 
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
(cherry picked from commit 6d7eedfd28cc1712690db2f6ca8a281b0901ee28)
 2019-01-17 15:13:42 -08:00
Weiwei Yang
2b549e32e1 YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 944cf872232d1a8899df0f0bbe90aa29f225cb90)
 2019-01-07 16:05:57 +08:00
Weiwei Yang
a24cca11f2 YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie. 
(cherry picked from commit cfe89e6f963ba25b5fff1ce48cad36d74b3c789c)
 2019-01-04 01:04:39 +08:00
Weiwei Yang
7deef08eb8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 16:31:03 +08:00
Jonathan Hung
3ab6ea7aca YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics 
(cherry picked from commit 978ab3e958227220cb6f1a08ae6e7cdb8a46628b)
 2018-12-07 10:45:47 -08:00
Jonathan Hung
6b01e4d2a8 YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu 
(cherry picked from commit 1c8bd7128c99d8215ef16438bd2ce6b1f025a966)
 2018-11-30 10:16:00 -08:00
bibinchundatt
183ec39c4b YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:16:32 +05:30
Weiwei Yang
650581a19d YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:23:09 +08:00
Weiwei Yang
2415f8a5be YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt. 
(cherry picked from commit c741109522d2913b87638957c64b94dee6b51029)
 2018-11-12 18:21:11 +08:00
Weiwei Yang
b3321d9933 YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang. 
(cherry picked from commit e1bbf7dcdfc30a61a2b10bef09c59ff17d290488)
 2018-11-08 17:50:16 +08:00
Weiwei Yang
b10ec0aa14 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:44:01 +08:00
Akira Ajisaka
41c4e9583d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f89059d64fda8456366f680eff4a7a6785)
 2018-11-07 11:18:48 +09:00
Jason Lowe
9265934201 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:47:30 -06:00
Weiwei Yang
f00125e2d9 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:52:10 +08:00
Weiwei Yang
b474239e0b YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:17:19 +08:00
Jonathan Hung
b9a3c988c0 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:09 -07:00
Weiwei Yang
5db4272d91 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23c5d1bd52d04c303506806d0dc61cd2)
 2018-10-29 11:55:22 +08:00
Jason Lowe
7097755925 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:54:04 -05:00
Weiwei Yang
042f2df19b YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea1c5f8978455e34f49716ddb0e3a143)
 2018-10-19 10:01:45 +08:00
Wangda Tan
7252f8e117 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07df2a3ed6273b96ae857c278fd72718)
 2018-10-18 12:12:19 -07:00
Sunil G
bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Sunil G
6380ee5512 YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach. 
(cherry picked from commit 5085e5fa9e8c7e489a8518e8541c12b14f3651df)
 2018-10-17 16:06:07 +05:30
Weiwei Yang
eb0147a4c7 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489d83744f3f47a3b884b0c6136885e4a)
 2018-10-10 09:42:45 +08:00
Haibo Chen
d0ee6fbe28 YARN-8621. Add test coverage of custom Resource Types for the apps/<appId> REST API endpoint. (Contributed by Szilard Nemeth) 2018-10-01 14:46:42 -07:00
Weiwei Yang
fd6be5898a YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach. 2018-09-29 17:47:12 +08:00
Eric E Payne
8598b498bc YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 2018-09-28 15:32:07 +00:00
Jason Lowe
6b988d821e YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 2018-09-26 14:43:00 -07:00
Rohith Sharma K S
e5287a4fe0 YARN-8824. App Nodelabel missed after RM restart for finished apps. Contributed by Bibin A Chundatt. 2018-09-26 12:30:26 +05:30
Akira Ajisaka
44edcdfd6a
YARN-8745. Misplaced the TestRMWebServicesFairScheduler.java file. Contributed by Y. SREENIVASULU REDDY. 2018-09-26 10:09:11 +09:00
Rohith Sharma K S
50bc7746d7 YARN-8815. RM fails to recover finished unmanaged AM. Contributed by Bibin A Chundatt. 2018-09-25 11:31:14 +05:30
Haibo Chen
29dad7d258 YARN-8616. systemClock should be used in RMAppImpl instead of System.currentTimeMills(), to be consistent. (Contributed by Szilard Nemeth) 2018-09-24 16:04:28 -07:00
Giovanni Matteo Fumarola
3090922805 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang. 2018-09-24 11:37:05 -07:00