Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,040 Commits 326 Branches 371 Tags
Commit Graph

4063 Commits

Author SHA1 Message Date
Varun Saxena
c9a46308a1 YARN-9825. Changes for initializing placement rules with ResourceScheduler in branch-2. Contributed by Jonathan Hung. 2019-09-13 17:03:47 +05:30
Jonathan Hung
c412fab728 YARN-9824. Fall back to configured queue ordering policy class name 
(cherry picked from commit f8f8598ea5e86330fa5b553963f5632da286409d)
(cherry picked from commit 1dbf87c9ff33b5e2f6cbb0ce8efd348c4314a0e2)
(cherry picked from commit d75693bd6ed4a9547f4932d41014a21de1398115)
 2019-09-10 15:36:45 -07:00
Jonathan Hung
1476cf5fbd YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph 2019-09-09 00:30:20 -07:00
Jonathan Hung
d2eabbd7fd YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar 2019-09-08 19:20:51 -07:00
Jonathan Hung
6e7dfc9981 YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar 2019-09-06 10:59:14 -07:00
Jonathan Hung
fb78b7c45c YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:03:33 -07:00
Jonathan Hung
6e02d8fcfb YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta 
(cherry picked from commit 0ccf4b0fe16a8c879a560f2a612a3185eb2df72b)
(cherry picked from commit cb806988d72bde1f9837c9e0fb82a3a6c032542c)
(cherry picked from commit 37d1f8c81eeee79b1ef5ce6316afbcdbc76acd8e)
 2019-09-05 14:09:08 -07:00
Akira Ajisaka
758085a87e YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena. 
(cherry picked from commit 5db7c4906235dd1070b9706ca4971f1eb5d74139)
(cherry picked from commit a453f38015fec454f63e9f5459b0330ac3bc7eb3)
(cherry picked from commit 3c9d2f53175e11ae12fe5857289980ee612124a8)
 2019-08-30 17:57:39 -07:00
Miklos Szegedi
655154cb45 YARN-7585. NodeManager should go unhealthy when state store throws DBException. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 7f515f57ede74dae787994f37bfafd5d20c9aa4c)
 2019-08-29 19:52:36 -07:00
Wangda Tan
056b54c62c YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2019-08-29 17:02:47 -07:00
Jonathan Hung
87ad52f076 YARN-9409. Port resource type changes from YARN-7237 to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
28fbc4e344 YARN-9272. Backport YARN-7738 for refreshing max allocation for multiple resource types 2019-08-29 17:02:47 -07:00
Jonathan Hung
8e5ce8055e YARN-9271. Backport YARN-6927 for resource type support in MapReduce 2019-08-29 17:02:47 -07:00
Wangda Tan
6f4ebe01b7 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
(cherry picked from commit bb3c504764f807fccba7f28298a12e2296f284cb)
(cherry picked from commit 3043a93d461fd8b9ccc2ff4b8d17e5430ed77615)
 2019-08-29 17:02:47 -07:00
Sunil G
1875231be5 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 2019-08-29 17:02:47 -07:00
Daniel Templeton
062fbd084d YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2019-08-29 17:02:47 -07:00
Wangda Tan
7025333d31 YARN-7345. GPU Isolation: Incorrect minor device numbers written to devices.deny file. (Jonathan Hung via wangda) 2019-08-29 17:02:47 -07:00
Jonathan Hung
22ba6939eb YARN-9291. Backport YARN-7637 to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
e102c3d74e YARN-9397. Fix empty NMResourceInfo object test failures in branch-2 2019-08-29 17:02:47 -07:00
Sunil G
86709564cb YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan. 2019-08-29 17:02:47 -07:00
Sunil G
c539427696 YARN-7594. TestNMWebServices#testGetNMResourceInfo fails on trunk. Contributed by Gergely Novák. 2019-08-29 17:02:47 -07:00
Wangda Tan
ca7d0fda4f YARN-7573. Gpu Information page could be empty for nodes without GPU. (Sunil G via wangda) 
Change-Id: I7f614e5a589a09ce4e4286c84b706e05c29abd14
 2019-08-29 17:02:47 -07:00
Jonathan Hung
7939ae1695 YARN-9289. Backport YARN-7330 for GPU in UI to branch-2 2019-08-29 17:02:47 -07:00
Jian He
4d82318eb1 YARN-7396. NPE when accessing container logs due to null dirsHandler. Contributed by Jonathan Hung 2019-08-29 17:02:47 -07:00
Jonathan Hung
16faceb0da YARN-9174. Backport YARN-7224 for refactoring of GpuDevice class 2019-08-29 17:02:47 -07:00
Jonathan Hung
7ec4d7c6ce YARN-9280. Backport YARN-6620 to YARN-8200/branch-2 for NodeManager-side GPU isolation 2019-08-29 17:02:47 -07:00
Jonathan Hung
631dfc7277 YARN-9180. Port YARN-7033 NM recovery of assigned resources to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
56259bcecb YARN-9187. Backport YARN-6852 for GPU-specific native changes to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
078dfb09fe YARN-9175. Null resources check in ResourceInfo for branch-3.0 
(cherry picked from commit a0291a015c1af0ea1282849bd8fb32824d7452fa)
 2019-08-29 17:02:47 -07:00
Sunil G
4d30c90a1b YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan. 
(cherry picked from commit da0b6a354bf6f6bf37ca5a05a4a8eece09aa4893)
(cherry picked from commit 74030d808cd95e26a0c48500c08d269fcb4150ee)
 2019-08-29 17:02:47 -07:00
Daniel Templeton
72d90e750b YARN-7270 addendum: Reapplied changes after YARN-3926 backports 2019-08-29 17:02:47 -07:00
Jonathan Hung
d6673c5491 YARN-9188. Port YARN-7136 to branch-2 2019-08-29 17:02:47 -07:00
Wangda Tan
26e87be3f2 YARN-7056. [Partial backport] Document Resource Profiles feature. (Sunil G via wangda) 
Change-Id: I5f5d52cdf7aba4ac09684aab573e6916dbad183e
(cherry picked from commit 6b88cd1901a65b977fed759e322dcd75a2cd33b6)
 2019-08-29 17:02:47 -07:00
Sunil G
45b50c45d6 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R. 
(cherry picked from commit 805095496dfd8e50f71b70cf20845e954d3ba47c)
 2019-08-29 17:02:47 -07:00
Wangda Tan
8cea242e94 YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda) 
Change-Id: I88928a747ee3eec17bc76fb71e9aaa632d091f0f
(cherry picked from commit 53df3eac503064b8c4cd8eb2b659567622f4d8a7)
 2019-08-29 17:02:46 -07:00
Wangda Tan
f340726c9a YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
(cherry picked from commit e490602e9b306d5b8a543b93fb15a7395bb9a03d)
 2019-08-29 17:02:46 -07:00
Sunil G
74bb2d6ae7 YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan. 
(cherry picked from commit a333ba54e3f5a1ca57b0fc94bfd008f3cbd5b14b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
 2019-08-29 17:02:46 -07:00
Wangda Tan
1f05fa041a YARN-7043. [Partial backport] Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
(cherry picked from commit 39240b61a163b127eec385decd30ffd96e694c28)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/OpportunisticContainerAllocatorAMService.java
 2019-08-29 17:02:46 -07:00
Sunil G
7c37fbef12 YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin. 
(cherry picked from commit 758b7719430c981a6d4d1644d9d03cd16b981cf3)
 2019-08-29 17:02:46 -07:00
Wangda Tan
c69dd9eea6 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541af98a3805958f2b5b9db6be705d52ab)
 2019-08-29 17:02:46 -07:00
Wangda Tan
43030aa423 YARN-7042. Clean up unit tests after YARN-6610. (Daniel Templeton via wangda) 
Change-Id: I8e40f704b6fcdd5b14faa9548a27986501044fa1
(cherry picked from commit b1fe3a222e7673fd84a878622969f958022061e9)
 2019-08-29 17:02:46 -07:00
Sunil G
7b1c58b037 YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan. 
(cherry picked from commit a6989af1d91d8b6795862c39d154823f8488c8de)
 2019-08-29 17:02:46 -07:00
Sunil G
2fafdc7f8b YARN-6610. [YARN-3926] DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources. Contributed by Daniel Templeton. 
(cherry picked from commit a0030c8c6f5489d8285632a651394b2b2320255c)
 2019-08-29 17:02:46 -07:00
Sunil G
06c38ee7a4 YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G. 
(cherry picked from commit 2b51b262aba0191b80dc93799574c0b959cb4f4e)
 2019-08-29 17:02:46 -07:00
Yufei Gu
59cfecd60e YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit d5e9939ebb59e4f6f8e5e2fe7f619fb032f2911c)
 2019-08-29 17:02:46 -07:00
Daniel Templeton
d8f388652e YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)

(cherry picked from commit 3aeaafecb823ef6c175ea5d0d9cb726faacaf32d)
 2019-08-29 17:02:46 -07:00
Sunil G
1dc66dbb3f YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup. Contributed by Yeliang Cang. 
(cherry picked from commit 4e5632d28eb8f6b1b8830c2aa89ec6321d477977)
 2019-08-29 17:02:46 -07:00
Sunil G
ad6f095774 YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. 
(cherry picked from commit ca2808d8bbe49766029e5dba88ff863f6c993287)
 2019-08-29 17:02:46 -07:00
Jonathan Hung
d5963b2be9 YARN-9182. Backport YARN-6445 resource profile performance improvements to branch-2 2019-08-29 17:02:46 -07:00
Jonathan Hung
08923e324a YARN-9177. Use resource map for app metrics in TestCombinedSystemMetricsPublisher for branch-2 2019-08-29 17:02:46 -07:00