Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,260 Commits 326 Branches 371 Tags
Commit Graph

15260 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wangda Tan
056b54c62c YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2019-08-29 17:02:47 -07:00
Jonathan Hung
87ad52f076 YARN-9409. Port resource type changes from YARN-7237 to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
28fbc4e344 YARN-9272. Backport YARN-7738 for refreshing max allocation for multiple resource types 2019-08-29 17:02:47 -07:00
Jonathan Hung
8e5ce8055e YARN-9271. Backport YARN-6927 for resource type support in MapReduce 2019-08-29 17:02:47 -07:00
Wangda Tan
6f4ebe01b7 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
(cherry picked from commit bb3c504764f807fccba7f28298a12e2296f284cb)
(cherry picked from commit 3043a93d461fd8b9ccc2ff4b8d17e5430ed77615)
 2019-08-29 17:02:47 -07:00
Sunil G
1875231be5 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 2019-08-29 17:02:47 -07:00
Daniel Templeton
062fbd084d YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2019-08-29 17:02:47 -07:00
Wangda Tan
7025333d31 YARN-7345. GPU Isolation: Incorrect minor device numbers written to devices.deny file. (Jonathan Hung via wangda) 2019-08-29 17:02:47 -07:00
Jonathan Hung
22ba6939eb YARN-9291. Backport YARN-7637 to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
e102c3d74e YARN-9397. Fix empty NMResourceInfo object test failures in branch-2 2019-08-29 17:02:47 -07:00
Sunil G
86709564cb YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan. 2019-08-29 17:02:47 -07:00
Sunil G
c539427696 YARN-7594. TestNMWebServices#testGetNMResourceInfo fails on trunk. Contributed by Gergely Novák. 2019-08-29 17:02:47 -07:00
Wangda Tan
ca7d0fda4f YARN-7573. Gpu Information page could be empty for nodes without GPU. (Sunil G via wangda) 
Change-Id: I7f614e5a589a09ce4e4286c84b706e05c29abd14
 2019-08-29 17:02:47 -07:00
Jonathan Hung
7939ae1695 YARN-9289. Backport YARN-7330 for GPU in UI to branch-2 2019-08-29 17:02:47 -07:00
Jian He
4d82318eb1 YARN-7396. NPE when accessing container logs due to null dirsHandler. Contributed by Jonathan Hung 2019-08-29 17:02:47 -07:00
Jonathan Hung
16faceb0da YARN-9174. Backport YARN-7224 for refactoring of GpuDevice class 2019-08-29 17:02:47 -07:00
Jonathan Hung
7ec4d7c6ce YARN-9280. Backport YARN-6620 to YARN-8200/branch-2 for NodeManager-side GPU isolation 2019-08-29 17:02:47 -07:00
Jonathan Hung
631dfc7277 YARN-9180. Port YARN-7033 NM recovery of assigned resources to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
56259bcecb YARN-9187. Backport YARN-6852 for GPU-specific native changes to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
078dfb09fe YARN-9175. Null resources check in ResourceInfo for branch-3.0 
(cherry picked from commit a0291a015c1af0ea1282849bd8fb32824d7452fa)
 2019-08-29 17:02:47 -07:00
Sunil G
4d30c90a1b YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan. 
(cherry picked from commit da0b6a354bf6f6bf37ca5a05a4a8eece09aa4893)
(cherry picked from commit 74030d808cd95e26a0c48500c08d269fcb4150ee)
 2019-08-29 17:02:47 -07:00
Daniel Templeton
72d90e750b YARN-7270 addendum: Reapplied changes after YARN-3926 backports 2019-08-29 17:02:47 -07:00
Jonathan Hung
d6673c5491 YARN-9188. Port YARN-7136 to branch-2 2019-08-29 17:02:47 -07:00
Wangda Tan
26e87be3f2 YARN-7056. [Partial backport] Document Resource Profiles feature. (Sunil G via wangda) 
Change-Id: I5f5d52cdf7aba4ac09684aab573e6916dbad183e
(cherry picked from commit 6b88cd1901a65b977fed759e322dcd75a2cd33b6)
 2019-08-29 17:02:47 -07:00
Sunil G
45b50c45d6 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R. 
(cherry picked from commit 805095496dfd8e50f71b70cf20845e954d3ba47c)
 2019-08-29 17:02:47 -07:00
Wangda Tan
8cea242e94 YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda) 
Change-Id: I88928a747ee3eec17bc76fb71e9aaa632d091f0f
(cherry picked from commit 53df3eac503064b8c4cd8eb2b659567622f4d8a7)
 2019-08-29 17:02:46 -07:00
Wangda Tan
f340726c9a YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
(cherry picked from commit e490602e9b306d5b8a543b93fb15a7395bb9a03d)
 2019-08-29 17:02:46 -07:00
Sunil G
74bb2d6ae7 YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan. 
(cherry picked from commit a333ba54e3f5a1ca57b0fc94bfd008f3cbd5b14b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
 2019-08-29 17:02:46 -07:00
Wangda Tan
1f05fa041a YARN-7043. [Partial backport] Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
(cherry picked from commit 39240b61a163b127eec385decd30ffd96e694c28)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/OpportunisticContainerAllocatorAMService.java
 2019-08-29 17:02:46 -07:00
Sunil G
7c37fbef12 YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin. 
(cherry picked from commit 758b7719430c981a6d4d1644d9d03cd16b981cf3)
 2019-08-29 17:02:46 -07:00
Wangda Tan
c69dd9eea6 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541af98a3805958f2b5b9db6be705d52ab)
 2019-08-29 17:02:46 -07:00
Wangda Tan
43030aa423 YARN-7042. Clean up unit tests after YARN-6610. (Daniel Templeton via wangda) 
Change-Id: I8e40f704b6fcdd5b14faa9548a27986501044fa1
(cherry picked from commit b1fe3a222e7673fd84a878622969f958022061e9)
 2019-08-29 17:02:46 -07:00
Sunil G
7b1c58b037 YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan. 
(cherry picked from commit a6989af1d91d8b6795862c39d154823f8488c8de)
 2019-08-29 17:02:46 -07:00
Sunil G
2fafdc7f8b YARN-6610. [YARN-3926] DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources. Contributed by Daniel Templeton. 
(cherry picked from commit a0030c8c6f5489d8285632a651394b2b2320255c)
 2019-08-29 17:02:46 -07:00
Sunil G
06c38ee7a4 YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G. 
(cherry picked from commit 2b51b262aba0191b80dc93799574c0b959cb4f4e)
 2019-08-29 17:02:46 -07:00
Yufei Gu
59cfecd60e YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit d5e9939ebb59e4f6f8e5e2fe7f619fb032f2911c)
 2019-08-29 17:02:46 -07:00
Daniel Templeton
d8f388652e YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)

(cherry picked from commit 3aeaafecb823ef6c175ea5d0d9cb726faacaf32d)
 2019-08-29 17:02:46 -07:00
Sunil G
1dc66dbb3f YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup. Contributed by Yeliang Cang. 
(cherry picked from commit 4e5632d28eb8f6b1b8830c2aa89ec6321d477977)
 2019-08-29 17:02:46 -07:00
Sunil G
ad6f095774 YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. 
(cherry picked from commit ca2808d8bbe49766029e5dba88ff863f6c993287)
 2019-08-29 17:02:46 -07:00
Jonathan Hung
d5963b2be9 YARN-9182. Backport YARN-6445 resource profile performance improvements to branch-2 2019-08-29 17:02:46 -07:00
Jonathan Hung
08923e324a YARN-9177. Use resource map for app metrics in TestCombinedSystemMetricsPublisher for branch-2 2019-08-29 17:02:46 -07:00
Jonathan Hung
7f614f7cee YARN-9181. Backport YARN-6232 for generic resource type usage to branch-2 2019-08-29 17:02:46 -07:00
Sunil G
c5e77e8b84 YARN-5588. [Partial backport] [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev. 
(cherry picked from commit 7805deed4896e470ebd2f6bbd1ba9962947c63cd)
 2019-08-29 17:02:46 -07:00
Arun Suresh
6db8c36ba9 YARN-5587. [Partial backport] Add support for resource profiles. (vvasudev via asuresh) 
(cherry picked from commit 6708ac330147b2d3816a31f2ee83e09c41fe0dd9)
 2019-08-29 17:02:46 -07:00
Rohith Sharma K S
f99d603aaf YARN-5586. Update the Resources class to consider all resource types. Contributed by Varun Vasudev. 
(cherry picked from commit 239c1824a07fede71dd751ed2c1f40036b22170e)
 2019-08-29 17:02:46 -07:00
Rohith Sharma K S
015a55a5c6 YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev. 
(cherry picked from commit 9e4ba6aff595b7a5b53d016c97c8334e10f1f2cb)
 2019-08-29 17:02:46 -07:00
Varun Vasudev
7d5050a768 YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev. 
(cherry picked from commit 759114b0063907d4c07ea6ee261e861bf5cc3a9a)
 2019-08-29 17:02:46 -07:00
Arun Suresh
817837e65c YARN-4829. Add support for binary units in Resource class.(vvasudev via asuresh) 
(cherry picked from commit 7ba698997b6845b664b2f8c82e384b457c30d176)
 2019-08-29 17:02:46 -07:00
Varun Vasudev
133eb31d54 YARN-4715. Add support to read resource types from a config file. Contributed by Varun Vasudev. 
(cherry picked from commit ceb12c59a34677518a198e19acb4e29cdfc8a0e1)
 2019-08-29 17:02:46 -07:00
Wangda Tan
be2afb4a5d YARN-4172. Extend DominantResourceCalculator to account for all resources. (Varun Vasudev via wangda) 
(cherry picked from commit 32c91223f1bd06561ea4ce2d1944e8d9a847f18c)
 2019-08-29 17:02:46 -07:00