Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,144 Commits 326 Branches 371 Tags
Commit Graph

687 Commits

Author SHA1 Message Date
Eric E Payne
c296e3c0aa YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e4d6b8098622a1aa830355cc18d9589)
 2020-01-09 17:52:26 +00:00
Jonathan Hung
c4a8c834e5 YARN-10039. Allow disabling app submission from REST endpoints 
(cherry picked from commit 406c35dd1213ef354570f8337e54a4fb4bbf5bad)
 2019-12-18 11:01:29 -08:00
Jonathan Hung
d74f64aa99 Setting version to 2.10.1-SNAPSHOT after branch-2 to branch-2.10 rename 2019-12-09 11:49:59 -08:00
Weiwei Yang
e7ecb4f51f YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 
(cherry picked from commit bc6374f282dbff3b9ed91fb5d7825d57e6720f5e)
 2019-12-03 13:56:54 -08:00
Jason Lowe
dbbe23f269 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton 
(cherry picked from commit 25df5054216a6a76d09d9c49984f8075ebc6a197)
 2019-11-21 11:55:20 -08:00
Robert Kanter
235d6c73e5 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c43644b4ad22d5385c22cad8ed573c0b1ba)
(cherry picked from commit 0506c762b2f96ccb2c12cce8fd4659536236354e)
 2019-11-14 15:47:24 -08:00
Daniel Templeton
7f1d2e196c YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory 
(cherry picked from commit 8498d287cd3beddcf8fe19625227e09982ec4be2)
 2019-11-14 14:56:23 -08:00
Eric Badger
128631c5ac YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 
(cherry picked from commit eef34f2d87a75e16b2cca870d99a5e1e28c31d9b)
 2019-10-25 17:45:40 +00:00
Jonathan Hung
108127238b Preparing for 2.11.0 development 2019-10-15 10:01:45 -07:00
Jonathan Hung
ab7181e9af YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:19:39 -07:00
bibinchundatt
c57e6bc385 YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:13:53 +05:30
Jonathan Hung
eedbf9d195 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)
(cherry picked from commit 10bdcb6f1da3b86146efa479c0bbc8d1da505789)
 2019-09-24 13:51:11 -07:00
Wangda Tan
727cbcb41d YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. (wangda) 
(cherry picked from commit df27824b80b8eca03d02837efe2da31f089e67ec)
 2019-09-19 15:27:30 -07:00
Jonathan Hung
fb78b7c45c YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:03:33 -07:00
Wangda Tan
056b54c62c YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2019-08-29 17:02:47 -07:00
Jonathan Hung
28fbc4e344 YARN-9272. Backport YARN-7738 for refreshing max allocation for multiple resource types 2019-08-29 17:02:47 -07:00
Jonathan Hung
8e5ce8055e YARN-9271. Backport YARN-6927 for resource type support in MapReduce 2019-08-29 17:02:47 -07:00
Sunil G
1875231be5 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 2019-08-29 17:02:47 -07:00
Daniel Templeton
062fbd084d YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2019-08-29 17:02:47 -07:00
Jonathan Hung
7939ae1695 YARN-9289. Backport YARN-7330 for GPU in UI to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung
7ec4d7c6ce YARN-9280. Backport YARN-6620 to YARN-8200/branch-2 for NodeManager-side GPU isolation 2019-08-29 17:02:47 -07:00
Sunil G
4d30c90a1b YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan. 
(cherry picked from commit da0b6a354bf6f6bf37ca5a05a4a8eece09aa4893)
(cherry picked from commit 74030d808cd95e26a0c48500c08d269fcb4150ee)
 2019-08-29 17:02:47 -07:00
Daniel Templeton
72d90e750b YARN-7270 addendum: Reapplied changes after YARN-3926 backports 2019-08-29 17:02:47 -07:00
Jonathan Hung
d6673c5491 YARN-9188. Port YARN-7136 to branch-2 2019-08-29 17:02:47 -07:00
Sunil G
45b50c45d6 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R. 
(cherry picked from commit 805095496dfd8e50f71b70cf20845e954d3ba47c)
 2019-08-29 17:02:47 -07:00
Wangda Tan
8cea242e94 YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda) 
Change-Id: I88928a747ee3eec17bc76fb71e9aaa632d091f0f
(cherry picked from commit 53df3eac503064b8c4cd8eb2b659567622f4d8a7)
 2019-08-29 17:02:46 -07:00
Wangda Tan
f340726c9a YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
(cherry picked from commit e490602e9b306d5b8a543b93fb15a7395bb9a03d)
 2019-08-29 17:02:46 -07:00
Sunil G
74bb2d6ae7 YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan. 
(cherry picked from commit a333ba54e3f5a1ca57b0fc94bfd008f3cbd5b14b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
 2019-08-29 17:02:46 -07:00
Wangda Tan
1f05fa041a YARN-7043. [Partial backport] Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
(cherry picked from commit 39240b61a163b127eec385decd30ffd96e694c28)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/OpportunisticContainerAllocatorAMService.java
 2019-08-29 17:02:46 -07:00
Sunil G
7c37fbef12 YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin. 
(cherry picked from commit 758b7719430c981a6d4d1644d9d03cd16b981cf3)
 2019-08-29 17:02:46 -07:00
Wangda Tan
c69dd9eea6 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541af98a3805958f2b5b9db6be705d52ab)
 2019-08-29 17:02:46 -07:00
Sunil G
7b1c58b037 YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan. 
(cherry picked from commit a6989af1d91d8b6795862c39d154823f8488c8de)
 2019-08-29 17:02:46 -07:00
Sunil G
06c38ee7a4 YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G. 
(cherry picked from commit 2b51b262aba0191b80dc93799574c0b959cb4f4e)
 2019-08-29 17:02:46 -07:00
Yufei Gu
59cfecd60e YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit d5e9939ebb59e4f6f8e5e2fe7f619fb032f2911c)
 2019-08-29 17:02:46 -07:00
Daniel Templeton
d8f388652e YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)

(cherry picked from commit 3aeaafecb823ef6c175ea5d0d9cb726faacaf32d)
 2019-08-29 17:02:46 -07:00
Sunil G
ad6f095774 YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. 
(cherry picked from commit ca2808d8bbe49766029e5dba88ff863f6c993287)
 2019-08-29 17:02:46 -07:00
Jonathan Hung
d5963b2be9 YARN-9182. Backport YARN-6445 resource profile performance improvements to branch-2 2019-08-29 17:02:46 -07:00
Jonathan Hung
7f614f7cee YARN-9181. Backport YARN-6232 for generic resource type usage to branch-2 2019-08-29 17:02:46 -07:00
Arun Suresh
6db8c36ba9 YARN-5587. [Partial backport] Add support for resource profiles. (vvasudev via asuresh) 
(cherry picked from commit 6708ac330147b2d3816a31f2ee83e09c41fe0dd9)
 2019-08-29 17:02:46 -07:00
Rohith Sharma K S
015a55a5c6 YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev. 
(cherry picked from commit 9e4ba6aff595b7a5b53d016c97c8334e10f1f2cb)
 2019-08-29 17:02:46 -07:00
Varun Vasudev
7d5050a768 YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev. 
(cherry picked from commit 759114b0063907d4c07ea6ee261e861bf5cc3a9a)
 2019-08-29 17:02:46 -07:00
Arun Suresh
817837e65c YARN-4829. Add support for binary units in Resource class.(vvasudev via asuresh) 
(cherry picked from commit 7ba698997b6845b664b2f8c82e384b457c30d176)
 2019-08-29 17:02:46 -07:00
Varun Vasudev
133eb31d54 YARN-4715. Add support to read resource types from a config file. Contributed by Varun Vasudev. 
(cherry picked from commit ceb12c59a34677518a198e19acb4e29cdfc8a0e1)
 2019-08-29 17:02:46 -07:00
Wangda Tan
afcf9c55b5 YARN-4081. Add support for multiple resource types in the Resource class. (Varun Vasudev via wangda) 
(cherry picked from commit 1bbab7c1570a2438b2fa6da70397dd1d5211a137)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/resource/Resources.java
 2019-08-29 17:02:46 -07:00
Haibo Chen
4a9fc45f6f YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)
(cherry picked from commit 8d357343c4bc9f18e25543583f8f217b8a2f621b)
(cherry picked from commit f943bff2546330bc9eb0914d1a0fa1ef4c709b72)
(cherry picked from commit 6109ecf16d6f0ebf1e314e163b5c41ad75758be1)
 2019-08-06 15:14:45 -07:00
Takanobu Asanuma
e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:41:21 +09:00
Akira Ajisaka
cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00
Vrushali C
396fcee0b0 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-03-07 23:04:36 -08:00
Sunil G
a91933620d YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 
(cherry picked from commit 6ffe6ea8999c1216b6e510e350535754431bac2d)
 2019-02-07 19:11:12 +05:30
Eric E Payne
c6c5e94de4 YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 
(cherry picked from commit 04105bbfdb041a41062c856632641140de84fba8)
 2019-01-29 18:22:12 +00:00