Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,484 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

697 Commits

Author SHA1 Message Date
Eric Yang 0d88d832cd YARN-7266. Fixed deadlock in Timeline Server thread initialization. 
Contributed by Prabhu Joseph

(cherry picked from commit 7b42e0e32a)
 2021-11-29 12:43:05 +09:00
Jim Brennan 33c4d4570d YARN-10697. Resources are displayed in bytes in UI for schedulers other than capacity. Contributed by Bilwa S T. 
(cherry picked from commit 34e507cb8c)
 2021-03-23 19:05:57 +00:00
Wangda Tan c6033eeeef YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
(cherry picked from commit c962371430)
 2021-03-16 11:03:56 -07:00
Masatake Iwasaki f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Eric E Payne e5bd8d2840 YARN-10177: Backport YARN-7307 to branch-2.10 Allow client/AM update supported resource types via YARN APIs 2020-09-04 18:23:08 +00:00
Eric E Payne 898ee03181 YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan. 2020-07-30 20:02:22 +00:00
Eric Badger a4b419cdf5 YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by 
Jim Brennan.

(cherry picked from commit 09f1547697)
 2020-07-14 18:33:56 +00:00
Eric E Payne d0b0fb6e04 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2020-05-11 17:07:35 +00:00
Jonathan Hung ebce5c74e6 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 
(cherry picked from commit cd6c10de442fc3a53c9ed5521ac1d944a6ac95c6)
(cherry picked from commit 2c79865b951d0fdea7f576ce31e310b4074ecedd)
 2020-04-17 10:35:39 -07:00
Jonathan Hung c0394c5434 YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
(cherry picked from commit e3a52804b03d646f15048c078f8c5292d5cbecfa)
(cherry picked from commit 54599b177c)
(cherry picked from commit 6271a2852e)
 2020-04-09 11:07:59 -07:00
Eric E Payne c296e3c0aa YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e)
 2020-01-09 17:52:26 +00:00
Jonathan Hung c4a8c834e5 YARN-10039. Allow disabling app submission from REST endpoints 
(cherry picked from commit 406c35dd12)
 2019-12-18 11:01:29 -08:00
Jonathan Hung d74f64aa99 Setting version to 2.10.1-SNAPSHOT after branch-2 to branch-2.10 rename 2019-12-09 11:49:59 -08:00
Weiwei Yang e7ecb4f51f YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 
(cherry picked from commit bc6374f282)
 2019-12-03 13:56:54 -08:00
Jason Lowe dbbe23f269 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton 
(cherry picked from commit 25df505421)
 2019-11-21 11:55:20 -08:00
Robert Kanter 235d6c73e5 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c4364)
(cherry picked from commit 0506c762b2)
 2019-11-14 15:47:24 -08:00
Daniel Templeton 7f1d2e196c YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory 
(cherry picked from commit 8498d287cd)
 2019-11-14 14:56:23 -08:00
Eric Badger 128631c5ac YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 
(cherry picked from commit eef34f2d87)
 2019-10-25 17:45:40 +00:00
Jonathan Hung 108127238b Preparing for 2.11.0 development 2019-10-15 10:01:45 -07:00
Jonathan Hung ab7181e9af YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:19:39 -07:00
bibinchundatt c57e6bc385 YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:13:53 +05:30
Jonathan Hung eedbf9d195 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)
(cherry picked from commit 10bdcb6f1da3b86146efa479c0bbc8d1da505789)
 2019-09-24 13:51:11 -07:00
Wangda Tan 727cbcb41d YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. (wangda) 
(cherry picked from commit df27824b80)
 2019-09-19 15:27:30 -07:00
Jonathan Hung fb78b7c45c YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:03:33 -07:00
Wangda Tan 056b54c62c YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2019-08-29 17:02:47 -07:00
Jonathan Hung 28fbc4e344 YARN-9272. Backport YARN-7738 for refreshing max allocation for multiple resource types 2019-08-29 17:02:47 -07:00
Jonathan Hung 8e5ce8055e YARN-9271. Backport YARN-6927 for resource type support in MapReduce 2019-08-29 17:02:47 -07:00
Sunil G 1875231be5 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 2019-08-29 17:02:47 -07:00
Daniel Templeton 062fbd084d YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2019-08-29 17:02:47 -07:00
Jonathan Hung 7939ae1695 YARN-9289. Backport YARN-7330 for GPU in UI to branch-2 2019-08-29 17:02:47 -07:00
Jonathan Hung 7ec4d7c6ce YARN-9280. Backport YARN-6620 to YARN-8200/branch-2 for NodeManager-side GPU isolation 2019-08-29 17:02:47 -07:00
Sunil G 4d30c90a1b YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan. 
(cherry picked from commit da0b6a354b)
(cherry picked from commit 74030d808c)
 2019-08-29 17:02:47 -07:00
Daniel Templeton 72d90e750b YARN-7270 addendum: Reapplied changes after YARN-3926 backports 2019-08-29 17:02:47 -07:00
Jonathan Hung d6673c5491 YARN-9188. Port YARN-7136 to branch-2 2019-08-29 17:02:47 -07:00
Sunil G 45b50c45d6 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R. 
(cherry picked from commit 805095496d)
 2019-08-29 17:02:47 -07:00
Wangda Tan 8cea242e94 YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda) 
Change-Id: I88928a747ee3eec17bc76fb71e9aaa632d091f0f
(cherry picked from commit 53df3eac50)
 2019-08-29 17:02:46 -07:00
Wangda Tan f340726c9a YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
(cherry picked from commit e490602e9b)
 2019-08-29 17:02:46 -07:00
Sunil G 74bb2d6ae7 YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan. 
(cherry picked from commit a333ba54e3)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/MetricsOverviewTable.java
 2019-08-29 17:02:46 -07:00
Wangda Tan 1f05fa041a YARN-7043. [Partial backport] Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
(cherry picked from commit 39240b61a1)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/OpportunisticContainerAllocatorAMService.java
 2019-08-29 17:02:46 -07:00
Sunil G 7c37fbef12 YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin. 
(cherry picked from commit 758b771943)
 2019-08-29 17:02:46 -07:00
Wangda Tan c69dd9eea6 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541a)
 2019-08-29 17:02:46 -07:00
Sunil G 7b1c58b037 YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan. 
(cherry picked from commit a6989af1d9)
 2019-08-29 17:02:46 -07:00
Sunil G 06c38ee7a4 YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G. 
(cherry picked from commit 2b51b262ab)
 2019-08-29 17:02:46 -07:00
Yufei Gu 59cfecd60e YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu) 
(cherry picked from commit d5e9939ebb)
 2019-08-29 17:02:46 -07:00
Daniel Templeton d8f388652e YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)

(cherry picked from commit 3aeaafecb8)
 2019-08-29 17:02:46 -07:00
Sunil G ad6f095774 YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. 
(cherry picked from commit ca2808d8bb)
 2019-08-29 17:02:46 -07:00
Jonathan Hung d5963b2be9 YARN-9182. Backport YARN-6445 resource profile performance improvements to branch-2 2019-08-29 17:02:46 -07:00
Jonathan Hung 7f614f7cee YARN-9181. Backport YARN-6232 for generic resource type usage to branch-2 2019-08-29 17:02:46 -07:00
Arun Suresh 6db8c36ba9 YARN-5587. [Partial backport] Add support for resource profiles. (vvasudev via asuresh) 
(cherry picked from commit 6708ac3301)
 2019-08-29 17:02:46 -07:00
Rohith Sharma K S 015a55a5c6 YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev. 
(cherry picked from commit 9e4ba6aff5)
 2019-08-29 17:02:46 -07:00