Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,307 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4145 Commits

Author SHA1 Message Date
Eric E Payne e5bd8d2840 YARN-10177: Backport YARN-7307 to branch-2.10 Allow client/AM update supported resource types via YARN APIs 2020-09-04 18:23:08 +00:00
Masatake Iwasaki ecf88b972f
YARN-10358. Fix findbugs warnings in hadoop-yarn-project on branch-2.10. (#2164) 2020-08-31 13:52:29 +09:00
Kihwal Lee 283def8bc5 MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 
(cherry picked from commit 4571351ccc)
 2020-08-13 18:54:31 -05:00
Eric E Payne 21788f9fd4 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan and Jim Brennan. 2020-08-10 20:52:44 +00:00
Jonathan Hung 865828ae63 YARN-10251. Show extended resources on legacy RM UI. Contributed by Eric Payne 2020-08-07 17:45:04 -07:00
Eric Badger 120cd9eae6 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan. 2020-08-06 19:40:33 +00:00
Eric Badger 9bf554deae YARN-4575. ApplicationResourceUsageReport should return ALL reserved resource. 
Contributed by Bibin Chundatt and Eric Payne.

(cherry picked from commit 647be0c0f6)
 2020-08-05 23:20:52 +00:00
Eric Badger c84ae8f3d9 YARN-10363. TestRMAdminCLI.testHelp is failing in branch-2.10. Contributed by 
Bilwa S T.

(cherry picked from commit 87816e8a51)
 2020-07-31 22:40:41 +00:00
Eric E Payne 898ee03181 YARN-1529: Add Localization overhead metrics to NM. Contributed by Jim_Brennan. 2020-07-30 20:02:22 +00:00
Jonathan Hung 50e68e67b6 YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne 2020-07-28 13:45:14 -07:00
Eric Badger 2fae1c5b05 YARN-4771. Some containers can be skipped during log aggregation after NM 
restart. Contributed by Jason Lowe and Jim Brennan.

(cherry picked from commit ac5f21dbef)
 2020-07-24 23:08:31 +00:00
Akira Ajisaka 8cd8b4152d YARN-6515. Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager. Contributed by Naganarasimha G R. 
(cherry picked from commit 1a18d5e514)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
 2020-07-21 22:02:41 +09:00
Akira Ajisaka e53186954d HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li. 
(cherry picked from commit 1defe3a65a)
(cherry picked from commit 0d949d375e)
(cherry picked from commit 635067d03d)
 2020-07-20 13:39:00 -07:00
Eric Badger a4b419cdf5 YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by 
Jim Brennan.

(cherry picked from commit 09f1547697)
 2020-07-14 18:33:56 +00:00
Eric E Payne 7190507aa2 YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 0427100b75)
 2020-07-13 21:51:32 +00:00
Eric E Payne 76fa956d3b YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan). 
(cherry picked from commit e6794f2fc4)
 2020-06-29 19:55:18 +00:00
Eric Badger a11c606910 YARN-10312. Add support for yarn logs -logFile to retain backward compatibility. 
Contributed by Jim Brennan

(cherry picked from commit 7363931942)
 2020-06-12 19:21:45 +00:00
Tao Yang a91d4d612f YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Jim Brennan. 2020-06-12 10:56:24 +08:00
Eric E Payne af324e3153 YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger). 
(cherry picked from commit 56247db302)
(cherry picked from commit 034d458511)
(cherry picked from commit 2e4892061a)
 2020-06-09 22:16:16 +00:00
Jonathan Hung b9a0f99966 YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R 2020-06-01 10:48:41 -07:00
Jonathan Hung b8c88f6968 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T 
(cherry picked from commit fff1d2c122)
(cherry picked from commit 564d3211f2)
(cherry picked from commit a7ea55e015)
(cherry picked from commit b3e9aff5f7)
 2020-05-12 10:53:37 -07:00
Eric E Payne d0b0fb6e04 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 2020-05-11 17:07:35 +00:00
Ahmed Hussein 0f0707fb0d YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-05-06 12:48:12 -05:00
Ahmed Hussein 712da57658 YARN-10255. fix intermittent failure TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic in branch-2.10 (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-04-30 17:50:55 -05:00
Jonathan Hung 27ad054696 YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 2020-04-30 12:16:15 -07:00
Miklos Szegedi 71063434a6 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
(cherry picked from commit 21948b048e)
 2020-04-27 14:14:42 -07:00
Wangda Tan 34804679e3 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
(cherry picked from commit 17aa40f669)
 2020-04-19 12:51:56 +09:00
Jonathan Hung ebce5c74e6 YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 
(cherry picked from commit cd6c10de442fc3a53c9ed5521ac1d944a6ac95c6)
(cherry picked from commit 2c79865b951d0fdea7f576ce31e310b4074ecedd)
 2020-04-17 10:35:39 -07:00
Eric Yang 7945ed40f0 HADOOP-16361. Fixed TestSecureLogins#testValidKerberosName on branch-2. 
Contributed by Jim Brennan via eyang
 2020-04-16 12:12:21 -04:00
Eric Badger 0e05b02136 YARN-8680. YARN NM: Implement Iterable Abstraction for LocalResourceTracker 
state. Contributed by Pradeep Ambati.
 2020-04-14 18:30:56 +00:00
Jonathan Hung c0394c5434 YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
(cherry picked from commit e3a52804b03d646f15048c078f8c5292d5cbecfa)
(cherry picked from commit 54599b177c)
(cherry picked from commit 6271a2852e)
 2020-04-09 11:07:59 -07:00
Eric E Payne 0ce55c699c YARN-8242. YARN NM: OOM error while reading back the state store on recovery. Contributed by Pradeep Ambati and Kanwaljeet Sachdev 
YARN-10227. Pull YARN-8242 back to branch-2.10. Contributed by Jim Brennan
 2020-04-09 17:18:07 +00:00
Eric Badger a3343c4097 YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed 
Hussein.
 2020-04-09 00:52:45 +00:00
Jonathan Hung a7556f1ec2 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 2020-03-27 16:10:39 -07:00
Jonathan Hung 1c8529f030 YARN-10200. Add number of containers to RMAppManager summary 
(cherry picked from commit 2de0572cdc1c6fdbfaab108b169b2d5b0c077e86)
(cherry picked from commit 5d3fb0ebe9)
(cherry picked from commit 9c6dd8c83a)
 2020-03-25 10:39:45 -07:00
Eric Badger d01e9c125b Revert "YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed" 
This reverts commit 85eac8b0df.
 2020-03-17 22:15:38 +00:00
Eric Badger 85eac8b0df YARN-2710. RM HA tests failed intermittently on trunk. Contributed by Ahmed 
Hussein.
 2020-03-16 22:31:43 +00:00
Eric E Payne e9a7b30fa0 YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit ede05b19d1)
 2020-03-10 15:48:49 +00:00
Inigo Goiri 2d44d7f139 YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan. 
(cherry picked from commit a43510e21d)
 2020-02-27 13:22:16 -08:00
Elixir Kook e844f6ffaa
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856) 
(cherry picked from commit d608e94f92)
 2020-02-26 17:33:05 +09:00
Eric E Payne 254b94bf5e YARN-10140: TestTimelineAuthFilterForV2 fails due to login failures in branch-2.10. Contributed by Ahmed Hussein (ahussein) 2020-02-25 16:36:50 +00:00
Jonathan Hung c34c87b1a8 YARN-8292. Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan and Eric Payne 2020-02-07 17:29:32 -08:00
Jonathan Hung 4fce8c8023 YARN-10116. Expose diagnostics in RMAppManager summary 
(cherry picked from commit 314e2f9d2e)
(cherry picked from commit 147897da4b420b4749f3c7b410f4c329632c3352)
(cherry picked from commit fa35b8370ce14c9b8ee911b73fda380817b964fd)
 2020-02-05 11:16:09 -08:00
Eric Badger 21970f6f67 YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-30 21:29:33 +00:00
Abhishek Modi 296786a647 YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4)
 2020-01-23 17:12:25 +00:00
Eric E Payne c296e3c0aa YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e)
 2020-01-09 17:52:26 +00:00
Eric Badger ab29b54b19 YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan. 2020-01-08 20:13:13 +00:00
Eric E Payne 5cca5ca81b YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit b1e07d27cc)
 2020-01-08 19:59:13 +00:00
Eric E Payne 2ae1b3568b YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 6899be5a17)
 2020-01-08 18:04:12 +00:00
Eric Badger cb5b80d6cb YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne. 2019-12-20 19:40:55 +00:00