Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,203 Commits 326 Branches 371 Tags
Commit Graph

5194 Commits

Author SHA1 Message Date
Szilard Nemeth
a7eb95f114 YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T. 2020-04-15 08:05:07 +02:00
Szilard Nemeth
9af7d905f9 YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja 2020-04-15 07:55:49 +02:00
Jonathan Hung
54599b177c YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
(cherry picked from commit e3a52804b03d646f15048c078f8c5292d5cbecfa)
 2020-04-09 10:37:36 -07:00
Szilard Nemeth
d2853d1bb0 YARN-10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke 2020-04-09 17:40:26 +02:00
Szilard Nemeth
9a66477e4d YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja 2020-04-09 12:22:12 +02:00
Akira Ajisaka
2a2c3a4094
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933) 
(cherry picked from commit e53d472bb07fc25d6cc5a13af1bd5ffdbf671164)
 2020-04-09 01:49:28 +09:00
Wilfred Spiegelenburg
a2067aafa9
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja 
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c

(cherry picked from commit 2214005c0f11955b2c50c4d2d4bd14947dd797ba)
 2020-04-08 13:12:31 +10:00
Akira Ajisaka
6d5f87b228
YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao. 
(cherry picked from commit c162648aff68552d87db8a013b850c17fee762c0)
 2020-04-01 16:06:41 +09:00
Jonathan Hung
5d3fb0ebe9 YARN-10200. Add number of containers to RMAppManager summary 
(cherry picked from commit 2de0572cdc1c6fdbfaab108b169b2d5b0c077e86)
 2020-03-25 10:27:48 -07:00
Siyao Meng
29e1880d27
HADOOP-16935. Backport HADOOP-10848. Cleanup calling of sun.security.krb5.Config. (#1912) 
(cherry picked from commit 0d47d283a611b10cb7a842712cb01fa0c043bdc4)

Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
 2020-03-24 16:01:33 -07:00
Szilard Nemeth
9e0d742025 YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori 2020-03-10 16:39:03 +01:00
Eric E Payne
153eac1d21 YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit ede05b19d1723147430fc426161326d46698507f)
 2020-03-10 14:28:13 +00:00
Inigo Goiri
733f9b76b6 YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan. 
(cherry picked from commit a43510e21d01e6c78e98e7ad9469cbea70a66466)
 2020-02-27 13:19:43 -08:00
Elixir Kook
7d0ff2dc85
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856) 
(cherry picked from commit d608e94f9208354897a59dc8e533535b1b8a30b9)
 2020-02-26 17:32:09 +09:00
Szilard Nemeth
92ad3bd099 YARN-10143. YARN-10101 broke Yarn logs CLI. Contributed by Adam Antal 2020-02-24 21:27:31 +01:00
Sunil G
de63115a2a YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph. 
(cherry picked from commit 6526f95bd281fc011f8776d21ff933087c5924de)
 2020-02-19 11:18:19 +05:30
Szilard Nemeth
6aec712c6c YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-11 09:18:44 +01:00
Sunil G
95b1cbcbd4 YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph 
(cherry picked from commit 28f730b317b23038bf9bd0775dd2cdb96518b13b)
 2020-02-09 21:15:31 +05:30
Jonathan Hung
aca930402c YARN-10116. Expose diagnostics in RMAppManager summary 
(cherry picked from commit 314e2f9d2e888fae1e5bf669aeeead84a928d282)
 2020-02-05 11:17:03 -08:00
Prabhu Joseph
7136ebbb7a YARN-10022. Add RM Rest API to validate a CapacityScheduler Config with delta change 
Contributed by Kinga Marton.

(cherry-picked from commit 1ab9c692fa9107f10c0d8fbfbb9e74f54bddc960)
 2020-02-04 14:06:23 +05:30
Eric Badger
5736ecd123 YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-29 04:07:28 +00:00
Abhishek Modi
be412546be YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4819704351c04dbeb90fc8154488f91)
 2020-01-23 15:23:45 +00:00
Szilard Nemeth
dcc16b07fa YARN-10083. Addendum to fix compilation error due to missing import 2020-01-22 17:18:03 +01:00
Szilard Nemeth
1e7679035f YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg 2020-01-22 16:50:15 +01:00
Szilard Nemeth
da416c826f YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:18:35 +01:00
Szilard Nemeth
bbdc39c13e YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph 2020-01-22 15:49:35 +01:00
Szilard Nemeth
c411a1468f YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph 2020-01-20 09:29:52 +01:00
Szilard Nemeth
805589fc71 YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R 2020-01-16 18:51:56 +01:00
Szilard Nemeth
0c2e312fef YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-15 10:51:02 +01:00
Szilard Nemeth
6a7dfb3bf3 YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal 2020-01-12 13:54:08 +01:00
Eric E Payne
3ba0fd1e50 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e4d6b8098622a1aa830355cc18d9589)
 2020-01-09 17:22:10 +00:00
Eric Badger
58db04ce15 YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan. 2020-01-08 19:44:43 +00:00
Eric E Payne
5e2be81fcb YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit b1e07d27cc1a26be4e5ebd1ab7b03ef15032bef0)
 2020-01-08 19:30:09 +00:00
Eric E Payne
b20ce118da YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 6899be5a1729e49cff45090acd2cf4f54aeac089)
 2020-01-08 17:42:41 +00:00
Prabhu Joseph
ad98a30810 YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg. 
(cherry Picked from commit 217b56ffdd5fa254f06734bc8cb6f04a02066f1a)
 2020-01-02 14:30:01 +05:30
Akira Ajisaka
85da5cb870
YARN-10055. bower install fails. (#1778) 
(cherry picked from commit 34ff7dbaf53cbbebcb163980f4b45221630c2882)
 2019-12-23 20:26:40 +09:00
Eric Badger
355ec33416 YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne 2019-12-20 19:32:36 +00:00
Jonathan Hung
0707d0a0ae YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92e8bad889c1ef15e0baaade6de6de4d)
 2019-12-18 13:18:22 -08:00
Jonathan Hung
01e2c996ff YARN-10039. Allow disabling app submission from REST endpoints 
(cherry picked from commit fddc3d55c3e309936216b8c61944e113509999e2)
 2019-12-18 10:51:57 -08:00
Eric Badger
18b515322c YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 
(cherry picked from commit f47dcf2d4cf437dd67d14dd05c60648d9ff87843)
 2019-12-17 17:36:52 +00:00
Akira Ajisaka
a3cdba7713
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena. 
(cherry picked from commit dc66de744826e0501040f8c2ca9e1edc076a80cf)
 2019-12-09 18:38:47 +09:00
Jonathan Hung
9228e3f0ad YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc5d03560169642b71636800680a3292)
 2019-12-08 16:36:08 -08:00
Jonathan Hung
8badcd989e Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources" 
This reverts commit b5235f1ed05f6506aa2c40825b2b647475a4d91a.
 2019-12-08 16:36:00 -08:00
Jonathan Hung
b5235f1ed0 YARN-10012. Guaranteed and max capacity queue metrics for custom resources 
(cherry picked from commit 92bce918dc5d03560169642b71636800680a3292)
 2019-12-08 16:03:34 -08:00
Sunil G
69dc329acc YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 
(cherry picked from commit 002dcc4ebf79bbaa5e603565640d8289991d781f)
 2019-12-08 14:40:45 -08:00
Szilard Nemeth
2ad7b90505 YARN-9993. Remove incorrectly committed files from YARN-9011. Contributed by Wilfred Spiegelenburg 2019-11-28 12:39:54 +01:00
Sunil G
f9b872b6ec YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:14:33 +05:30
Szilard Nemeth
8eda9fcab8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:24:06 +05:30
Szilard Nemeth
3fc8930129 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:26:58 +01:00
HUAN-PING SU
59a6261e81
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4e65f173bf2330715dceafdb6111190)
 2019-11-25 15:29:37 +09:00