Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,288 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3744 Commits

Author SHA1 Message Date
Szilard Nemeth 30d7a06686 YARN-10295. CapacityScheduler NPE can cause apps to get stuck without resources. Contributed by Benjamin Teke 2020-06-10 18:16:21 +02:00
Eric E Payne 034d458511 YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger). 
(cherry picked from commit 56247db302)
 2020-06-09 21:09:11 +00:00
Szilard Nemeth 54c89ffad4 YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori 2020-06-05 09:49:54 +02:00
Jonathan Hung f31146bc1f YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R 
(cherry picked from commit c30c23cb66)
(cherry picked from commit 7a323a45aa)
 2020-05-29 10:43:33 -07:00
Jonathan Hung a7ea55e015 YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T 
(cherry picked from commit fff1d2c122)
(cherry picked from commit 564d3211f2)
 2020-05-12 10:52:58 -07:00
Ahmed Hussein 7740b88ee9 YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 92e3ebb401)
 2020-05-06 12:32:36 -05:00
Ahmed Hussein b23a585cb1 YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit f5081a9a5d)
 2020-05-04 10:49:45 -05:00
Szilard Nemeth 1dabbd5006 YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph 2020-04-28 17:52:14 +02:00
Szilard Nemeth f445487d50 YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke 2020-04-27 09:59:13 +02:00
Szilard Nemeth 0dbb02a76c YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke 2020-04-24 11:31:33 +02:00
Szilard Nemeth 3b67dc24aa YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke 2020-04-24 11:15:53 +02:00
Akira Ajisaka 7b036c512f
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939) 
(cherry picked from commit 9827ff2961)
 2020-04-24 10:47:36 +09:00
Wei-Chiu Chuang 48f1c8ffb6 Revert "YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja" 
This reverts commit a2067aafa9.
 2020-04-23 12:37:21 -07:00
Szilard Nemeth c81844d8a5 YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja 2020-04-23 14:57:18 +02:00
Szilard Nemeth 764fa92c9f YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori 2020-04-23 14:51:07 +02:00
Szilard Nemeth 73cb3d3cb3 YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja 2020-04-18 09:38:22 +02:00
Jonathan Hung d1af4e0fae YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 
(cherry picked from commit 49ae9b2137)
 2020-04-17 10:36:16 -07:00
Szilard Nemeth 2f01a91428 YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke 2020-04-15 08:24:15 +02:00
Szilard Nemeth 58e559b5ac YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori 2020-04-15 08:16:15 +02:00
Jonathan Hung 54599b177c YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
(cherry picked from commit e3a52804b03d646f15048c078f8c5292d5cbecfa)
 2020-04-09 10:37:36 -07:00
Szilard Nemeth d2853d1bb0 YARN-10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke 2020-04-09 17:40:26 +02:00
Wilfred Spiegelenburg a2067aafa9
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja 
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c

(cherry picked from commit 2214005c0f)
 2020-04-08 13:12:31 +10:00
Jonathan Hung 5d3fb0ebe9 YARN-10200. Add number of containers to RMAppManager summary 
(cherry picked from commit 2de0572cdc1c6fdbfaab108b169b2d5b0c077e86)
 2020-03-25 10:27:48 -07:00
Szilard Nemeth 9e0d742025 YARN-9419. Log a warning if GPU isolation is enabled but LinuxContainerExecutor is disabled. Contribued by Andras Gyori 2020-03-10 16:39:03 +01:00
Eric E Payne 153eac1d21 YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit ede05b19d1)
 2020-03-10 14:28:13 +00:00
Inigo Goiri 733f9b76b6 YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan. 
(cherry picked from commit a43510e21d)
 2020-02-27 13:19:43 -08:00
Sunil G de63115a2a YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph. 
(cherry picked from commit 6526f95bd2)
 2020-02-19 11:18:19 +05:30
Szilard Nemeth 6aec712c6c YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-11 09:18:44 +01:00
Sunil G 95b1cbcbd4 YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph 
(cherry picked from commit 28f730b317)
 2020-02-09 21:15:31 +05:30
Jonathan Hung aca930402c YARN-10116. Expose diagnostics in RMAppManager summary 
(cherry picked from commit 314e2f9d2e)
 2020-02-05 11:17:03 -08:00
Prabhu Joseph 7136ebbb7a YARN-10022. Add RM Rest API to validate a CapacityScheduler Config with delta change 
Contributed by Kinga Marton.

(cherry-picked from commit 1ab9c692fa)
 2020-02-04 14:06:23 +05:30
Eric Badger 5736ecd123 YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-29 04:07:28 +00:00
Abhishek Modi be412546be YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4)
 2020-01-23 15:23:45 +00:00
Szilard Nemeth 1e7679035f YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg 2020-01-22 16:50:15 +01:00
Szilard Nemeth da416c826f YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:18:35 +01:00
Szilard Nemeth bbdc39c13e YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph 2020-01-22 15:49:35 +01:00
Szilard Nemeth 805589fc71 YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R 2020-01-16 18:51:56 +01:00
Szilard Nemeth 0c2e312fef YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-15 10:51:02 +01:00
Szilard Nemeth 6a7dfb3bf3 YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal 2020-01-12 13:54:08 +01:00
Eric E Payne 3ba0fd1e50 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e)
 2020-01-09 17:22:10 +00:00
Eric Badger 58db04ce15 YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan. 2020-01-08 19:44:43 +00:00
Eric E Payne 5e2be81fcb YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit b1e07d27cc)
 2020-01-08 19:30:09 +00:00
Eric E Payne b20ce118da YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 6899be5a17)
 2020-01-08 17:42:41 +00:00
Prabhu Joseph ad98a30810 YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg. 
(cherry Picked from commit 217b56ffdd)
 2020-01-02 14:30:01 +05:30
Eric Badger 355ec33416 YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne 2019-12-20 19:32:36 +00:00
Jonathan Hung 0707d0a0ae YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92)
 2019-12-18 13:18:22 -08:00
Jonathan Hung 01e2c996ff YARN-10039. Allow disabling app submission from REST endpoints 
(cherry picked from commit fddc3d55c3)
 2019-12-18 10:51:57 -08:00
Eric Badger 18b515322c YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 
(cherry picked from commit f47dcf2d4c)
 2019-12-17 17:36:52 +00:00
Jonathan Hung 9228e3f0ad YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc)
 2019-12-08 16:36:08 -08:00
Jonathan Hung 8badcd989e Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources" 
This reverts commit b5235f1ed0.
 2019-12-08 16:36:00 -08:00