Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,565 Commits 326 Branches 371 Tags
Commit Graph

5160 Commits

Author SHA1 Message Date
Jonathan Hung
9c6dd8c83a YARN-10200. Add number of containers to RMAppManager summary 
(cherry picked from commit 2de0572cdc1c6fdbfaab108b169b2d5b0c077e86)
(cherry picked from commit 5d3fb0ebe9d3f3395320b82a76194ba6fad01e00)
 2020-03-25 10:33:18 -07:00
Eric E Payne
80e394e81b YARN-942. TestContainerSchedulerQueuing.testKillOnlyRequiredOpportunisticContainers fails sporadically Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit ede05b19d1723147430fc426161326d46698507f)
 2020-03-10 15:25:10 +00:00
Wei-Chiu Chuang
58b025c8f4 HADOOP-16882. Update jackson-databind to 2.9.10.2 in branch-3.1, branch-2.10. Contributed by Lisheng Sun. 2020-02-28 16:24:38 -08:00
Inigo Goiri
4924622e6e YARN-10161. TestRouterWebServicesREST is corrupting STDOUT. Contributed by Jim Brennan. 
(cherry picked from commit a43510e21d01e6c78e98e7ad9469cbea70a66466)
 2020-02-27 13:20:17 -08:00
Elixir Kook
9ccefe2262
YARN-10156. Fix typo 'complaint' which means quite different in Federation.md (#1856) 
(cherry picked from commit d608e94f9208354897a59dc8e533535b1b8a30b9)
 2020-02-26 17:32:47 +09:00
Sunil G
a6124cd2b8 YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph. 
(cherry picked from commit 6526f95bd281fc011f8776d21ff933087c5924de)
 2020-02-19 11:18:53 +05:30
Prabhu Joseph
00fde836f7 YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change. 
Contributed by Kinga Marton.

(cherry-picked from commit 1ab9c692fa9107f10c0d8fbfbb9e74f54bddc960)
 2020-02-11 22:23:07 +05:30
Jonathan Hung
5dfd1dcfe3 YARN-10116. Expose diagnostics in RMAppManager summary 
(cherry picked from commit 314e2f9d2e888fae1e5bf669aeeead84a928d282)
(cherry picked from commit 147897da4b420b4749f3c7b410f4c329632c3352)
 2020-02-05 11:17:24 -08:00
Eric Badger
4af7d14ce2 YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne. 2020-01-29 22:45:02 +00:00
Abhishek Modi
868a0f5ef0 YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4819704351c04dbeb90fc8154488f91)
 2020-01-23 15:50:28 +00:00
Szilard Nemeth
9638985428 YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg 2020-01-22 16:30:59 +01:00
Szilard Nemeth
2cdaeca84b YARN-8148. Update decimal values for queue capacities shown on queue status CLI. Contributed by Prabhu Joseph 2020-01-20 09:30:48 +01:00
Sunil G
af89b5b086 YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg. 2020-01-15 17:07:34 +05:30
Eric E Payne
ec40c1f400 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e4d6b8098622a1aa830355cc18d9589)
 2020-01-09 17:41:10 +00:00
Eric Badger
cba0bbe98c YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan. 2020-01-08 20:02:43 +00:00
Eric E Payne
a110024fcc YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit b1e07d27cc1a26be4e5ebd1ab7b03ef15032bef0)
 2020-01-08 19:41:53 +00:00
Eric E Payne
08a1464444 YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 6899be5a1729e49cff45090acd2cf4f54aeac089)
 2020-01-08 17:50:43 +00:00
Prabhu Joseph
807d7aa3af YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 217b56ffdd5fa254f06734bc8cb6f04a02066f1a)
 2020-01-02 14:41:47 +05:30
Akira Ajisaka
01edc654df
YARN-10055. bower install fails. (#1778) 
(cherry picked from commit 34ff7dbaf53cbbebcb163980f4b45221630c2882)
 2019-12-23 20:27:06 +09:00
Eric Badger
0f01206e15 YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne 
(cherry picked from commit 355ec3341685deb93a40daffa3f28965b7609286)
 2019-12-20 19:34:29 +00:00
Jonathan Hung
750fb4c321 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92e8bad889c1ef15e0baaade6de6de4d)
(cherry picked from commit 0707d0a0ae36456f3467cbb408c3a9a0073c70f7)
 2019-12-18 13:25:30 -08:00
Jonathan Hung
406c35dd12 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:58:32 -08:00
Eric Badger
86120637a4 YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 
(cherry picked from commit f47dcf2d4cf437dd67d14dd05c60648d9ff87843)
 2019-12-17 17:39:51 +00:00
Jonathan Hung
9b4368a62f YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc5d03560169642b71636800680a3292)
(cherry picked from commit 9228e3f0ad4e94cedd59c5b02a87ce6c267db5b5)
 2019-12-08 16:38:21 -08:00
Sunil G
67cf1f94cd YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 
(cherry picked from commit 002dcc4ebf79bbaa5e603565640d8289991d781f)
(cherry picked from commit 69dc329acc0174b133e8b68f16b3e894329a6fd0)
 2019-12-08 14:43:20 -08:00
Sunil G
57c499ef19 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:01:16 +05:30
Szilard Nemeth
1172ebd6e8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:30:10 +05:30
Szilard Nemeth
80a97e6ac2 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:27:55 +01:00
HUAN-PING SU
fb2df5234d
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4e65f173bf2330715dceafdb6111190)
 2019-11-25 15:58:09 +09:00
Szilard Nemeth
df157211c9 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 13:00:18 +01:00
Tao Yang
c95e772d5d YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:10:07 +08:00
Eric Yang
1669a5c1cb YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:18:38 -05:00
Sunil G
9a51380208 YARN-9873. Version Number for each Scheduler Config Change. Contributed by Prabhu Joseph. 
(cherry picked from commit be901f49628a553cfe6a3942478cb528cce2b266)
 2019-11-19 17:17:45 +05:30
Abhishek Modi
96e07b7e97 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph. 
(cherry picked from commit 751b5a1ac871734c46cb934d100a5367f1dbf791)
 2019-11-19 16:41:04 +05:30
Sunil G
dab4cc6320 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph. 2019-11-19 16:19:19 +05:30
Sunil G
9b17fa7091 YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph. 
(cherry picked from commit 51e7d1b37eb06165657befdae82c015055c58334)
 2019-11-19 15:43:19 +05:30
Sunil G
f24d1fac35 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 215f2052fc3b7e366e8bd1bd332663966fa9206c)
 2019-11-19 14:05:51 +05:30
Prabhu Joseph
d271d174ca YARN-9900. Revert Invalid Config and Refresh Support in SchedulerConfig Format. 
Signed-off-by: prabhujoseph <pjoseph@cloudera.com>
 2019-11-18 15:02:59 +05:30
Jonathan Hung
4d274f60bc Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aac608746946dc2ed5711549a01a2055)
 2019-10-30 21:00:55 -07:00
Eric Badger
f5a3a9e138 YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 
(cherry picked from commit eef34f2d87a75e16b2cca870d99a5e1e28c31d9b)
 2019-10-25 17:17:14 +00:00
Zhankun Tang
7175fc91d0 YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi 2019-10-24 10:07:40 +08:00
Eric E Payne
d8b4348036 YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph. 
(cherry picked from commit 83d148074f9299de02d5c896a3ed4e11292cba73)
 2019-10-21 21:14:23 +00:00
Eric E Payne
889255576b YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R. 
(cherry picked from commit a5034c7988b6bf54bb7dab208100a2d205e3929e)
 2019-10-16 21:23:38 +00:00
Haibo Chen
a70c6e9665 YARN-8842. Expose metrics for custom resource types in QueueMetrics. (Contributed by Szilard Nemeth) 
(cherry picked from commit 84e22a6af46db2859d7d2caf192861cae9b6a1a8)
 2019-10-15 22:20:56 +00:00
Haibo Chen
4ec409fe09 YARN-8750. Refactor TestQueueMetrics. (Contributed by Szilard Nemeth) 
(cherry picked from commit e60b797c88541f94cecc7fdbcaad010c4742cfdb)
 2019-10-15 15:41:20 +00:00
Szilard Nemeth
c44902212a YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal 2019-10-15 13:25:57 +02:00
Szilard Nemeth
aabc18a777 Revert "YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl" 
This reverts commit 73bc8ef9b86cb9b3ee9bbc1786b49600e0ee600d.
 2019-10-09 19:59:02 +02:00
Szilard Nemeth
57e88a63cf YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko. 2019-10-09 14:19:56 +02:00
Szilard Nemeth
73bc8ef9b8 YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl 
(cherry picked from commit 6f1ab9516869dbfdee034f4a17bccc379a39fc31)
(cherry picked from commit 42177e8b786efd7fa1365b68617382de022a5146)
 2019-10-09 13:28:44 +02:00
Szilard Nemeth
d1415b66b7 YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl 
(cherry picked from commit 35f093f5b3f310af49a8bb476e62173862f5af51)
(cherry picked from commit 0ddb48a3038d9c5132efd9f48110f9d23c53ac67)
 2019-10-09 13:13:51 +02:00