81da221c75
YARN-10388. RMNode updatedCapability flag not set while RecommissionNodeTransition. Contributed by Pranjal Protim Borah
2020-08-07 08:36:52 +05:30
c7e71a6c0b
YARN-10361. Make custom DAO classes configurable into RMWebApp#JAXBContextResolver.
...
Contributed by Bilwa ST.
2020-08-06 13:19:52 +05:30
5edd8b925e
YARN-4575. ApplicationResourceUsageReport should return ALL reserved resource.
...
Contributed by Bibin Chundatt and Eric Payne.
2020-08-05 18:47:49 +00:00
0277856738
YARN-10377. Fix filter index to show apps while clicking on queue in RM UI
...
Contributed by Tarun Parimi.
2020-08-04 23:10:45 +05:30
aa5afa72c5
YARN-10381. Add application attempt state in AppAttempts RM REST API
...
Contributed by Siddharth Ahuja. Reviewed by Bilwa ST.
2020-08-04 23:07:28 +05:30
5d8600e80a
YARN-10369. Make NMTokenSecretManagerInRM sending NMToken for nodeId DEBUG. Contributed by Jim Brennan.
2020-07-29 23:26:08 +05:30
3eaf62726f
YARN-10343. Legacy RM UI should include labeled metrics for allocated, total, and reserved resources. Contributed by Eric Payne
2020-07-28 13:43:19 -07:00
5dadf963d3
YARN-10208. Add capacityScheduler metric for NODE_UPDATE interval. Contributed by Pranjal Protim Borah.
2020-07-28 11:55:47 +05:30
247eb0979b
YARN-10319. Record Last N Scheduler Activities from ActivitiesManager
...
Reviewed by Tao Yang and Adam Antal.
2020-07-24 11:43:31 +05:30
bfcd775381
YARN-10315. Avoid sending RMNodeResourceupdate event if resource is same. Contributed by Sushil Ks.
2020-07-23 16:41:35 +05:30
6bcb24d269
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-18 14:33:43 +05:30
8980c23fc2
YARN-10350. TestUserGroupMappingPlacementRule fails
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-07-15 14:31:34 +09:00
48f90115b5
YARN-10348. Allow RM to always cancel tokens after app completes. Contributed by
...
Jim Brennan
2020-07-13 23:10:39 +00:00
0427100b75
YARN-10297. TestContinuousScheduling#testFairSchedulerContinuousSchedulingInitTime fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
2020-07-13 18:57:50 +00:00
f91a8ad88b
HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. ( #2085 )
2020-07-09 11:33:37 -07:00
3a4d05b850
YARN-8047. RMWebApp make external class pluggable.
...
Contributed by Bilwa S T.
2020-07-08 12:43:20 +05:30
2bbd00dff4
YARN-10337. Fix failing testcase TestRMHATimelineCollectors.
...
Contributed by Bilwa S T.
2020-07-07 17:10:59 +05:30
639acb6d89
HADOOP-17111. Replace Guava Optional with Java8+ Optional. Contributed by Ahmed Hussein.
2020-07-06 16:08:36 +09:00
04abd0eb17
YARN-10330. Add missing test scenarios to TestUserGroupMappingPlacementRule and TestAppNameMappingPlacementRule. Contributed by Peter Bacsko
2020-07-01 14:10:55 +02:00
e8dc862d38
YARN-9809. Added node manager health status to resource manager registration call.
...
Contributed by Eric Badger via eyang
2020-06-30 11:39:16 -07:00
74fc13cf91
YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan).
2020-06-29 18:39:53 +00:00
33d3df7ee2
YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth
2020-06-26 11:14:24 +02:00
6a8fd73b27
YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky
2020-06-25 17:27:58 +02:00
03f855e3e7
YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko
2020-06-23 12:13:04 +02:00
ce1008fe61
YARN-10321. Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios
...
Contributed by Szilard Nemeth. Reviewed by Gergely Pollak.
2020-06-22 12:49:14 +05:30
b2facc84a1
YARN-9460. QueueACLsManager and ReservationsACLManager should not use instanceof checks. Contributed by Bilwa S T.
2020-06-20 20:09:31 +05:30
469841446f
YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko
2020-06-19 14:50:24 +02:00
5b1a56f9f1
YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak
2020-06-17 14:34:40 +02:00
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:16:33 +05:30
7c4de59fc1
YARN-10293. Reserved Containers not allocated from available space of other nodes in CandidateNodeSet for MultiNodePlacement. Contributed by Prabhu Joseph.
2020-06-12 22:10:25 +08:00
56247db302
YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).
2020-06-09 18:43:16 +00:00
8abff5151a
YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke
2020-06-05 15:00:39 +02:00
8b146c17b3
YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak
2020-06-05 11:38:32 +02:00
b5efdea4fd
YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak
2020-06-02 18:29:15 +02:00
e0a0741ac8
YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori
2020-06-02 15:42:39 +02:00
593af878c0
YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
2020-05-27 13:05:52 +02:00
fb030f3239
Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak"
...
This reverts commit ac4540dd8e
2020-05-27 13:04:48 +02:00
c30c23cb66
YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R
2020-05-26 16:29:42 -07:00
ac4540dd8e
FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
2020-05-21 13:11:44 +02:00
6ce295b787
YARN-10259. Fix reservation logic in Multi Node Placement.
...
Reviewed by Wangda Tan.
2020-05-14 16:52:11 +05:30
450e5aa9dd
YARN-10154. Addendum Patch which fixes below bugs
...
1. RM fails to start when LeafQueueTemplate max capacity is not specified.
2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue
Capacity is set to NaN during RM startup with clusterResource is zero.
Reviewed by Sunil G and Manikandan R.
2020-05-13 14:35:37 +05:30
fff1d2c122
YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T
2020-05-12 10:48:54 -07:00
35010120fb
YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T.
2020-05-06 13:22:54 -07:00
92e3ebb401
YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-05-06 12:27:17 -05:00
cb6399c109
YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko
2020-05-05 16:00:43 +02:00
0debe55d6c
YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2020-05-05 13:11:28 +02:00
d125d39108
YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T.
2020-04-29 11:23:58 -07:00
410c605aec
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
2020-04-29 15:53:30 +02:00
6ef01646ba
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
2020-04-28 05:35:36 +09:00
f91e21ac10
YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
2020-04-27 08:26:11 +02:00
d4874585f4
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-24 17:04:47 +02:00
92b28e38b7
YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
2020-04-18 10:13:37 +02:00
76900b4f5b
YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
2020-04-18 09:57:27 +02:00
3601054822
YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
2020-04-17 10:34:44 -07:00
2fe122e322
YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R.
2020-04-17 18:51:19 +05:30
4db598e0e6
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-15 13:18:44 +02:00
55fcbcb5c2
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
2020-04-15 07:24:04 +02:00
23481ad378
YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T
2020-04-09 11:06:11 -07:00
6fb29d0f17
YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.
2020-04-09 18:51:42 +05:30
45362a9f4c
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
2020-04-07 15:51:55 +02:00
8914cf9167
Preparing for 3.4.0 development
2020-03-29 23:24:25 +05:30
3d5ade1839
YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R
2020-03-26 13:23:43 +01:00
9a297ff31d
YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko
2020-03-26 12:35:10 +01:00
348685dcb9
YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke
2020-03-26 12:18:15 +01:00
6ce189c621
YARN-10200. Add number of containers to RMAppManager summary
2020-03-25 10:41:59 -07:00
cdb2107066
YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.
2020-03-25 16:50:19 +05:30
d312991ead
YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue
...
Contributed by Peter Bacsko. Reviewed by Manikandan R.
2020-03-23 18:01:14 +05:30
f2d3ac2a3f
YARN-10034. Remove Allocation Tags from released container from Decommission node
...
Contributed by Kyungwan Nam. Reviewed by Adam Antal.
2020-03-19 12:55:14 +05:30
b9d825f178
YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.
2020-03-13 14:39:36 -07:00
5ead9c15ca
YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori
2020-03-12 12:29:03 +01:00
38d87883b6
YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko
2020-03-12 12:18:37 +01:00
0fd8bf5f6b
YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.
2020-03-12 15:03:12 +05:30
cf9cf83a43
YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori
2020-03-10 16:44:48 +01:00
61f4cf3055
YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke
2020-03-10 16:35:04 +01:00
9314ef947f
YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko
2020-03-10 16:07:46 +01:00
ea0444851d
YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke
2020-03-06 17:49:45 +01:00
004e955348
YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko
2020-03-05 09:56:42 +01:00
c0d084247c
YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko
2020-03-03 09:46:32 +01:00
e9eecedf69
YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton
2020-03-02 16:25:14 +01:00
b420ddeada
YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.
2020-02-27 12:53:20 -08:00
10461e0193
YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton
2020-02-27 20:10:32 +01:00
d68616b045
YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal
2020-02-25 21:30:04 +01:00
72fa9c7f1b
YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko
2020-02-24 21:54:07 +01:00
34cf63c7d0
YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko
2020-02-24 21:45:49 +01:00
6526f95bd2
YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph.
2020-02-19 11:17:22 +05:30
e637797211
YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter
...
Contributed by Peter Bacsko.
2020-02-11 22:01:58 +05:30
28f730b317
YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph
2020-02-09 21:14:53 +05:30
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
314e2f9d2e
YARN-10116. Expose diagnostics in RMAppManager summary
2020-02-04 17:44:05 -08:00
a7d72c523a
YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko
2020-01-30 16:03:38 +01:00
e578e52aae
YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne.
2020-01-29 03:54:43 +00:00
5abd0148eb
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-28 10:06:37 -08:00
1ab9c692fa
YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change
...
Contributed by Kinga Marton.
2020-01-28 23:16:04 +05:30
94f060242e
YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko
2020-01-28 15:36:04 +01:00
ca29768035
YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko
2020-01-28 15:22:48 +01:00
9520b2ad79
YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal
2020-01-22 16:25:07 +01:00
b4870bce3a
Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R."
...
This reverts commit 0696828a09
2020-01-21 17:45:17 -08:00
0696828a09
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-21 13:41:01 -08:00
581072a8f0
YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg
2020-01-20 13:10:32 +01:00
8b3ee2f7e9
YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph
2020-01-20 09:54:22 +01:00
bibinchundatt
Prabhu Joseph
Eric Badger
Jonathan Hung
Ayush Saxena
Bilwa S T
Eric E Payne
Xiaoyu Yao
Akira Ajisaka
Szilard Nemeth
Eric Yang
Surendra Singh Lilhore
Vinayakumar B
Tao Yang
Prabhu Joseph
Inigo Goiri
Ahmed Hussein
Sunil G
Brahma Reddy Battula
Weiwei Yang