Eric E Payne
74fc13cf91
YARN-9903: Support reservations continue looking for Node Labels. Contributed by Jim Brennan (Jim_Brennan).
2020-06-29 18:39:53 +00:00
Szilard Nemeth
33d3df7ee2
YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy. Contributed by Szilard Nemeth
2020-06-26 11:14:24 +02:00
Szilard Nemeth
6a8fd73b27
YARN-10279. Avoid unnecessary QueueMappingEntity creations. Contributed by Marton Hudaky
2020-06-25 17:27:58 +02:00
Szilard Nemeth
03f855e3e7
YARN-10316. FS-CS converter: convert maxAppsDefault, maxRunningApps settings. Contributed by Peter Bacsko
2020-06-23 12:13:04 +02:00
Prabhu Joseph
ce1008fe61
YARN-10321. Break down TestUserGroupMappingPlacementRule#testMapping into test scenarios
...
Contributed by Szilard Nemeth. Reviewed by Gergely Pollak.
2020-06-22 12:49:14 +05:30
Surendra Singh Lilhore
b2facc84a1
YARN-9460. QueueACLsManager and ReservationsACLManager should not use instanceof checks. Contributed by Bilwa S T.
2020-06-20 20:09:31 +05:30
Szilard Nemeth
469841446f
YARN-9930. Support max running app logic for CapacityScheduler. Contributed by Peter Bacsko
2020-06-19 14:50:24 +02:00
Szilard Nemeth
5b1a56f9f1
YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak
2020-06-17 14:34:40 +02:00
Vinayakumar B
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:16:33 +05:30
Tao Yang
7c4de59fc1
YARN-10293. Reserved Containers not allocated from available space of other nodes in CandidateNodeSet for MultiNodePlacement. Contributed by Prabhu Joseph.
2020-06-12 22:10:25 +08:00
Eric E Payne
56247db302
YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).
2020-06-09 18:43:16 +00:00
Szilard Nemeth
8abff5151a
YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke
2020-06-05 15:00:39 +02:00
Szilard Nemeth
8b146c17b3
YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak
2020-06-05 11:38:32 +02:00
Szilard Nemeth
b5efdea4fd
YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak
2020-06-02 18:29:15 +02:00
Szilard Nemeth
e0a0741ac8
YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori
2020-06-02 15:42:39 +02:00
Szilard Nemeth
593af878c0
YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
2020-05-27 13:05:52 +02:00
Szilard Nemeth
fb030f3239
Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak"
...
This reverts commit ac4540dd8e
.
Reverting because of missing Jira-ID in the commit message.
2020-05-27 13:04:48 +02:00
Jonathan Hung
c30c23cb66
YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R
2020-05-26 16:29:42 -07:00
Szilard Nemeth
ac4540dd8e
FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
2020-05-21 13:11:44 +02:00
Prabhu Joseph
6ce295b787
YARN-10259. Fix reservation logic in Multi Node Placement.
...
Reviewed by Wangda Tan.
2020-05-14 16:52:11 +05:30
Prabhu Joseph
450e5aa9dd
YARN-10154. Addendum Patch which fixes below bugs
...
1. RM fails to start when LeafQueueTemplate max capacity is not specified.
2. Job stuck in ACCEPTED state with DominantResourceCalculator as Queue
Capacity is set to NaN during RM startup with clusterResource is zero.
Reviewed by Sunil G and Manikandan R.
2020-05-13 14:35:37 +05:30
Jonathan Hung
fff1d2c122
YARN-10260. Allow transitioning queue from DRAINING to RUNNING state. Contributed by Bilwa S T
2020-05-12 10:48:54 -07:00
Inigo Goiri
35010120fb
YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T.
2020-05-06 13:22:54 -07:00
Ahmed Hussein
92e3ebb401
YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-05-06 12:27:17 -05:00
Szilard Nemeth
cb6399c109
YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko
2020-05-05 16:00:43 +02:00
Szilard Nemeth
0debe55d6c
YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2020-05-05 13:11:28 +02:00
Inigo Goiri
d125d39108
YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T.
2020-04-29 11:23:58 -07:00
Szilard Nemeth
410c605aec
YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G
2020-04-29 15:53:30 +02:00
Akira Ajisaka
6ef01646ba
YARN-9848. Revert YARN-4946. Contributed by Steven Rand.
2020-04-28 05:35:36 +09:00
Szilard Nemeth
f91e21ac10
YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph
2020-04-27 08:26:11 +02:00
Szilard Nemeth
d4874585f4
YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph
2020-04-24 17:04:47 +02:00
Szilard Nemeth
92b28e38b7
YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja
2020-04-18 10:13:37 +02:00
Szilard Nemeth
76900b4f5b
YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke
2020-04-18 09:57:27 +02:00
Jonathan Hung
3601054822
YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T
2020-04-17 10:34:44 -07:00
Sunil G
2fe122e322
YARN-10154. CS Dynamic Queues cannot be configured with absolute resources. Contributed by Manikandan R.
2020-04-17 18:51:19 +05:30
Szilard Nemeth
4db598e0e6
YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke
2020-04-15 13:18:44 +02:00
Szilard Nemeth
55fcbcb5c2
YARN-10234. FS-CS converter: don't enable auto-create queue property for root. Contributed by Peter Bacsko
2020-04-15 07:24:04 +02:00
Jonathan Hung
23481ad378
YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T
2020-04-09 11:06:11 -07:00
Sunil G
6fb29d0f17
YARN-10226. NPE in Capacity Scheduler while using %primary_group queue mapping. Contributed by Peter Bacsko.
2020-04-09 18:51:42 +05:30
Szilard Nemeth
45362a9f4c
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
2020-04-07 15:51:55 +02:00
Brahma Reddy Battula
8914cf9167
Preparing for 3.4.0 development
2020-03-29 23:24:25 +05:30
Szilard Nemeth
3d5ade1839
YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R
2020-03-26 13:23:43 +01:00
Szilard Nemeth
9a297ff31d
YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko
2020-03-26 12:35:10 +01:00
Szilard Nemeth
348685dcb9
YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke
2020-03-26 12:18:15 +01:00
Jonathan Hung
6ce189c621
YARN-10200. Add number of containers to RMAppManager summary
2020-03-25 10:41:59 -07:00
Sunil G
cdb2107066
YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.
2020-03-25 16:50:19 +05:30
Prabhu Joseph
d312991ead
YARN-10198. Remove Validation of Queue Exist under Managed Parent Queue
...
Contributed by Peter Bacsko. Reviewed by Manikandan R.
2020-03-23 18:01:14 +05:30
Prabhu Joseph
f2d3ac2a3f
YARN-10034. Remove Allocation Tags from released container from Decommission node
...
Contributed by Kyungwan Nam. Reviewed by Adam Antal.
2020-03-19 12:55:14 +05:30
Weiwei Yang
b9d825f178
YARN-9567. Add diagnostics for outstanding resource requests on app attempts page. Contributed by Tao Yang.
2020-03-13 14:39:36 -07:00
Szilard Nemeth
5ead9c15ca
YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori
2020-03-12 12:29:03 +01:00
Szilard Nemeth
38d87883b6
YARN-10193. FS-CS converter: fix incorrect capacity conversion. Contributed by Peter Bacsko
2020-03-12 12:18:37 +01:00
Sunil G
0fd8bf5f6b
YARN-10191. FS-CS converter: call System.exit function call for every code path in main method. Contributed by Peter Bacsko.
2020-03-12 15:03:12 +05:30
Szilard Nemeth
cf9cf83a43
YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori
2020-03-10 16:44:48 +01:00
Szilard Nemeth
61f4cf3055
YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke
2020-03-10 16:35:04 +01:00
Szilard Nemeth
9314ef947f
YARN-10168. FS-CS Converter: tool doesn't handle min/max resource conversion correctly. Contributed by Peter Bacsko
2020-03-10 16:07:46 +01:00
Szilard Nemeth
ea0444851d
YARN.10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke
2020-03-06 17:49:45 +01:00
Szilard Nemeth
004e955348
YARN-10167. FS-CS Converter: Need to validate c-s.xml after converting. Contributed by Peter Bacsko
2020-03-05 09:56:42 +01:00
Szilard Nemeth
c0d084247c
YARN-10175. FS-CS converter: only convert placement rules if a cmd line switch is defined. Contributed by Peter Bacsko
2020-03-03 09:46:32 +01:00
Szilard Nemeth
e9eecedf69
YARN-10148. addendum: Fix method call parameter order of setAdminAndSubmitACL in TestCapacitySchedulerQueueACLs. Contributed by Kinga Marton
2020-03-02 16:25:14 +01:00
Inigo Goiri
b420ddeada
YARN-10155. TestDelegationTokenRenewer.testTokenThreadTimeout fails in trunk. Contributed by Manikandan R.
2020-02-27 12:53:20 -08:00
Szilard Nemeth
10461e0193
YARN-10148. Add Unit test for queue ACL for both FS and CS. Contributed by Kinga Marton
2020-02-27 20:10:32 +01:00
Szilard Nemeth
d68616b045
YARN-10130. FS-CS converter: Do not allow output dir to be the same as input dir. Contributed by Adam Antal
2020-02-25 21:30:04 +01:00
Szilard Nemeth
72fa9c7f1b
YARN-10135. FS-CS converter tool: issue warning on dynamic auto-create mapping rules. Contributed by Peter Bacsko
2020-02-24 21:54:07 +01:00
Szilard Nemeth
34cf63c7d0
YARN-10157. FS-CS converter: initPropertyActions() is not called without rules file. Contributed by Peter Bacsko
2020-02-24 21:45:49 +01:00
Sunil G
6526f95bd2
YARN-10139. ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB. Contributed by Prabhu Joseph.
2020-02-19 11:17:22 +05:30
Prabhu Joseph
e637797211
YARN-10127. Remove setting App Ordering Policy to ParentQueue in FSQueueConverter
...
Contributed by Peter Bacsko.
2020-02-11 22:01:58 +05:30
Sunil G
28f730b317
YARN-10109. Allow stop and convert from leaf to parent queue in a single Mutation API call. Contributed by Prabhu Joseph
2020-02-09 21:14:53 +05:30
Vinayakumar B
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
Jonathan Hung
314e2f9d2e
YARN-10116. Expose diagnostics in RMAppManager summary
2020-02-04 17:44:05 -08:00
Szilard Nemeth
a7d72c523a
YARN-10099. FS-CS converter: handle allow-undeclared-pools and user-as-default-queue properly and fix misc issues. Contributed by Peter Bacsko
2020-01-30 16:03:38 +01:00
Eric Badger
e578e52aae
YARN-10084. Allow inheritance of max app lifetime / default app lifetime. Contributed by Eric Payne.
2020-01-29 03:54:43 +00:00
Inigo Goiri
5abd0148eb
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-28 10:06:37 -08:00
Prabhu Joseph
1ab9c692fa
YARN-10022. RM Rest API to validate the CapacityScheduler Configuration change
...
Contributed by Kinga Marton.
2020-01-28 23:16:04 +05:30
Szilard Nemeth
94f060242e
YARN-10104. FS-CS converter: dry run should work without output defined. Contributed by Peter Bacsko
2020-01-28 15:36:04 +01:00
Szilard Nemeth
ca29768035
YARN-10085. FS-CS converter: remove mixed ordering policy check. Contributed by Peter Bacsko
2020-01-28 15:22:48 +01:00
Szilard Nemeth
9520b2ad79
YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal
2020-01-22 16:25:07 +01:00
Inigo Goiri
b4870bce3a
Revert "YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R."
...
This reverts commit 0696828a09
.
2020-01-21 17:45:17 -08:00
Inigo Goiri
0696828a09
YARN-9768. RM Renew Delegation token thread should timeout and retry. Contributed by Manikandan R.
2020-01-21 13:41:01 -08:00
Szilard Nemeth
581072a8f0
YARN-7913. Improve error handling when application recovery fails with exception. Contributed by Wilfred Spiegelenburg
2020-01-20 13:10:32 +01:00
Szilard Nemeth
8b3ee2f7e9
YARN-9462. TestResourceTrackerService.testNodeRemovalGracefully fails sporadically. Contributed by Prabhu Joseph
2020-01-20 09:54:22 +01:00
Prabhu Joseph
a0ff42d761
YARN-10070. Fix NPE if no queue mapping defined for proxy user when using application tag based placement.
...
Contributed by Kinga Marton. Reviewed by Adam Antal.
2020-01-16 12:55:27 +05:30
Szilard Nemeth
2aa065d98f
YARN-10082. FS-CS converter: disable terminal placement rule checking. Contributed by Peter Bacsko
2020-01-15 12:42:13 +01:00
Szilard Nemeth
7c5cecc3b3
YARN-9970. Refactor TestUserGroupMappingPlacementRule#verifyQueueMapping. Contributed by Manikandan R
2020-01-15 09:55:41 +01:00
Szilard Nemeth
1c51f36be7
YARN-9788. Queue Management API does not support parallel updates. Contributed by Prabhu Joseph
2020-01-14 12:26:03 +01:00
Szilard Nemeth
621c5eac38
YARN-9912. Capacity scheduler: support u:user2:%secondary_group queue mapping. Contributed by Manikandan R
2020-01-13 17:23:00 +01:00
Szilard Nemeth
b7ef8a333f
YARN-9868. Validate %primary_group queue in CS queue manager. Contributed by Manikandan R
2020-01-13 17:15:09 +01:00
Szilard Nemeth
d842dfffa5
YARN-9866. u:user2:%primary_group is not working as expected. Contributed by Manikandan R
2020-01-12 14:04:15 +01:00
Szilard Nemeth
24e6a9e43a
YARN-10067. Add dry-run feature to FS-CS converter tool. Contributed by Peter Bacsko
2020-01-12 13:48:39 +01:00
Eric E Payne
b1e07d27cc
YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
2020-01-08 19:26:01 +00:00
Eric E Payne
6899be5a17
YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan)
2020-01-08 17:29:56 +00:00
Szilard Nemeth
768ee22e9e
YARN-10035. Add ability to filter the Cluster Applications API request by name. Contributed by Adam Antal
2020-01-06 16:26:33 +01:00
Prabhu Joseph
eca7e14c2f
YARN-10053. Use Shared Group Mapping Service in Placement Rules.
...
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:13:57 +05:30
Prabhu Joseph
21ada4d1b0
Revert "YARN-10053. Use Shared Group Mapping Service in Placement Rules."
...
This reverts commit 217b56ffdd
.
2020-01-02 14:12:43 +05:30
Prabhu Joseph
217b56ffdd
YARN-10053. Use Shared Group Mapping Service in Placement Rules.
...
Contributed by Wilfred Spiegelenburg.
2020-01-02 14:07:49 +05:30
Eric Badger
412035b47a
YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne.
2019-12-20 18:07:00 +00:00
Giovanni Matteo Fumarola
ef59ffd362
YARN-10038. [UI] Finish Time is not correctly parsed in the RM Apps page. Contributed by Inigo Goiri.
2019-12-19 11:37:17 -08:00
Jonathan Hung
7b93575b92
YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne
2019-12-18 13:18:11 -08:00
Jonathan Hung
fddc3d55c3
YARN-10039. Allow disabling app submission from REST endpoints
2019-12-18 10:48:05 -08:00
Eric Badger
f47dcf2d4c
YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne.
2019-12-17 17:11:23 +00:00
prabhujoseph
d4bde134e3
YARN-7721. Fix TestContinuousScheduling fails sporadically with NPE.
...
Contributed by Wilfred Spiegelenburg.
2019-12-10 15:00:23 +05:30
Jonathan Hung
6d970d8100
YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R
2019-12-08 16:35:02 -08:00
Jonathan Hung
cd4779f317
Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources"
...
This reverts commit 92bce918dc
.
2019-12-08 16:34:46 -08:00
Jonathan Hung
92bce918dc
YARN-10012. Guaranteed and max capacity queue metrics for custom resources
2019-12-08 15:52:17 -08:00
prabhujoseph
e575df5ca6
YARN-10006. IOException used in place of YARNException in CapaitySheduler
...
Contributed by Adam Antal. Reviewed by Szilard Nemeth.
2019-12-06 12:51:28 +05:30
Szilard Nemeth
cf68857631
YARN-9938. Validate Parent Queue for QueueMapping contains dynamic group as parent queue. Contributed by Manikandan R
2019-12-05 21:04:42 +01:00
Szilard Nemeth
5cc6f945da
YARN-9789. Disable Option for Write Ahead Logs of LogMutation. Contributed by Prabhu Joseph
2019-12-05 20:43:26 +01:00
Szilard Nemeth
4627dd6708
YARN-9780. SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call. Contributed by Prabhu Joseph
2019-12-05 20:38:37 +01:00
Szilard Nemeth
c71befaf8f
YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph
2019-12-05 20:31:24 +01:00
Szilard Nemeth
520fe2c99b
YARN-5106. Provide a builder interface for FairScheduler allocations for use in tests. Contributed by Adam Antal
2019-12-05 17:37:40 +01:00
Sunil G
682e6fdeda
YARN-9052. Replace all MockRM submit method definitions with a builder. Contributed by Szilard Nemeth.
2019-12-05 13:26:23 +05:30
Szilard Nemeth
ef950b0863
YARN-9290. Invalid SchedulingRequest not rejected in Scheduler PlacementConstraintsHandler. Contributed by Prabhu Joseph
2019-11-26 22:04:07 +01:00
prabhujoseph
aa7ab2719f
YARN-9991. Fix Application Tag prefix to userid. Contributed by Szilard Nemeth.
2019-11-27 01:59:12 +05:30
Szilard Nemeth
8c9018d5c7
YARN-9899. Migration tool that help to generate CS config based on FS config [Phase 2]. Contributed by Peter Bacsko
2019-11-26 21:22:35 +01:00
Szilard Nemeth
448ffb12ec
YARN-9937. addendum: Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph
2019-11-26 13:41:41 +01:00
HUAN-PING SU
f8e36e03b4
YARN-9966. Code duplication in UserGroupMappingPlacementRule ( #1709 )
2019-11-25 15:28:53 +09:00
Tao Yang
8afabe41e4
YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu.
2019-11-22 16:06:14 +08:00
Szilard Nemeth
dfdc6d6dd9
YARN-9886. Queue mapping based on userid passed through application tag. Contributed by Julia Kinga Marton
2019-11-19 21:14:15 +01:00
Prabhu Joseph
30b2096933
YARN-9986. Allow app owner to signal container via RMWebServices. Contributed by kyungwan nam.
2019-11-19 11:35:00 +00:00
Sunil G
ea68756c0c
YARN-8373. RM Received RMFatalEvent of type CRITICAL_THREAD_CRASH. Contributed by Wilfred Spiegelenburg.
2019-11-19 14:10:41 +05:30
Sunil G
215f2052fc
YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg.
2019-11-19 14:03:02 +05:30
Szilard Nemeth
27642367ef
YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko
2019-11-18 16:29:39 +01:00
Yufei Gu
b83b9ab418
YARN-9537. Add configuration to disable AM preemption. Contributed by ZhouKang
2019-11-12 09:40:16 -08:00
Abhishek Modi
fb512f5087
YARN-9697. Efficient allocation of Opportunistic containers. Contributed by Abhishek Modi.
2019-11-12 16:34:04 +05:30
Szilard Nemeth
30b93f914b
YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R
2019-11-11 13:28:02 +01:00
Sunil G
51e7d1b37e
YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph.
2019-11-04 23:27:49 +05:30
Sunil G
d462308e04
YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph.
2019-11-03 08:47:27 +05:30
Sunil G
9a2e43e29e
YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph.
2019-10-31 00:34:27 +05:30
Szilard Nemeth
e3e7daa4f5
YARN-9889. [UI] Add Application Tag column to RM All Applications table. Contributed by Kinga Marton
2019-10-30 16:26:56 +01:00
Abhishek Modi
ed45c13f67
YARN-2442. ResourceManager JMX UI does not give HA State. Contributed by Rohith Sharma and Cyrus Jackson.
2019-10-29 15:54:55 +05:30
Eric Yang
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
Zhankun Tang
fd84ca5161
YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi
2019-10-24 10:05:00 +08:00
Eric E Payne
83d148074f
YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph.
2019-10-21 20:30:50 +00:00
Eric Yang
3f7756dc6c
YARN-9922. Fixed YARN javadoc errors from YARN-9699.
...
Contributed by Peter Bacsko
2019-10-21 16:16:49 -04:00
Eric Yang
155864da00
YARN-9875. Improve fair scheduler configuration store on HDFS.
...
Contributed by Prabhu Joseph
2019-10-18 20:30:11 -04:00
Mate Szalay-Beko
6d92aa7c30
HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 ( #1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko
...
* HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5
- Add a static initializer for the unit tests using ZooKeeper to enable
the four-letter-words diagnostic telnet commands. (this is an interface
that become disabled by default, so to keep the ZooKeeper 3.4.x behavior
we enabled it for the tests)
- Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt
records. The new ZooKeeper seems to respond quicker during the fail-over
tests than the ZooKeeper, so we made sure to catch all the relevant records
by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt().
Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>
2019-10-18 13:26:20 -07:00
Szilard Nemeth
f0699a7406
YARN-9841. Capacity scheduler: add support for combined %user + %primary_group mapping. Contributed by Manikandan R
2019-10-18 11:19:49 +02:00
Sunil G
3990ffa083
YARN-9909. Offline format of YarnConfigurationStore. Contributed by Prabhu Joseph.
2019-10-17 06:43:06 +05:30
Eric E Payne
a5034c7988
YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R.
2019-10-16 21:10:08 +00:00
Sunil G
090f73a9aa
YARN-9900. Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format. Contributed by Prabhu Joseph.
2019-10-16 18:14:31 +05:30
Szilard Nemeth
a8ef03e961
YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R
2019-10-15 17:22:52 +02:00
Szilard Nemeth
5cc7873a47
YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko
2019-10-14 17:19:39 +02:00
Szilard Nemeth
ec86f42e40
YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal
2019-10-11 14:01:19 +02:00
Sunil G
be901f4962
YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph
2019-10-09 15:53:14 +05:30
Jonathan Hung
5462d2176f
YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena
2019-10-08 11:20:13 -07:00
Sunil G
fb1ecff6a2
Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph"
...
This reverts commit 4510970e2f
.
2019-10-05 09:15:17 +05:30
Sunil G
4510970e2f
YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph
2019-10-04 21:49:07 +05:30
Abhishek Modi
61a8436004
YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi.
2019-10-02 17:55:07 +05:30
bibinchundatt
425a6c89ae
YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.
2019-10-01 16:02:26 +05:30
Sunil G
137546a78a
YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph
2019-10-01 09:08:41 +05:30
Abhishek Modi
4d3c580b03
YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi.
2019-09-30 23:40:15 +05:30
Eric Badger
18a8c2404e
YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein
2019-09-26 16:15:08 +00:00
Jonathan Hung
606e341c1a
Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-25 17:54:53 -07:00
Jonathan Hung
c2731d4b63
YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-24 13:51:32 -07:00
Duo Zhang
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes ( #1496 )
...
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2019-09-24 08:50:36 +08:00
Jonathan Hung
3d78b1223d
YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar
2019-09-23 13:12:16 -07:00
Abhishek Modi
da2b4fe204
YARN-9794. RM crashes due to runtime errors in TimelineServiceV2Publisher. Contributed by Tarun Parimi.
2019-09-15 14:16:18 +05:30
Abhishek Modi
3b06f0bf9e
YARN-9819. Make TestOpportunisticContainerAllocatorAMService more resilient. Contribued by Abhishek Modi
2019-09-12 08:09:57 +05:30
Eric Badger
56b7571131
YARN-9815 ReservationACLsTestBase fails with NPE. Contributed by Ahmed Hussein
2019-09-11 21:59:28 +00:00
Jonathan Hung
f8f8598ea5
YARN-9824. Fall back to configured queue ordering policy class name
2019-09-10 15:19:07 -07:00
Eric Yang
10144a580e
YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API.
...
Contributed by Prabhu Joseph
2019-09-10 17:04:39 -04:00
Jonathan Hung
387c332b64
YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph
2019-09-09 00:23:04 -07:00
Jonathan Hung
43e389b980
YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar
2019-09-08 19:07:11 -07:00
Abhishek Modi
162af6f554
YARN-9784. org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue is flaky. Contributed by Julia Kinga Marton.
2019-09-07 18:08:04 +05:30
Wangda Tan
34b82e6da0
YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda)
...
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
2019-09-06 19:18:13 -07:00
Tao Yang
73575701ab
YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li.
2019-09-07 07:52:39 +08:00
Jonathan Hung
b71a7f195c
YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar
2019-09-06 11:00:07 -07:00
Jonathan Hung
e7d44e48f7
YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar
2019-09-06 10:17:07 -07:00
Jonathan Hung
0ccf4b0fe1
YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta
2019-09-05 14:01:42 -07:00
bibinchundatt
03489124ea
YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T.
2019-09-03 11:53:34 +05:30
Abhishek Modi
751b5a1ac8
YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph.
2019-09-01 22:40:40 +05:30
Abhishek Modi
c7ef4fb74a
YARN-8678. Queue Management API - rephrase error messages. Contributed by Prabhu Joseph.
2019-09-01 10:00:04 +05:30
Abhishek Modi
d2d963f3d4
YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam.
2019-09-01 09:54:46 +05:30
Abhishek Modi
7d998cb627
YARN-9800. TestRMDelegationTokens can fail in testRemoveExpiredMasterKeyInRMStateStore. Contributed by Adam Antal.
2019-08-31 10:03:09 +05:30
Abhishek Modi
eeccd2f6f6
YARN-9798. ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently. Contributed by Tao Yang.
2019-08-31 09:57:45 +05:30
Abhishek Modi
c929b383f8
YARN-9540. TestRMAppTransitions fails intermittently. Contributed by Tao Yang.
2019-08-30 17:17:43 +05:30
Rohith Sharma K S
5b557de721
YARN-9714. ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby. Contributed by Tao Yang.
2019-08-30 10:35:09 +05:30
Weiwei Yang
8c0759d02a
YARN-9664. Improve response of scheduler/app activities for better understanding. Contributed by Tao Yang.
2019-08-29 18:14:39 +08:00
Eric E Payne
6f2226a013
YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).
2019-08-28 20:40:34 +00:00
Jonathan Hung
8ef46595da
YARN-9438. launchTime not written to state store for running applications
2019-08-27 15:53:43 -07:00
Eric Yang
717c853873
YARN-9755. Fixed RM failing to start when FileSystemBasedConfigurationProvider is configured.
...
Contributed by Prabhu Joseph
2019-08-27 13:14:59 -04:00
Rohith Sharma K S
d70f5231a7
YARN-9640. Slow event processing could cause too many attempt unregister events. Contributed by Bibin A Chundatt.
2019-08-27 08:38:12 +05:30
Jonathan Hung
8660e48ca1
YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph
2019-08-26 15:50:33 -07:00
bibinchundatt
d3ce53e507
YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt.
2019-08-26 23:21:33 +05:30
Rohith Sharma K S
689d2e6105
YARN-8917. Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource. Contributed by Tao Yang.
2019-08-26 21:06:15 +05:30
Szilard Nemeth
7ab88dbfa6
YARN-7291. Better input parsing for resource in allocation file. Contributed by Zoltan Siegl
2019-08-21 17:01:18 +02:00
Sunil G
0e0ddfaf24
YARN-2599. Standby RM should expose jmx endpoint. Contributed by Rohith Sharma K S.
2019-08-17 15:43:19 +05:30
Szilard Nemeth
9b8359bb08
YARN-9461. TestRMWebServicesDelegationTokenAuthentication.testCancelledDelegationToken fails with HTTP 400. Contributed by Peter Bacsko
2019-08-16 12:31:58 +02:00
Szilard Nemeth
4456ea67b9
YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko
2019-08-16 11:36:14 +02:00
Szilard Nemeth
1845a83cec
YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
2019-08-15 17:15:38 +02:00
Szilard Nemeth
e4b538bbda
YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
2019-08-12 15:15:43 +02:00
Abhishek Modi
13a5803ccf
YARN-9464. Support pending resource metrics in RM's RESTful API. Contributed by Prabhu Joseph.
2019-08-12 14:31:24 +05:30
Abhishek Modi
8fbf8b2eb0
YARN-9722. PlacementRule logs object ID in place of queue name. Contributed by Prabhu Joseph.
2019-08-12 10:44:46 +05:30
Abhishek Modi
a79564fed0
YARN-9732. yarn.system-metrics-publisher.enabled=false is not honored by RM. Contributed by KWON BYUNGCHANG.
2019-08-09 22:25:30 +05:30
Eric E Payne
3b38f2019e
YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
2019-08-08 12:37:50 +00:00
Eric E Payne
42683aef1a
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
2019-07-30 18:58:36 +00:00
bibinchundatt
7a93be0f60
YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
2019-07-16 14:03:22 +05:30
Szilard Nemeth
91ce09e706
YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko
2019-07-15 10:47:20 +02:00
Haibo Chen
9b54dd7186
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung)
2019-07-11 13:57:08 -07:00
Weiwei Yang
15d82fcb75
YARN-9658. Fix UT failures in TestLeafQueue. Contributed by Tao Yang.
2019-07-03 12:08:45 +08:00
Sunil G
e966edd025
YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
2019-07-02 12:18:16 +05:30
Weiwei Yang
cbae241320
YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang.
2019-06-28 23:24:53 +08:00
bibinchundatt
be80334cdf
YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
2019-06-27 09:59:44 +05:30
Weiwei Yang
83dcb9d87e
YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
2019-06-21 17:41:05 +08:00
Zhankun Tang
304a47e22c
YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
2019-06-17 17:09:56 +08:00
Weiwei Yang
970b0b0c02
YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
2019-06-13 10:44:47 +08:00
bibinchundatt
60c95e9b6a
YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
2019-06-11 23:11:49 +05:30
Suma Shivaprasad
9191e08f0a
YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
2019-06-10 14:33:24 -07:00
Weiwei Yang
0976392502
YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
2019-06-06 21:59:01 +08:00
Eric Yang
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
Weiwei Yang
bd2590d71b
YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
2019-06-03 22:59:02 +08:00
Ahmed Hussein
abf76ac371
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2019-05-29 11:24:08 -05:00
Weiwei Yang
544876fe12
YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
2019-05-29 16:34:48 +08:00
Akira Ajisaka
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
Akira Ajisaka
9078e28a24
YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
2019-05-28 15:52:39 +09:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
Weiwei Yang
9f056d905f
YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
2019-05-26 09:56:36 -04:00
Giovanni Matteo Fumarola
55bd35921c
YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.
2019-05-15 11:50:46 -07:00
bibinchundatt
570fa2da20
YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
2019-05-15 13:30:09 +05:30
Giovanni Matteo Fumarola
29ff7fb140
YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji.
2019-05-13 10:57:12 -07:00
Weiwei Yang
1a47c2b7ae
YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang.
2019-05-12 22:31:39 -07:00
Weiwei Yang
90add05caa
YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang.
2019-05-09 21:54:09 +08:00
Akira Ajisaka
3172f6cbf9
YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola.
2019-05-08 14:56:23 +09:00
Weiwei Yang
c336af3847
YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang.
2019-05-08 09:54:16 +08:00
Giovanni Matteo Fumarola
8ecbf61cca
YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal.
2019-05-07 10:59:02 -07:00
Weiwei Yang
12b7059ddc
YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang.
2019-05-06 20:00:15 +08:00
Sean Mackrory
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
Giovanni Matteo Fumarola
fec9bf4b0b
YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:42:56 -07:00
Giovanni Matteo Fumarola
4a0ba24959
YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:27:04 -07:00
Inigo Goiri
c504eee0c2
YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi.
2019-04-23 12:40:44 -07:00
Weiwei Yang
1c8046d67e
YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 14:21:13 +08:00
Inigo Goiri
96e3027e46
YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi.
2019-04-22 09:49:03 -07:00
Eric Yang
df76cdc895
YARN-6695. Fixed NPE in publishing appFinished events to ATSv2.
...
Contributed by Prabhu Joseph
2019-04-18 12:29:37 -04:00
Prabhu Joseph
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-18 15:40:37 +09:00
Weiwei Yang
7fa73fac26
YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang.
2019-04-16 00:12:43 +08:00
Inigo Goiri
7a68e7abd5
YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi.
2019-04-14 20:11:20 -07:00
Giovanni Matteo Fumarola
ed3747c1cc
YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi.
2019-04-11 11:49:19 -07:00
Weiwei Yang
8c1bba375b
YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
2019-04-10 22:51:28 +08:00
Igor Rudenko
32722d2661
YARN-9433. Remove unused constants in YARN resource manager
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-10 18:37:27 +09:00
Giovanni Matteo Fumarola
cfec455c45
YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri.
2019-04-09 10:59:43 -07:00
Weiwei Yang
fc05b0e70e
YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang.
2019-04-08 13:40:53 +08:00
Weiwei Yang
ec143cbf67
YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
2019-04-06 19:59:36 +08:00
Eric Yang
8d150067e2
YARN-9396. Fixed duplicated RM Container created event to ATS.
...
Contributed by Prabhu Joseph
2019-04-04 13:01:56 -04:00
Sunil G
002dcc4ebf
YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
2019-04-03 18:57:28 +05:30
Yufei Gu
2f752830ba
YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
2019-04-01 20:05:15 -07:00
Giovanni Matteo Fumarola
da7f8c244d
YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
2019-04-01 11:21:31 -07:00
Giovanni Matteo Fumarola
332cab5518
YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
2019-04-01 11:06:51 -07:00
yufei
5257f50abb
YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-25 22:47:24 -07:00
Giovanni Matteo Fumarola
509b20b292
YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
2019-03-22 11:45:39 -07:00
Zoltan Siegl
ce5eb9cb2e
YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322
...
(Contributed by Zoltan Siegl via Daniel Templeton)
Change-Id: I92d52c0ca630e71afb26b2b7587cbdbe79254a05
2019-03-22 12:28:34 +01:00
Giovanni Matteo Fumarola
548997d6c9
YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
2019-03-21 12:04:05 -07:00
Giovanni Matteo Fumarola
5d8bd0e5cb
YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-03-19 11:00:21 -07:00
Eric Yang
09eabda314
YARN-9364. Remove commons-logging dependency from YARN.
...
Contributed by Prabhu Joseph
2019-03-18 19:58:42 -04:00
Eric Yang
5f6e225166
YARN-9363. Replaced debug logging with SLF4J parameterized log message.
...
Contributed by Prabhu Joseph
2019-03-18 13:57:18 -04:00
Eric Yang
2064ca015d
YARN-9349. Changed logging to use slf4j api.
...
Contributed by Prabhu Joseph
2019-03-15 19:20:59 -04:00
Eric Yang
03f3c8aed2
YARN-4404. Corrected typo in javadoc.
...
Contributed by Yesha Vora
2019-03-15 18:04:04 -04:00
Eric Yang
39b4a37e02
YARN-9341. Fixed enentrant lock usage in YARN project.
...
Contributed by Prabhu Joseph
2019-03-07 16:47:45 -05:00
Vrushali C
491313ab84
YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
2019-03-06 21:47:29 -08:00
Yufei Gu
0aefe2846f
YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-04 23:49:07 -08:00
Prabhu Joseph
e40e2d6ad5
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 14:10:08 +09:00
bibinchundatt
15098df744
Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
...
This reverts commit a63c358b78
.
2019-03-04 16:57:31 +05:30
Szilard Nemeth
538bb4880d
YARN-9323. FSLeafQueue#computeMaxAMResource does not override zero values for custom resources
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Id844ccf09488f367c0c7de0a3b2d4aca1bba31cc
2019-02-27 19:59:48 -08:00
Szilard Nemeth
7b928f19a4
YARN-9322. Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: I14c12f1265999d62102f2ec5506d90015efeefe8
2019-02-27 19:43:50 -08:00
Weiwei Yang
1779fc57a1
YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
2019-02-28 09:56:29 +08:00
Weiwei Yang
8c30114b00
YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
2019-02-27 17:29:02 +08:00
Rohith Sharma K S
8eae260af5
YARN-9311. Fix TestRMRestart hangs. Contributed by Prabhu Joseph.
2019-02-27 10:28:16 +05:30
Giovanni Matteo Fumarola
95372657fc
YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena.
2019-02-25 11:45:37 -08:00
Weiwei Yang
3e1739d589
YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
2019-02-26 00:08:13 +08:00
Weiwei Yang
9cd5c5447f
YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
2019-02-24 22:42:27 +08:00
Weiwei Yang
50094d7fef
YARN-9300. Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler. Contributed by Tao Yang.
2019-02-24 22:17:29 +08:00
bibinchundatt
ed13cf8406
YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
2019-02-22 21:01:34 +05:30
Weiwei Yang
9c88695bcd
YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
2019-02-22 20:58:33 +08:00
Weiwei Yang
eedcc8e26e
YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
2019-02-21 17:21:21 +08:00
bibinchundatt
a63c358b78
YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph.
2019-02-21 13:29:01 +05:30
Akira Ajisaka
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
Giovanni Matteo Fumarola
7536488bbd
YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri.
2019-02-11 15:27:43 -08:00
Weiwei Yang
0a1637c750
YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
2019-02-11 15:53:50 +08:00
Rohith Sharma K S
7fa62e150c
YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi.
2019-02-07 09:58:53 +05:30
Sunil G
604b2489a9
YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
2019-02-04 12:48:28 +05:30
Weiwei Yang
28ad20a711
YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
2019-02-02 23:02:45 +08:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
Eric E Payne
04105bbfdb
YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
2019-01-29 17:22:24 +00:00
Jonathan Hung
6cace58e21
YARN-9222. Print launchTime in ApplicationSummary
2019-01-25 13:17:00 -08:00
Eric Yang
a33ef4fd31
YARN-8867. Added resource localization status to YARN service status call.
...
Contributed by Chandni Singh
2019-01-24 18:43:21 -05:00
Weiwei Yang
a4bd64e724
YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu.
2019-01-24 17:53:43 +08:00
Weiwei Yang
721d5c2a5f
YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
2019-01-23 17:59:36 +08:00
Weiwei Yang
e3e076dc78
YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
2019-01-23 14:14:49 +08:00
Weiwei Yang
d43df31751
YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
2019-01-22 10:23:32 +08:00
Weiwei Yang
abde1e1f58
YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
2019-01-21 16:54:14 +08:00
Wangda Tan
6d7eedfd28
YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
...
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
2019-01-17 14:20:10 -08:00
Akira Ajisaka
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo.
2019-01-15 12:59:59 +09:00
Jason Lowe
6a923464af
YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R
2019-01-08 16:54:05 -06:00
Weiwei Yang
0921b706f7
YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan.
2019-01-08 14:30:53 +08:00
Weiwei Yang
944cf87223
YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
2019-01-07 15:57:31 +08:00
Sunil G
f4906ac019
YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang.
2019-01-04 12:10:00 +05:30
Weiwei Yang
cfe89e6f96
YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie.
2019-01-03 23:56:28 +08:00
Akira Ajisaka
128f340f4f
YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR.
2018-12-28 17:43:47 +09:00
Akira Ajisaka
15fe1b34dd
YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR.
2018-12-28 17:29:22 +09:00
Erik Krogen
091ad974cd
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
Eric Yang
f82922dcfa
YARN-5168. Added exposed port information for Docker container.
...
Contributed by Xun Liu
2018-12-21 19:44:07 -05:00
Weiwei Yang
f659485ee8
YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang.
2018-12-21 10:56:42 +08:00