30b93f914b
YARN-9865. Capacity scheduler: add support for combined %user + %secondary_group mapping. Contributed by Manikandan R
2019-11-11 13:28:02 +01:00
51e7d1b37e
YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph.
2019-11-04 23:27:49 +05:30
d462308e04
YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. Contributed by Prabhu Joseph.
2019-11-03 08:47:27 +05:30
9a2e43e29e
YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph.
2019-10-31 00:34:27 +05:30
e3e7daa4f5
YARN-9889. [UI] Add Application Tag column to RM All Applications table. Contributed by Kinga Marton
2019-10-30 16:26:56 +01:00
ed45c13f67
YARN-2442. ResourceManager JMX UI does not give HA State. Contributed by Rohith Sharma and Cyrus Jackson.
2019-10-29 15:54:55 +05:30
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
fd84ca5161
YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi
2019-10-24 10:05:00 +08:00
83d148074f
YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph.
2019-10-21 20:30:50 +00:00
3f7756dc6c
YARN-9922. Fixed YARN javadoc errors from YARN-9699.
...
Contributed by Peter Bacsko
2019-10-21 16:16:49 -04:00
155864da00
YARN-9875. Improve fair scheduler configuration store on HDFS.
...
Contributed by Prabhu Joseph
2019-10-18 20:30:11 -04:00
6d92aa7c30
HADOOP-16579. Upgrade to Curator 4.2.0 and ZooKeeper 3.5.5 ( #1656 ). Contributed by Norbert Kalmár, Mate Szalay-Beko
...
* HADOOP-16579 - Upgrade to Apache Curator 4.2.0 and ZooKeeper 3.5.5
- Add a static initializer for the unit tests using ZooKeeper to enable
the four-letter-words diagnostic telnet commands. (this is an interface
that become disabled by default, so to keep the ZooKeeper 3.4.x behavior
we enabled it for the tests)
- Also fix ZKFailoverController to look for relevant fail-over ActiveAttempt
records. The new ZooKeeper seems to respond quicker during the fail-over
tests than the ZooKeeper, so we made sure to catch all the relevant records
by adding a new parameter to ZKFailoverontroller.waitForActiveAttempt().
Co-authored-by: Norbert Kalmár <nkalmar@cloudera.com>
2019-10-18 13:26:20 -07:00
f0699a7406
YARN-9841. Capacity scheduler: add support for combined %user + %primary_group mapping. Contributed by Manikandan R
2019-10-18 11:19:49 +02:00
3990ffa083
YARN-9909. Offline format of YarnConfigurationStore. Contributed by Prabhu Joseph.
2019-10-17 06:43:06 +05:30
a5034c7988
YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R.
2019-10-16 21:10:08 +00:00
090f73a9aa
YARN-9900. Revert to previous state when Invalid Config is applied and Refresh Support in SchedulerConfig Format. Contributed by Prabhu Joseph.
2019-10-16 18:14:31 +05:30
a8ef03e961
YARN-9840. Capacity scheduler: add support for Secondary Group rule mapping. Contributed by Manikandan R
2019-10-15 17:22:52 +02:00
5cc7873a47
YARN-9699. Migration tool that help to generate CS config based on FS config [Phase 1]. Contributed by Peter Bacsko
2019-10-14 17:19:39 +02:00
ec86f42e40
YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal
2019-10-11 14:01:19 +02:00
be901f4962
YARN-9873. Mutation API Config Change need to update Version Number. Contributed by Prabhu Joseph
2019-10-09 15:53:14 +05:30
5462d2176f
YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena
2019-10-08 11:20:13 -07:00
fb1ecff6a2
Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph"
...
This reverts commit 4510970e2f
2019-10-05 09:15:17 +05:30
4510970e2f
YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph
2019-10-04 21:49:07 +05:30
61a8436004
YARN-9870. Remove unused function from OpportunisticContainerAllocatorAMService. Contributed by Abhishek Modi.
2019-10-02 17:55:07 +05:30
425a6c89ae
YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.
2019-10-01 16:02:26 +05:30
137546a78a
YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph
2019-10-01 09:08:41 +05:30
4d3c580b03
YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi.
2019-09-30 23:40:15 +05:30
18a8c2404e
YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein
2019-09-26 16:15:08 +00:00
606e341c1a
Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-25 17:54:53 -07:00
c2731d4b63
YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-24 13:51:32 -07:00
bfe1dac29d
HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes ( #1496 )
...
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2019-09-24 08:50:36 +08:00
3d78b1223d
YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar
2019-09-23 13:12:16 -07:00
da2b4fe204
YARN-9794. RM crashes due to runtime errors in TimelineServiceV2Publisher. Contributed by Tarun Parimi.
2019-09-15 14:16:18 +05:30
3b06f0bf9e
YARN-9819. Make TestOpportunisticContainerAllocatorAMService more resilient. Contribued by Abhishek Modi
2019-09-12 08:09:57 +05:30
56b7571131
YARN-9815 ReservationACLsTestBase fails with NPE. Contributed by Ahmed Hussein
2019-09-11 21:59:28 +00:00
f8f8598ea5
YARN-9824. Fall back to configured queue ordering policy class name
2019-09-10 15:19:07 -07:00
10144a580e
YARN-9728. Bugfix for escaping illegal xml characters for Resource Manager REST API.
...
Contributed by Prabhu Joseph
2019-09-10 17:04:39 -04:00
387c332b64
YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph
2019-09-09 00:23:04 -07:00
43e389b980
YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar
2019-09-08 19:07:11 -07:00
162af6f554
YARN-9784. org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue is flaky. Contributed by Julia Kinga Marton.
2019-09-07 18:08:04 +05:30
34b82e6da0
YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda)
...
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
2019-09-06 19:18:13 -07:00
73575701ab
YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li.
2019-09-07 07:52:39 +08:00
b71a7f195c
YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar
2019-09-06 11:00:07 -07:00
e7d44e48f7
YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar
2019-09-06 10:17:07 -07:00
0ccf4b0fe1
YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta
2019-09-05 14:01:42 -07:00
03489124ea
YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T.
2019-09-03 11:53:34 +05:30
751b5a1ac8
YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph.
2019-09-01 22:40:40 +05:30
c7ef4fb74a
YARN-8678. Queue Management API - rephrase error messages. Contributed by Prabhu Joseph.
2019-09-01 10:00:04 +05:30
d2d963f3d4
YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam.
2019-09-01 09:54:46 +05:30
7d998cb627
YARN-9800. TestRMDelegationTokens can fail in testRemoveExpiredMasterKeyInRMStateStore. Contributed by Adam Antal.
2019-08-31 10:03:09 +05:30
eeccd2f6f6
YARN-9798. ApplicationMasterServiceTestBase#testRepeatedFinishApplicationMaster fails intermittently. Contributed by Tao Yang.
2019-08-31 09:57:45 +05:30
c929b383f8
YARN-9540. TestRMAppTransitions fails intermittently. Contributed by Tao Yang.
2019-08-30 17:17:43 +05:30
5b557de721
YARN-9714. ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby. Contributed by Tao Yang.
2019-08-30 10:35:09 +05:30
8c0759d02a
YARN-9664. Improve response of scheduler/app activities for better understanding. Contributed by Tao Yang.
2019-08-29 18:14:39 +08:00
6f2226a013
YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06).
2019-08-28 20:40:34 +00:00
8ef46595da
YARN-9438. launchTime not written to state store for running applications
2019-08-27 15:53:43 -07:00
717c853873
YARN-9755. Fixed RM failing to start when FileSystemBasedConfigurationProvider is configured.
...
Contributed by Prabhu Joseph
2019-08-27 13:14:59 -04:00
d70f5231a7
YARN-9640. Slow event processing could cause too many attempt unregister events. Contributed by Bibin A Chundatt.
2019-08-27 08:38:12 +05:30
8660e48ca1
YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph
2019-08-26 15:50:33 -07:00
d3ce53e507
YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt.
2019-08-26 23:21:33 +05:30
689d2e6105
YARN-8917. Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource. Contributed by Tao Yang.
2019-08-26 21:06:15 +05:30
7ab88dbfa6
YARN-7291. Better input parsing for resource in allocation file. Contributed by Zoltan Siegl
2019-08-21 17:01:18 +02:00
0e0ddfaf24
YARN-2599. Standby RM should expose jmx endpoint. Contributed by Rohith Sharma K S.
2019-08-17 15:43:19 +05:30
9b8359bb08
YARN-9461. TestRMWebServicesDelegationTokenAuthentication.testCancelledDelegationToken fails with HTTP 400. Contributed by Peter Bacsko
2019-08-16 12:31:58 +02:00
4456ea67b9
YARN-8586. Extract log aggregation related fields and methods from RMAppImpl. Contributed by Peter Bacsko
2019-08-16 11:36:14 +02:00
1845a83cec
YARN-9488. Skip YARNFeatureNotEnabledException from ClientRMService. Contributed by Prabhu Joseph
2019-08-15 17:15:38 +02:00
e4b538bbda
YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph
2019-08-12 15:15:43 +02:00
13a5803ccf
YARN-9464. Support pending resource metrics in RM's RESTful API. Contributed by Prabhu Joseph.
2019-08-12 14:31:24 +05:30
8fbf8b2eb0
YARN-9722. PlacementRule logs object ID in place of queue name. Contributed by Prabhu Joseph.
2019-08-12 10:44:46 +05:30
a79564fed0
YARN-9732. yarn.system-metrics-publisher.enabled=false is not honored by RM. Contributed by KWON BYUNGCHANG.
2019-08-09 22:25:30 +05:30
3b38f2019e
YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang.
2019-08-08 12:37:50 +00:00
42683aef1a
YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan.
2019-07-30 18:58:36 +00:00
7a93be0f60
YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T.
2019-07-16 14:03:22 +05:30
91ce09e706
YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko
2019-07-15 10:47:20 +02:00
9b54dd7186
YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung)
2019-07-11 13:57:08 -07:00
15d82fcb75
YARN-9658. Fix UT failures in TestLeafQueue. Contributed by Tao Yang.
2019-07-03 12:08:45 +08:00
e966edd025
YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt.
2019-07-02 12:18:16 +05:30
cbae241320
YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang.
2019-06-28 23:24:53 +08:00
be80334cdf
YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T.
2019-06-27 09:59:44 +05:30
83dcb9d87e
YARN-9209. When nodePartition is not set in Placement Constraints, containers are allocated only in default partition. Contributed by Tarun Parimi.
2019-06-21 17:41:05 +08:00
304a47e22c
YARN-9608. DecommissioningNodesWatcher should get lists of running applications on node from RMNode. Contributed by Abhishek Modi.
2019-06-17 17:09:56 +08:00
970b0b0c02
YARN-9578. Add limit/actions/summarize options for app activities REST API. Contributed by Tao Yang.
2019-06-13 10:44:47 +08:00
60c95e9b6a
YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T.
2019-06-11 23:11:49 +05:30
9191e08f0a
YARN-9569. Auto-created leaf queues do not honor cluster-wide min/max memory/vcores. Contributed by Craig Condit.
2019-06-10 14:33:24 -07:00
0976392502
YARN-9590. Correct incompatible, incomplete and redundant activities. Contributed by Tao Yang.
2019-06-06 21:59:01 +08:00
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
bd2590d71b
YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang.
2019-06-03 22:59:02 +08:00
abf76ac371
YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles)
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2019-05-29 11:24:08 -05:00
544876fe12
YARN-8693. Add signalToContainer REST API for RMWebServices. Contributed by Tao Yang.
2019-05-29 16:34:48 +08:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
9078e28a24
YARN-9503. Fix JavaDoc error in TestSchedulerOvercommit. Contributed by Wanqiang Ji.
2019-05-28 15:52:39 +09:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
9f056d905f
YARN-9497. Support grouping by diagnostics for query results of scheduler and app activities. Contributed by Tao Yang.
2019-05-26 09:56:36 -04:00
55bd35921c
YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko.
2019-05-15 11:50:46 -07:00
570fa2da20
YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T.
2019-05-15 13:30:09 +05:30
29ff7fb140
YARN-9493. Scheduler Page does not display the right page by query string. Contributed by Wanqiang Ji.
2019-05-13 10:57:12 -07:00
1a47c2b7ae
YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang.
2019-05-12 22:31:39 -07:00
90add05caa
YARN-9489. Support filtering by request-priorities and allocation-request-ids for query results of app activities. Contributed by Tao Yang.
2019-05-09 21:54:09 +08:00
3172f6cbf9
YARN-9513. Addendum patch: Fix ASF License warnings. Contributed by Giovanni Matteo Fumarola.
2019-05-08 14:56:23 +09:00
c336af3847
YARN-9432. Reserved containers leak after its request has been cancelled or satisfied when multi-nodes enabled. Contributed by Tao Yang.
2019-05-08 09:54:16 +08:00
8ecbf61cca
YARN-9513. [JDK11] Fix TestMetricsInvariantChecker#testManyRuns in case of JDK greater than 8. Contributed by Adam Antal.
2019-05-07 10:59:02 -07:00
12b7059ddc
YARN-9440. Improve diagnostics for scheduler and app activities. Contributed by Tao Yang.
2019-05-06 20:00:15 +08:00
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
fec9bf4b0b
YARN-9501. TestCapacitySchedulerOvercommit#testReducePreemptAndCancel fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:42:56 -07:00
4a0ba24959
YARN-9491. TestApplicationMasterServiceFair#ApplicationMasterServiceTestBase.testUpdateTrackingUrl fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 15:27:04 -07:00
c504eee0c2
YARN-9339. Apps pending metric incorrect after moving app to a new queue. Contributed by Abhishek Modi.
2019-04-23 12:40:44 -07:00
1c8046d67e
YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph.
2019-04-23 14:21:13 +08:00
96e3027e46
YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi.
2019-04-22 09:49:03 -07:00
df76cdc895
YARN-6695. Fixed NPE in publishing appFinished events to ATSv2.
...
Contributed by Prabhu Joseph
2019-04-18 12:29:37 -04:00
aa4c744aef
YARN-9470. Fix order of actual and expected expression in assert statements
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-18 15:40:37 +09:00
7fa73fac26
YARN-9439. Support asynchronized scheduling mode and multi-node lookup mechanism for app activities. Contributed by Tao Yang.
2019-04-16 00:12:43 +08:00
7a68e7abd5
YARN-9474. Remove hard coded sleep from Opportunistic Scheduler tests. Contributed by Abhishek Modi.
2019-04-14 20:11:20 -07:00
ed3747c1cc
YARN-9435. Add Opportunistic Scheduler metrics in ResourceManager. Contributed by Abhishek Modi.
2019-04-11 11:49:19 -07:00
8c1bba375b
YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu.
2019-04-10 22:51:28 +08:00
32722d2661
YARN-9433. Remove unused constants in YARN resource manager
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-10 18:37:27 +09:00
cfec455c45
YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri.
2019-04-09 10:59:43 -07:00
fc05b0e70e
YARN-9313. Support asynchronized scheduling mode and multi-node lookup mechanism for scheduler activities. Contributed by Tao Yang.
2019-04-08 13:40:53 +08:00
ec143cbf67
YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang.
2019-04-06 19:59:36 +08:00
8d150067e2
YARN-9396. Fixed duplicated RM Container created event to ATS.
...
Contributed by Prabhu Joseph
2019-04-04 13:01:56 -04:00
002dcc4ebf
YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
2019-04-03 18:57:28 +05:30
2f752830ba
YARN-9214. Add AbstractYarnScheduler#getValidQueues method to remove duplication. Contributed by Wanqiang Ji.
2019-04-01 20:05:15 -07:00
da7f8c244d
YARN-9431. Fix flaky junit test fair.TestAppRunnability after YARN-8967. Contributed by Wilfred Spiegelenburg.
2019-04-01 11:21:31 -07:00
332cab5518
YARN-9418. ATSV2 /apps//entities/YARN_CONTAINER rest api does not show metrics. Contributed by Prabhu Joseph.
2019-04-01 11:06:51 -07:00
5257f50abb
YARN-8967. Change FairScheduler to use PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-25 22:47:24 -07:00
509b20b292
YARN-9404. TestApplicationLifetimeMonitor#testApplicationLifetimeMonitor fails intermittent. Contributed by Prabhu Joseph.
2019-03-22 11:45:39 -07:00
ce5eb9cb2e
YARN-9358. Add javadoc to new methods introduced in FSQueueMetrics with YARN-9322
...
(Contributed by Zoltan Siegl via Daniel Templeton)
Change-Id: I92d52c0ca630e71afb26b2b7587cbdbe79254a05
2019-03-22 12:28:34 +01:00
548997d6c9
YARN-9402. Opportunistic containers should not be scheduled on Decommissioning nodes. Contributed by Abhishek Modi.
2019-03-21 12:04:05 -07:00
5d8bd0e5cb
YARN-9392. Handle missing scheduler events in Opportunistic Scheduler. Contributed by Abhishek Modi.
2019-03-19 11:00:21 -07:00
09eabda314
YARN-9364. Remove commons-logging dependency from YARN.
...
Contributed by Prabhu Joseph
2019-03-18 19:58:42 -04:00
5f6e225166
YARN-9363. Replaced debug logging with SLF4J parameterized log message.
...
Contributed by Prabhu Joseph
2019-03-18 13:57:18 -04:00
2064ca015d
YARN-9349. Changed logging to use slf4j api.
...
Contributed by Prabhu Joseph
2019-03-15 19:20:59 -04:00
03f3c8aed2
YARN-4404. Corrected typo in javadoc.
...
Contributed by Yesha Vora
2019-03-15 18:04:04 -04:00
39b4a37e02
YARN-9341. Fixed enentrant lock usage in YARN project.
...
Contributed by Prabhu Joseph
2019-03-07 16:47:45 -05:00
491313ab84
YARN-8218 Add application launch time to ATSV1. Contributed by Abhishek Modi
2019-03-06 21:47:29 -08:00
0aefe2846f
YARN-9298. Implement FS placement rules using PlacementRule interface. Contributed by Wilfred Spiegelenburg.
2019-03-04 23:49:07 -08:00
e40e2d6ad5
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-03-05 14:10:08 +09:00
15098df744
Revert "YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph."
...
This reverts commit a63c358b78
2019-03-04 16:57:31 +05:30
538bb4880d
YARN-9323. FSLeafQueue#computeMaxAMResource does not override zero values for custom resources
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: Id844ccf09488f367c0c7de0a3b2d4aca1bba31cc
2019-02-27 19:59:48 -08:00
7b928f19a4
YARN-9322. Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
...
(Contributed by Szilard Nemeth via Daniel Templeton)
Change-Id: I14c12f1265999d62102f2ec5506d90015efeefe8
2019-02-27 19:43:50 -08:00
1779fc57a1
YARN-9324. TestSchedulingRequestContainerAllocation(Async) fails with junit-4.11. Contributed by Prabhu Joseph.
2019-02-28 09:56:29 +08:00
8c30114b00
YARN-9248. RMContainerImpl:Invalid event: ACQUIRED at KILLED. Contributed by lujie.
2019-02-27 17:29:02 +08:00
8eae260af5
YARN-9311. Fix TestRMRestart hangs. Contributed by Prabhu Joseph.
2019-02-27 10:28:16 +05:30
95372657fc
YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena.
2019-02-25 11:45:37 -08:00
3e1739d589
YARN-9329. updatePriority is blocked when using FairScheduler. Contributed by Jiandan Yang.
2019-02-26 00:08:13 +08:00
9cd5c5447f
YARN-9316. TestPlacementConstraintsUtil#testInterAppConstraintsByAppID fails intermittently. Contributed by Prabhu Joseph.
2019-02-24 22:42:27 +08:00
50094d7fef
YARN-9300. Lazy preemption should trigger an update on queue preemption metrics for CapacityScheduler. Contributed by Tao Yang.
2019-02-24 22:17:29 +08:00
ed13cf8406
YARN-9317. Avoid repeated YarnConfiguration#timelineServiceV2Enabled check. Contributed by Prabhu Joseph
2019-02-22 21:01:34 +05:30
9c88695bcd
YARN-9238. Avoid allocating opportunistic containers to previous/removed/non-exist application attempt. Contributed by lujie.
2019-02-22 20:58:33 +08:00
eedcc8e26e
YARN-9315. TestCapacitySchedulerMetrics fails intermittently. Contributed by Prabhu Joseph.
2019-02-21 17:21:21 +08:00
a63c358b78
YARN-8132. Final Status of applications shown as UNDEFINED in ATS app queries. Contributed by Prabhu Joseph.
2019-02-21 13:29:01 +05:30
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
7536488bbd
YARN-996. REST API support for node resource configuration. Contributed by Inigo Goiri.
2019-02-11 15:27:43 -08:00
0a1637c750
YARN-8555. Parameterize TestSchedulingRequestContainerAllocation(Async) to cover both PC handler options. Contributed by Prabhu Joseph.
2019-02-11 15:53:50 +08:00
7fa62e150c
YARN-8219. Add application launch time to ATSV2. Contributed by Abhishek Modi.
2019-02-07 09:58:53 +05:30
604b2489a9
YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla.
2019-02-04 12:48:28 +05:30
28ad20a711
YARN-9262. TestRMAppAttemptTransitions is failing with an NPE. Contributed by lujie.
2019-02-02 23:02:45 +08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
04105bbfdb
YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph
2019-01-29 17:22:24 +00:00
6cace58e21
YARN-9222. Print launchTime in ApplicationSummary
2019-01-25 13:17:00 -08:00
a33ef4fd31
YARN-8867. Added resource localization status to YARN service status call.
...
Contributed by Chandni Singh
2019-01-24 18:43:21 -05:00
a4bd64e724
YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu.
2019-01-24 17:53:43 +08:00
721d5c2a5f
YARN-8101. Add UT to verify node-attributes in RM nodes rest API. Contributed by Prabhu Joseph.
2019-01-23 17:59:36 +08:00
e3e076dc78
YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang.
2019-01-23 14:14:49 +08:00
d43df31751
YARN-9210. RM nodes web page can not display node info. Contributed by Jiandan Yang.
2019-01-22 10:23:32 +08:00
abde1e1f58
YARN-9204. RM fails to start if absolute resource is specified for partition capacity in CS queues. Contributed by Jiandan Yang.
2019-01-21 16:54:14 +08:00
6d7eedfd28
YARN-9194. Invalid event: REGISTERED and LAUNCH_FAILED at FAILED, and NullPointerException happens in RM while shutdown a NM. (lujie via wangda)
...
Change-Id: I4359f59a73a278a941f4bb9d106dd38c9cb471fe
2019-01-17 14:20:10 -08:00
614af50625
YARN-9179. Fix NPE in AbstractYarnScheduler#updateNewContainerInfo.
2019-01-15 12:59:59 +09:00
6a923464af
YARN-6523. Optimize system credentials sent in node heartbeat responses. Contributed by Manikandan R
2019-01-08 16:54:05 -06:00
0921b706f7
YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan.
2019-01-08 14:30:53 +08:00
944cf87223
YARN-9173. FairShare calculation broken for large values after YARN-8833. Contributed by Wilfred Spiegelenburg.
2019-01-07 15:57:31 +08:00
f4906ac019
YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang.
2019-01-04 12:10:00 +05:30
cfe89e6f96
YARN-9164. Shutdown NM may cause NPE when opportunistic container scheduling is enabled. Contributed by lujie.
2019-01-03 23:56:28 +08:00
128f340f4f
YARN-8894. Improve InMemoryPlan#toString. Contributed by BELUGA BEHR.
2018-12-28 17:43:47 +09:00
15fe1b34dd
YARN-8878. Remove StringBuffer from ManagedParentQueue.java. Contributed by BELUGA BEHR.
2018-12-28 17:29:22 +09:00
091ad974cd
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
f82922dcfa
YARN-5168. Added exposed port information for Docker container.
...
Contributed by Xun Liu
2018-12-21 19:44:07 -05:00
f659485ee8
YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang.
2018-12-21 10:56:42 +08:00
881230da21
YARN-9051. Integrate multiple CustomResourceTypesConfigurationProvider implementations into one. (Contributed by Szilard Nemeth)
2018-12-11 11:41:32 -08:00
64411a6ff7
YARN-8738. FairScheduler should not parse negative maxResources or minResources values as positive. (Contributed by Szilard Nemeth)
2018-12-10 11:06:50 -08:00
cc51607ccd
YARN-9085. Add Guaranteed and MaxCapacity to CSQueueMetrics
2018-12-07 10:45:37 -08:00
5d4a432660
YARN-9025. TestFairScheduler#testChildMaxResources is flaky. (Contributed by Szilard Nemeth)
2018-12-05 16:36:13 -08:00
b3c75c1f1d
YARN-8985. Improve debug log in FSParentQueue when assigning container. (Contributed by Wilfred Spiegelenburg)
2018-12-05 15:02:57 -08:00
2bba329c14
YARN-8994. Fix race condition between move app and queue cleanup in Fair Scheduler. (Contributed by Wilfred Spiegelenburg)
2018-12-05 14:38:15 -08:00
e89941fdbb
YARN-9041. Performance Optimization of method FSPreemptionThread#identifyContainersToPreempt. Contributed by Wanqiang Ji.
2018-12-04 10:08:45 -08:00
6d7b44c489
YARN-9036. Escape newlines in health report in YARN UI. Contributed by Keqiu Hu
2018-11-30 10:13:52 -08:00
07142f54a8
YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T.
2018-11-29 22:02:59 +05:30
a68d766e87
YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt.
2018-11-29 21:43:34 +05:30
a41b648e98
YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg)
2018-11-20 14:12:37 -08:00
5fb14e0635
YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang.
2018-11-20 08:02:11 +05:30
d027a24f03
YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun.
2018-11-18 23:18:26 +08:00
c741109522
YARN-8987. Usability improvements node-attributes CLI. Contributed by Bibin A Chundatt.
2018-11-12 18:18:23 +08:00
4e7284443e
YARN-8902. [CSI] Add volume manager that manages CSI volume lifecycle. Contributed by Weiwei Yang.
2018-11-12 11:57:02 +05:30
524a7523c4
YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg)
2018-11-08 16:02:48 -08:00
e1bbf7dcdf
YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang.
2018-11-08 17:47:18 +08:00
c96cbe8659
YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji.
2018-11-07 22:42:22 +08:00
951c98f890
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang.
2018-11-07 11:17:35 +09:00
ab6aa4c726
YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
2018-11-06 08:40:59 -06:00
5d6554c722
YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang.
2018-11-06 14:50:09 +08:00
c7fcca0d7e
YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji.
2018-11-06 13:14:57 +08:00
babc946d40
YARN-8954. Reservations list field in ReservationListInfo is not accessible. Contributed by Oleksandr Shevchenko.
2018-11-02 11:10:08 -07:00
Szilard Nemeth
Sunil G
Abhishek Modi
Eric Yang
Zhankun Tang
Eric E Payne
Mate Szalay-Beko
Jonathan Hung
bibinchundatt
Eric Badger
Duo Zhang
Wangda Tan
Tao Yang
Rohith Sharma K S
Weiwei Yang
Haibo Chen
Suma Shivaprasad
Ahmed Hussein
Akira Ajisaka
Giovanni Matteo Fumarola
Sean Mackrory
Inigo Goiri
Prabhu Joseph
Igor Rudenko
Yufei Gu
Zoltan Siegl
Vrushali C
Yufei Gu
Szilard Nemeth
Jason Lowe
Erik Krogen
Jonathan Hung
Jonathan Hung