Yufei Gu
ce609d06c0
YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu)
2017-08-03 13:52:58 -07:00
Sunil G
82ec700241
YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang.
...
(cherry picked from commit f64cfeaf61
)
2017-08-03 19:28:16 +05:30
Eric Payne
223229b99c
YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe.
...
(cherry picked from commit 8ce8672b6b
)
2017-08-02 12:58:07 -05:00
Jian He
6ee0fe70c7
YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G
2017-08-02 00:03:59 -07:00
Jian He
4006cc266b
YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G
2017-08-01 10:00:04 -07:00
Yufei Gu
e29d1c75ea
YARN-6864. FSPreemptionThread cleanup for readability. (Daniel Templeton via Yufei Gu)
...
(cherry picked from commit 9902be72cb
)
2017-07-27 23:21:30 -07:00
Varun Saxena
05bbbcbdbe
Addendum for YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)
...
(cherry picked from commit 11ece0bda1
)
2017-07-28 00:33:41 +05:30
Varun Saxena
0b342a644a
YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena)
...
(cherry picked from commit f66fd11e51
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
2017-07-28 00:33:37 +05:30
Sunil G
08d5f3acf6
Addendum patch for YARN-6102.
2017-07-26 18:40:52 +05:30
Daniel Templeton
00f4a80e8f
YARN-6307. Refactor FairShareComparator#compare (Contributed by Yufei Gu via Daniel Templeton)
...
(cherry picked from commit f81a4efb8c
)
2017-07-25 13:02:00 -07:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor.
...
(cherry picked from commit cca51e916b
)
2017-07-25 15:25:36 +09:00
Daniel Templeton
9b6e787230
YARN-6845. Variable scheduler of FSLeafQueue duplicates the one of its parent FSQueue.
...
(Contributed by Yufei Gu via Daniel Templeton)
(cherry picked from commit 10583625c1
)
2017-07-24 13:45:09 -07:00
Daniel Templeton
eb1751670a
YARN-6779. DominantResourceFairnessPolicy.DominantResourceFairnessComparator.calculateShares() should be @VisibleForTesting
...
(Contributed by Yeliang Cang via Daniel Templeton)
(cherry picked from commit bb30bd3771
)
2017-07-24 12:15:35 -07:00
Sunil G
a657472b42
YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S.
2017-07-24 20:57:56 +05:30
Wangda Tan
5070c9ba39
YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda)
...
Change-Id: Iaf9bb7e5ed3aa1300abdccf87ae6fcbddbd25e3e
2017-07-19 15:25:21 -07:00
Rohith Sharma K S
f806e8627b
YARN-6819. Application report fails if app rejected due to nodesize. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 845c4e52bd
)
2017-07-19 11:16:17 +05:30
Yufei Gu
d04bb465d2
YARN-6778. In ResourceWeights, weights and setWeights() should be final. (Daniel Templeton via Yufei Gu)
...
(cherry picked from commit daaf530fce
)
2017-07-18 16:40:13 -07:00
Sunil G
487b0a3d7f
Addendum patch for YARN-5731
...
(cherry picked from commit 0b7afc060c
)
2017-07-18 13:46:43 +05:30
Sunil G
9b1abb448d
YARN-6714. IllegalStateException while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by Tao Yang.
2017-07-17 13:22:04 +05:30
Sunil G
d462e48336
YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin.
2017-07-17 12:25:41 +05:30
Jason Lowe
9eb93c6f3f
YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt
...
(cherry picked from commit a5ae5ac50e
)
2017-07-14 15:03:48 -05:00
bibinchundatt
af80d10e54
YARN-6759. Fix TestRMRestart.testRMRestartWaitForPreviousAMToFinish failure. Contributed by Naganarasimha G R
...
(cherry picked from commit 75c0220b44
)
2017-07-14 14:05:38 +05:30
Yufei Gu
b2a34fe8d6
YARN-6769. Make schedulables without demand less needy in FairSharePolicy#compare. (Yunfan Zhou via Yufei Gu)
...
(cherry picked from commit 4a574e9a84
)
2017-07-13 23:11:27 -07:00
Sunil G
fd6935709f
YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. Contributed by Wangda Tan.
2017-07-14 08:45:17 +05:30
Sunil G
f5cdee6bfd
YARN-6792. Incorrect XML convertion in NodeIDsInfo and LabelsToNodesInfo. Contributed by Giovanni Matteo Fumarola.
...
(cherry picked from commit 228ddaa31d
)
2017-07-14 08:08:32 +05:30
Sunil G
d83e8712f6
YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.
...
(cherry picked from commit ca13b224b2
)
2017-07-13 14:50:28 -05:00
Wangda Tan
cbb5f601f1
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
...
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
(cherry picked from commit c583ab02c7
)
2017-07-13 10:44:07 -05:00
Xuan
edc0aa95f1
YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong)
2017-07-12 15:45:51 -07:00
Wangda Tan
f2d440b3b3
YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda)
...
(cherry picked from commit 5fb723bb77
)
2017-07-12 10:06:02 -05:00
Eric Payne
e6cdf770ca
Revert "YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)"
...
This reverts commit eda4ac07c1
.
Commit eda4ac07c1
was a separate patch from trunk rather than a cherry-pick. I will cherryp-ick dependencies and then cherry-pick the trunk commit for YARN-2113.
2017-07-12 08:56:40 -05:00
Yufei Gu
20a2770d72
YARN-6764. Simplify the logic in FairScheduler#attemptScheduling. Contributed by Yufei Gu.
...
(cherry picked from commit 3de47ab5ea
)
2017-07-09 16:11:26 -07:00
Naganarasimha
b08ef44d9e
YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 626d730bfc
)
2017-07-09 17:27:50 +05:30
Daniel Templeton
fcef28e53b
YARN-6410. FSContext.scheduler should be final (Contributed by Yeliang Cang via Daniel Templeton)
...
(cherry picked from commit 9d27880442
)
2017-07-09 19:38:37 +09:00
Daniel Templeton
a6d194d720
YARN-6746. SchedulerUtils.checkResourceRequestMatchingNodePartition() is dead code
...
(Contributed by Deepti Sawhney via Daniel Templeton)
(cherry picked from commit c5a0c38d8e
)
2017-07-09 19:31:20 +09:00
Daniel Templeton
edcfae8258
YARN-6752. Display reserved resources in web UI per application
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
(cherry picked from commit 0615985886
)
2017-07-09 19:19:07 +09:00
Eric Payne
eda4ac07c1
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G)
2017-07-06 10:43:00 -05:00
Naganarasimha
87c81cfad3
YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp.
...
(cherry picked from commit fa1aaee87b
)
2017-07-01 12:32:01 +05:30
Daniel Templeton
93c5421a43
YARN-6751. Display reserved resources in web UI per queue
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
(cherry picked from commit ec97519779
)
2017-06-29 16:57:23 -07:00
Naganarasimha
a5ae7c0cae
YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R.
2017-06-29 00:09:30 +05:30
Naganarasimha
92e266c2b6
YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt.
2017-06-24 23:14:51 +05:30
Yufei Gu
4c59b446e2
YARN-5876. TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk. (Robert Kanter via Yufei Gu)
...
(cherry picked from commit 0b77262890
)
2017-06-23 13:30:20 -07:00
Akira Ajisaka
35f6c7c9c7
YARN-6713. Fix dead link in the Javadoc of FairSchedulerEventLog.java. Contributed by Weiwei Yang.
...
(cherry picked from commit 099dfe92a9
)
2017-06-20 17:23:06 +09:00
Naganarasimha
578318473c
YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp
...
(cherry picked from commit ee89ac84e6
)
2017-06-19 22:45:21 +05:30
bibinchundatt
3f9acf65fb
YARN-6703. RM startup failure with old state store due to version mismatch. Contributed by Varun Saxena
...
(cherry picked from commit d64c842743
)
2017-06-12 18:32:23 +05:30
Eric Payne
33605e0896
YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G.
...
(cherry picked from commit 5578af8603
)
2017-06-09 16:18:16 -05:00
Naganarasimha
85d561a5b3
YARN-6681. Improve performance of AssignmentInformation datastructures. Contributed by Daryn sharp.
2017-06-09 11:46:51 +05:30
Daniel Templeton
10b7a5fa98
YARN-6246. Identifying starved apps does not need the scheduler writelock
...
(Contributed by Karthik Kambatla via Daniel Templeton)
(cherry picked from commit d5b71e4175
)
2017-05-31 15:50:11 -07:00
Jonathan Eagles
f5f12b576e
YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles)
...
(cherry picked from commit cbfed0e82f
)
2017-05-31 10:19:51 -05:00
Yufei Gu
fc6cb4b2dd
YARN-6582. FSAppAttempt demand can be updated atomically in updateDemand(). (Karthik Kambatla via Yufei Gu)
...
(cherry picked from commit 87590090c8
)
2017-05-25 14:25:49 -07:00
Jason Lowe
4c44ff69df
YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter
...
(cherry picked from commit 3fd6a2da4e
)
2017-05-25 16:09:08 -05:00
Subru Krishnan
d1724ef919
YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via Subru).
2017-05-24 13:40:49 -07:00
Wangda Tan
9a8fdfa385
YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda)
...
Change-Id: Ib20326daac10acad363befb0ae0998441fa5b32d
(cherry picked from commit 8e0f83e49a
)
2017-05-22 14:43:44 -07:00
Yufei Gu
50e263a316
YARN-6249. TestFairSchedulerPreemption fails inconsistently. (Tao Jie via Yufei Gu)
...
(cherry picked from commit 66bba8c024
)
2017-05-19 15:22:14 -07:00
Rohith Sharma K S
379e5bd26f
HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe.
2017-05-17 08:27:50 +05:30
Jason Lowe
feb7e92121
YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe
...
(cherry picked from commit 489f85933c
)
2017-05-16 09:28:42 -05:00
Haibo Chen
6284cc4663
YARN-6580. Incorrect logger for FairSharePolicy. (Vrushali C via Haibo Chen)
...
(cherry picked from commit 6c35001b9f
)
2017-05-12 13:07:17 -07:00
Daniel Templeton
717f03ecf7
YARN-6380. FSAppAttempt keeps redundant copy of the queue
...
(cherry picked from commit 90cb5b4635
)
2017-05-11 13:48:46 -07:00
Min Shen
9310d24b08
YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen.
...
(cherry picked from commit 2ada100da7
)
2017-05-11 13:29:02 -07:00
Brahma Reddy Battula
3766e54893
HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang
...
(cherry picked from commit 4aae2d40a3
)
2017-05-11 23:38:18 +05:30
Daniel Templeton
df3d7415c0
YARN-6571. Fix JavaDoc issues in SchedulingPolicy
...
(Contributed by Weiwei Yang via Daniel Templeton)
(cherry picked from commit 18d54b5ae2f76513548c453e05cdd6d87777cd06)
2017-05-10 14:37:36 -07:00
Jason Lowe
3230dd1ab6
YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
...
(cherry picked from commit 6099deebcb
)
2017-05-10 08:48:47 -05:00
Karthik Kambatla
3403ed3148
YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha)
...
(cherry picked from commit 166be0ee95
)
2017-05-09 14:44:39 -07:00
Daniel Templeton
b337a1da31
YARN-6481. Yarn top shows negative container number in FS
...
(Contributed by Tao Jie via Daniel Templeton)
(cherry picked from commit 9f0aea0ee2
)
2017-05-02 13:15:18 -07:00
Naganarasimha
5e28311341
YARN-6519. Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager. Contributed by Weiwei Yang.
2017-05-03 00:36:33 +05:30
Subru Krishnan
6d53224afc
YARN-5331. Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru).
...
(cherry picked from commit 6bf42e48ef
)
2017-05-01 18:49:14 -07:00
Subru Krishnan
2030f408e0
YARN-6542. Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent.
...
(cherry picked from commit e514fc432a
)
2017-05-01 18:31:08 -07:00
Subru Krishnan
0eae1c6368
YARN-4359. Update LowCost agents logic to take advantage of YARN-4358. (Jonathan Yaniv and Ishai Menache via Subru).
...
(cherry picked from commit a3a615eeab
)
2017-05-01 18:30:58 -07:00
Daniel Templeton
a8512d5aeb
YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton)
...
(cherry picked from commit 2e52789edf
)
2017-04-28 13:39:11 -07:00
Wangda Tan
503dd650bf
YARN-6392. Add submit time to Application Summary log. (Zhihai Xu via wangda)
...
(cherry picked from commit 2ba21d6376
)
2017-04-24 13:18:47 -07:00
Eric Payne
421e51cb9c
YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe.
...
(cherry picked from commit 52adf71914
)
2017-04-24 12:08:35 -05:00
Sunil G
a9495b206b
YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu.
2017-04-24 12:16:18 +05:30
Jason Lowe
f5a9287bb1
YARN-6501. FSSchedulerNode.java fails to compile with JDK7. Contributed by John Zhuge
2017-04-20 09:52:44 -05:00
Junping Du
9289f4527d
YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S.
...
(cherry picked from commit 6ed9d36224
)
2017-04-17 15:52:02 -07:00
Karthik Kambatla
6f5912c106
YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha)
2017-04-17 14:34:28 -07:00
Wangda Tan
6ade9e6ce6
YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda)
...
(cherry picked from commit 87e2ef8c98
)
2017-04-15 21:34:29 -07:00
Jason Lowe
1699b20526
YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger
...
(cherry picked from commit 4168805502
)
2017-04-14 17:17:46 -05:00
Eric Payne
157a9f3104
YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger
...
(cherry-picked from commit a41f8dd58e
)
2017-04-14 11:04:18 -05:00
Wangda Tan
5756256280
YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)
...
(cherry picked from commit eac6b4c35c
)
2017-04-13 13:16:31 -07:00
Wangda Tan
408d23477f
YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (wangda)
2017-04-13 11:11:57 -07:00
Wangda Tan
2f08c86b7a
YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda)
...
(cherry picked from commit 4d4ad0ebb7
)
2017-04-11 14:59:34 -07:00
Naganarasimha
b04c09163b
YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang.
...
(cherry picked from commit 7d873c465b
)
2017-04-11 22:46:09 +05:30
Jason Lowe
0cb22a0f15
YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu
...
(cherry picked from commit 0e065f2ede
)
2017-04-11 08:45:16 -05:00
Konstantinos Karanasos
9b3ba25980
YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos)
...
(cherry picked from commit 3587b46164514e6751d7cebb09c8ab5e988853b4)
2017-04-10 17:22:29 -07:00
Karthik Kambatla
e0b143e51a
Revert "YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)"
...
This reverts commit dcfce5fcdd
.
2017-04-07 17:37:31 -07:00
Karthik Kambatla
dcfce5fcdd
YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)
...
(cherry picked from commit 2aa8967809
)
2017-04-07 17:36:39 -07:00
Daniel Templeton
59ad5cd68d
YARN-6381. FSAppAttempt has several variables that should be final
...
(Contributed by Ameet Zaveri via Daniel Templeton)
(cherry picked from commit 984df31a31c25ffc8541467dc41913f62f69e1df)
2017-04-05 16:44:35 -07:00
Karthik Kambatla
7c0f9bd5e7
YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha)
...
(cherry picked from commit b4c4f36594
)
2017-04-05 16:00:30 -07:00
Varun Saxena
0a7ec1fcf7
YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena)
...
(cherry picked from commit 7d963c477a
)
2017-04-06 01:58:32 +05:30
Varun Saxena
05c008256a
YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena)
...
(cherry picked from commit 19b89c4c7b
)
2017-04-05 09:40:09 +05:30
Karthik Kambatla
1938f97c0b
YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha)
...
(cherry picked from commit 9cc04b4708
)
2017-04-04 17:08:59 -07:00
Wangda Tan
70a810adc0
YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda)
...
(cherry picked from commit 3fdae0a2b6
)
2017-04-04 14:39:38 -07:00
Eric Payne
4396c8e938
YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe
...
(cherry picked from commit 318bfb01bc
)
2017-03-31 12:36:01 -05:00
Jason Lowe
3ae529ba54
YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu
...
(cherry picked from commit 4d1fac5df2
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClientOnRMRestart.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
2017-03-31 10:21:44 -05:00
Daniel Templeton
c2636468d7
YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled
...
(cherry picked from commit 640ba1d23f
)
2017-03-29 12:38:50 -07:00
Robert Kanter
a65011a128
YARN-6050. AMs can't be scheduled on racks or nodes (rkanter)
2017-03-28 14:33:08 -07:00
Jason Lowe
bea8f8190f
YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter
2017-03-28 09:45:18 -05:00
Wangda Tan
c97da481d7
YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda)
...
(cherry picked from commit cd014d57aa
)
2017-03-27 13:59:36 -07:00
Akira Ajisaka
e4320305d3
YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki.
...
(cherry picked from commit b63ae854f9
)
2017-03-27 11:26:23 +09:00
Sunil G
f51ee482a7
YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki.
2017-03-26 17:00:06 +05:30
Daniel Templeton
d4bba011cc
YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable
...
(Contributed by Laura Adams via Daniel Templeton)
(cherry picked from commit 0a05c5c598
)
2017-03-21 14:27:27 -07:00
Varun Saxena
e2cabb4d21
YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena)
...
(cherry picked from commit 35034653d0
)
2017-03-20 23:51:15 +05:30
Sunil G
caaa6fa96b
YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202
...
(cherry picked from commit 2d0e24eb01
)
2017-03-15 14:58:20 +05:30
Ray Chiang
fa59f4e490
YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang)
2017-03-14 14:45:13 -07:00
Jason Lowe
ef95e74eb1
YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger
...
(cherry picked from commit 9649c27864
)
2017-03-10 13:07:20 -06:00
Karthik Kambatla
4218671f3c
YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha)
...
(cherry picked from commit a96afae125
)
2017-03-09 23:16:19 -08:00
Daniel Templeton
c60cd88ad1
YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton)
2017-03-09 18:18:03 -08:00
Daniel Templeton
6774f9c3c2
YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton)
...
(cherry picked from commit 8bc8804be1b96f8a4fc33b5d24dd7f7af93a437e)
2017-03-09 17:53:24 -08:00
Jason Lowe
4a1187238a
YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne
...
(cherry picked from commit d7762a5511
)
2017-03-08 16:47:31 -06:00
Sunil G
ec4c2d42fe
YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 1eb8186703
)
2017-03-08 12:13:14 +05:30
Daniel Templeton
83fbb051bb
YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton)
...
(cherry picked from commit e0c239cdbd
)
2017-03-07 12:01:38 -08:00
Junping Du
a58dfcb421
YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He.
...
(cherry picked from commit 8db7a8c3ae
)
2017-03-03 14:40:32 -08:00
Ray Chiang
303ee13e3c
YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang)
...
(cherry picked from commit 30b91ff9540e35195af834d1bd5983114a556c6d)
2017-03-03 12:57:17 -08:00
Jason Lowe
5eca427da2
YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen
...
(cherry picked from commit e58fc76030
)
2017-03-03 08:24:05 -06:00
Jian He
c7ddf95d4a
YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam
...
(cherry picked from commit 235203dffd
)
2017-02-28 13:26:55 -08:00
Xuan
95bd3c3d55
YARN-6189: Improve application status log message when RM restarted when
...
app is in NEW state. Contributed by Junping Du
(cherry picked from commit e0bb867c3f
)
2017-02-28 11:06:07 -08:00
Karthik Kambatla
58eefc71c7
YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha)
...
(cherry picked from commit f187d63816
)
2017-02-27 22:16:32 -08:00
Naganarasimha
1eec911cd9
YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R.
...
(cherry picked from commit 5f5b031d1f
)
2017-02-27 20:42:10 +05:30
Sunil G
68b08e96a0
YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan.
2017-02-27 17:54:47 +05:30
Karthik Kambatla
c24325bbcb
YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha)
...
(cherry picked from commit fbfe86deea
)
2017-02-26 20:37:18 -08:00
Karthik Kambatla
a95d3e1fe7
YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha)
...
(cherry picked from commit 815d53506f
)
2017-02-26 20:31:03 -08:00
Wangda Tan
650ff95e00
YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
...
(cherry picked from commit 0a55bd841e
)
2017-02-23 17:22:25 -08:00
Karthik Kambatla
40bc9e7ddb
YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha)
...
(cherry picked from commit 694e680d20
)
2017-02-23 15:30:29 -08:00
Wangda Tan
a82480c7d4
YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda)
2017-02-23 13:59:20 -08:00
Sunil G
801357726d
YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt.
...
(cherry picked from commit a207aa9930
)
2017-02-23 14:25:14 +05:30
Karthik Kambatla
37edbd35f1
YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha)
...
(cherry picked from commit b10e962224
)
2017-02-22 15:59:11 -08:00
Karthik Kambatla
ca7a6a7365
YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha)
...
(cherry picked from commit 718ad9f6ee
)
2017-02-22 15:52:44 -08:00
Daniel Templeton
93219b6454
YARN-6125. The application attempt's diagnostic message should have a maximum size
...
(Contributed by Andras Piros via Daniel Templeton)
(cherry picked from commit c7a36e6130
)
2017-02-17 14:38:06 -08:00
Karthik Kambatla
5575328bb6
YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha)
...
(cherry picked from commit dbbfcf74ab
)
2017-02-17 14:07:47 -08:00
Daniel Templeton
10fd5627bb
YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with
...
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)
(cherry picked from commit 585168cc1bdb87398d4f388f3eeccd1c71114982)
2017-02-17 13:14:37 -08:00
Karthik Kambatla
afc8124ff6
YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha)
...
(cherry picked from commit a77f432449
)
2017-02-16 15:03:35 -08:00
Rohith Sharma K S
f07b52dc5a
YARN-6200. Reverting since the same functionality achieved by YARN-1623.
...
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"
This reverts commit a37b3694ea
.
2017-02-16 17:58:38 +05:30
Karthik Kambatla
9fecd5063f
YARN-5798. Set UncaughtExceptionHandler for all FairScheduler threads. (Yufei Gu via kasha)
...
(cherry picked from commit 74dd142250
)
2017-02-16 00:03:26 -08:00
Karthik Kambatla
fefac1276a
YARN-4212. FairScheduler: Can't create a DRF queue under a FAIR policy queue. (Yufei Gu via kasha)
...
(cherry picked from commit 11be3f70e0
)
2017-02-15 23:51:51 -08:00
Karthik Kambatla
46b6c95e0a
YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha)
...
(cherry picked from commit 6c25dbcdc0
)
2017-02-15 23:23:18 -08:00
Varun Saxena
9cab25a63d
YARN-6156. AM blacklisting to consider node label partition (Bibin A Chundatt via Varun Saxena)
...
(cherry picked from commit b7613e0f40
)
2017-02-15 14:58:39 +05:30
Karthik Kambatla
58acb676a2
YARN-6061. Addendum. Remove extraneous change.
...
(cherry picked from commit 353a9b2d91
)
2017-02-14 15:20:11 -08:00
Karthik Kambatla
37921b3fef
YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha)
...
(cherry picked from commit 652679aa8a
)
2017-02-14 13:40:06 -08:00
Karthik Kambatla
30ff5bff1a
YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha)
...
(cherry picked from commit 646c6d6509
)
2017-02-13 11:27:02 -08:00
Karthik Kambatla
e259a818ad
YARN-6144. FairScheduler: preempted resources can become negative. (Miklos Szegedi via kasha)
...
(cherry picked from commit 08f93978f3
)
2017-02-09 14:45:21 -08:00
Karthik Kambatla
41361ec39e
YARN-6112. UpdateCallDuration is calculated only when debug logging is enabled. (Yufei Gu via kasha)
2017-02-09 13:12:21 -08:00
Daniel Templeton
be14bda5c0
YARN-4658. Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment (Contributed by Udai Potluri via Daniel Templeton)
...
(cherry picked from commit 3e06475307
)
2017-01-31 16:26:41 -08:00
Daniel Templeton
ccf33bc36c
YARN-6103. Log updates for ZKRMStateStore (Contributed by Daniel Sturman via Daniel Templeton)
...
(cherry picked from commit 87852b6ef4
)
2017-01-30 21:46:38 -08:00
Karthik Kambatla
c9824c5ff6
YARN-6131. FairScheduler: Lower update interval for faster tests. (kasha)
...
(cherry picked from commit 11e44bdda7
)
2017-01-30 13:39:35 -08:00
Karthik Kambatla
1b1a25cca6
YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha)
...
(cherry picked from commit 4d3f73acc0
)
2017-01-26 14:11:50 -08:00
Daniel Templeton
c4eeac02de
YARN-4975. Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues
...
(Contributed by Yufei Gu via Daniel Templeton)
(cherry picked from commit f85b74ccf9
)
2017-01-26 14:08:46 -08:00
Daniel Templeton
443f2803ac
YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton)
...
(cherry picked from commit b3befc021b
)
2017-01-26 14:07:46 -08:00
Karthik Kambatla
73497f08fc
YARN-5830. FairScheduler: Avoid preempting AM containers. (Yufei Gu via kasha)
...
(cherry picked from commit abedb8a9d8
)
2017-01-25 12:19:27 -08:00
Karthik Kambatla
71026cc3f6
YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha)
...
(cherry picked from commit e224c96234
)
2017-01-25 12:19:20 -08:00
Karthik Kambatla
0ceb7149d9
YARN-4752. Improved preemption in FairScheduler. (kasha)
2017-01-25 10:45:02 -08:00
Wangda Tan
1309accd68
YARN-5864. Capacity Scheduler - Queue Priorities. (wangda)
2017-01-24 14:44:42 -08:00