Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,848 Commits 326 Branches 371 Tags
Commit Graph

1532 Commits

Author SHA1 Message Date
Jason Lowe
ef95e74eb1 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 
(cherry picked from commit 9649c27864a23ea156bae904368c1d3cf94c6e9d)
 2017-03-10 13:07:20 -06:00
Karthik Kambatla
4218671f3c YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 
(cherry picked from commit a96afae125ba02fb4480542d3fb0891623ee4c37)
 2017-03-09 23:16:19 -08:00
Daniel Templeton
c60cd88ad1 YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton) 2017-03-09 18:18:03 -08:00
Daniel Templeton
6774f9c3c2 YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton) 
(cherry picked from commit 8bc8804be1b96f8a4fc33b5d24dd7f7af93a437e)
 2017-03-09 17:53:24 -08:00
Jason Lowe
4a1187238a YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)
 2017-03-08 16:47:31 -06:00
Sunil G
ec4c2d42fe YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt. 
(cherry picked from commit 1eb81867032b016a59662043cbae50daa52dafa9)
 2017-03-08 12:13:14 +05:30
Daniel Templeton
83fbb051bb YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton) 
(cherry picked from commit e0c239cdbda336e09a35d112d451c2e17d74a3fc)
 2017-03-07 12:01:38 -08:00
Junping Du
a58dfcb421 YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He. 
(cherry picked from commit 8db7a8c3aea3d989361f32cca5b271e9653773b6)
 2017-03-03 14:40:32 -08:00
Ray Chiang
303ee13e3c YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang) 
(cherry picked from commit 30b91ff9540e35195af834d1bd5983114a556c6d)
 2017-03-03 12:57:17 -08:00
Jason Lowe
5eca427da2 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc7603053e3ac1bc2464f9622995017db5245)
 2017-03-03 08:24:05 -06:00
Jian He
c7ddf95d4a YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam 
(cherry picked from commit 235203dffda1482fb38762fde544c4dd9c3e1fa8)
 2017-02-28 13:26:55 -08:00
Xuan
95bd3c3d55 YARN-6189: Improve application status log message when RM restarted when 
app is in NEW state. Contributed by Junping Du

(cherry picked from commit e0bb867c3fa638c9f689ee0b044b400481cf02b5)
 2017-02-28 11:06:07 -08:00
Karthik Kambatla
58eefc71c7 YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha) 
(cherry picked from commit f187d63816584b783fbfe238475c8f37decdb6dc)
 2017-02-27 22:16:32 -08:00
Naganarasimha
1eec911cd9 YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R. 
(cherry picked from commit 5f5b031d1f20cb7f621db41979e963eaa42cf52f)
 2017-02-27 20:42:10 +05:30
Sunil G
68b08e96a0 YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan. 2017-02-27 17:54:47 +05:30
Karthik Kambatla
c24325bbcb YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha) 
(cherry picked from commit fbfe86deea5f2aa857cd13fee913b7becee57f93)
 2017-02-26 20:37:18 -08:00
Karthik Kambatla
a95d3e1fe7 YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha) 
(cherry picked from commit 815d53506fb0c5ca029c993d6b094db2ac0ca6eb)
 2017-02-26 20:31:03 -08:00
Wangda Tan
650ff95e00 YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 
(cherry picked from commit 0a55bd841ec0f2eb89a0383f4c589526e8b138d4)
 2017-02-23 17:22:25 -08:00
Karthik Kambatla
40bc9e7ddb YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha) 
(cherry picked from commit 694e680d20dc07f634b539537021b09d9316601c)
 2017-02-23 15:30:29 -08:00
Wangda Tan
a82480c7d4 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-23 13:59:20 -08:00
Sunil G
801357726d YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt. 
(cherry picked from commit a207aa9930e7ee4f10228e2db4b4e733794eb8ea)
 2017-02-23 14:25:14 +05:30
Karthik Kambatla
37edbd35f1 YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha) 
(cherry picked from commit b10e962224a8ae1c6031a05322b0cc5e564bd078)
 2017-02-22 15:59:11 -08:00
Karthik Kambatla
ca7a6a7365 YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 
(cherry picked from commit 718ad9f6ee93d4145f2bb19b7582ce4e1174feaf)
 2017-02-22 15:52:44 -08:00
Daniel Templeton
93219b6454 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)

(cherry picked from commit c7a36e613053ec8b111146004b887c2f13535469)
 2017-02-17 14:38:06 -08:00
Karthik Kambatla
5575328bb6 YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha) 
(cherry picked from commit dbbfcf74ab44d7bfdc805b63affd0defc57182b8)
 2017-02-17 14:07:47 -08:00
Daniel Templeton
10fd5627bb YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with 
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)

(cherry picked from commit 585168cc1bdb87398d4f388f3eeccd1c71114982)
 2017-02-17 13:14:37 -08:00
Karthik Kambatla
afc8124ff6 YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha) 
(cherry picked from commit a77f432449aad67da31bd8bf8644b71def741bde)
 2017-02-16 15:03:35 -08:00
Rohith Sharma K S
f07b52dc5a YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit a37b3694ead2ab4e8753e68213f31868b68376b8.
 2017-02-16 17:58:38 +05:30
Karthik Kambatla
9fecd5063f YARN-5798. Set UncaughtExceptionHandler for all FairScheduler threads. (Yufei Gu via kasha) 
(cherry picked from commit 74dd14225059322825f706120aa57cf673820daf)
 2017-02-16 00:03:26 -08:00
Karthik Kambatla
fefac1276a YARN-4212. FairScheduler: Can't create a DRF queue under a FAIR policy queue. (Yufei Gu via kasha) 
(cherry picked from commit 11be3f70e029c2324b167563168c8a254d234aef)
 2017-02-15 23:51:51 -08:00
Karthik Kambatla
46b6c95e0a YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha) 
(cherry picked from commit 6c25dbcdc0517a825b92fb16444aa1d3761e160c)
 2017-02-15 23:23:18 -08:00
Varun Saxena
9cab25a63d YARN-6156. AM blacklisting to consider node label partition (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit b7613e0f406fb2b9bd5b1b3c79658e801f63c587)
 2017-02-15 14:58:39 +05:30
Karthik Kambatla
58acb676a2 YARN-6061. Addendum. Remove extraneous change. 
(cherry picked from commit 353a9b2d9165a221491395edbadf8acc3a39990b)
 2017-02-14 15:20:11 -08:00
Karthik Kambatla
37921b3fef YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha) 
(cherry picked from commit 652679aa8ad6f9e61b8ed8e2b04b3e0332025e94)
 2017-02-14 13:40:06 -08:00
Karthik Kambatla
30ff5bff1a YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha) 
(cherry picked from commit 646c6d6509f515b1373288869fb92807fa2ddc9b)
 2017-02-13 11:27:02 -08:00
Karthik Kambatla
e259a818ad YARN-6144. FairScheduler: preempted resources can become negative. (Miklos Szegedi via kasha) 
(cherry picked from commit 08f93978f3ec724b24a93d7ef538f158da75802f)
 2017-02-09 14:45:21 -08:00
Karthik Kambatla
41361ec39e YARN-6112. UpdateCallDuration is calculated only when debug logging is enabled. (Yufei Gu via kasha) 2017-02-09 13:12:21 -08:00
Daniel Templeton
be14bda5c0 YARN-4658. Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment (Contributed by Udai Potluri via Daniel Templeton) 
(cherry picked from commit 3e06475307e30377092547dfdebe6c2c6ac6e78f)
 2017-01-31 16:26:41 -08:00
Daniel Templeton
ccf33bc36c YARN-6103. Log updates for ZKRMStateStore (Contributed by Daniel Sturman via Daniel Templeton) 
(cherry picked from commit 87852b6ef4b9d973b7b3999974d41c8860fb1495)
 2017-01-30 21:46:38 -08:00
Karthik Kambatla
c9824c5ff6 YARN-6131. FairScheduler: Lower update interval for faster tests. (kasha) 
(cherry picked from commit 11e44bdda74b3357714f4b11a66f703d64c0f14f)
 2017-01-30 13:39:35 -08:00
Karthik Kambatla
1b1a25cca6 YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha) 
(cherry picked from commit 4d3f73acc0a5cabc748132889dbe670bea178a3f)
 2017-01-26 14:11:50 -08:00
Daniel Templeton
c4eeac02de YARN-4975. Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues 
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit f85b74ccf9f1c1c1444cc00750b03468cbf40fb9)
 2017-01-26 14:08:46 -08:00
Daniel Templeton
443f2803ac YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton) 
(cherry picked from commit b3befc021b0e2d63d1a3710ea450797d1129f1f5)
 2017-01-26 14:07:46 -08:00
Karthik Kambatla
73497f08fc YARN-5830. FairScheduler: Avoid preempting AM containers. (Yufei Gu via kasha) 
(cherry picked from commit abedb8a9d86b4593a37fd3d2313fbcb057c7846a)
 2017-01-25 12:19:27 -08:00
Karthik Kambatla
71026cc3f6 YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha) 
(cherry picked from commit e224c9623493d6c4c2f3ff731fd3c72c0f448b19)
 2017-01-25 12:19:20 -08:00
Karthik Kambatla
0ceb7149d9 YARN-4752. Improved preemption in FairScheduler. (kasha) 2017-01-25 10:45:02 -08:00
Wangda Tan
1309accd68 YARN-5864. Capacity Scheduler - Queue Priorities. (wangda) 2017-01-24 14:44:42 -08:00
Rohith Sharma K S
26c4cfb653 YARN-6082. Invalid REST api response for getApps since queueUsagePercentage is coming as INF. Contributed by Sunil G. 
(cherry picked from commit 0101267d9d801eab4cb3b4df289c402ecb591685)
 2017-01-24 11:01:52 +05:30
Jason Lowe
94b326f2c8 YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 
(cherry picked from commit 69fa81679f59378fd19a2c65db8019393d7c05a2)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
 2017-01-23 17:23:32 +00:00
Sunil G
f8e49bb343 YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang. 
(cherry picked from commit 3fa0d540dfca579f3c2840a959b748a7528b02ed)
 2017-01-23 12:24:39 +05:30