Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,865 Commits 326 Branches 371 Tags
Commit Graph

1857 Commits

Author SHA1 Message Date
Wangda Tan
408d23477f YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (wangda) 2017-04-13 11:11:57 -07:00
Wangda Tan
2f08c86b7a YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda) 
(cherry picked from commit 4d4ad0ebb76198c90c5317f19ce85a6696132c8f)
 2017-04-11 14:59:34 -07:00
Naganarasimha
b04c09163b YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang. 
(cherry picked from commit 7d873c465b815816e9955cf40f4e141dd2e7d82b)
 2017-04-11 22:46:09 +05:30
Jason Lowe
0cb22a0f15 YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu 
(cherry picked from commit 0e065f2edead41f6f894c9d70def389af1c0f239)
 2017-04-11 08:45:16 -05:00
Konstantinos Karanasos
9b3ba25980 YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos) 
(cherry picked from commit 3587b46164514e6751d7cebb09c8ab5e988853b4)
 2017-04-10 17:22:29 -07:00
Karthik Kambatla
e0b143e51a Revert "YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)" 
This reverts commit dcfce5fcddcfdd23392586c87adf7c14e8ef265e.
 2017-04-07 17:37:31 -07:00
Karthik Kambatla
dcfce5fcdd YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha) 
(cherry picked from commit 2aa8967809b05e558b36ab6fc8c5110ddb723b45)
 2017-04-07 17:36:39 -07:00
Daniel Templeton
59ad5cd68d YARN-6381. FSAppAttempt has several variables that should be final 
(Contributed by Ameet Zaveri via Daniel Templeton)

(cherry picked from commit 984df31a31c25ffc8541467dc41913f62f69e1df)
 2017-04-05 16:44:35 -07:00
Karthik Kambatla
7c0f9bd5e7 YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha) 
(cherry picked from commit b4c4f365948d36b36942f912ef994c1c21ba59e3)
 2017-04-05 16:00:30 -07:00
Varun Saxena
0a7ec1fcf7 YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena) 
(cherry picked from commit 7d963c477ab997e0daad760dd3c568d7010fe511)
 2017-04-06 01:58:32 +05:30
Varun Saxena
05c008256a YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 19b89c4c7be0236aafded560b7bcad7e39d1c4ed)
 2017-04-05 09:40:09 +05:30
Karthik Kambatla
1938f97c0b YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha) 
(cherry picked from commit 9cc04b4708879012da1f9ef690679d8d1e45c0cc)
 2017-04-04 17:08:59 -07:00
Wangda Tan
70a810adc0 YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda) 
(cherry picked from commit 3fdae0a2b66c5fb6853875b66fcf50bc96d6e2e9)
 2017-04-04 14:39:38 -07:00
Eric Payne
4396c8e938 YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe 
(cherry picked from commit 318bfb01bc6793da09e32e9cc292eb63224b6ca2)
 2017-03-31 12:36:01 -05:00
Jason Lowe
3ae529ba54 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 
(cherry picked from commit 4d1fac5df2508011adfc4c5d683beb00fd5ced45)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClientOnRMRestart.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2017-03-31 10:21:44 -05:00
Daniel Templeton
c2636468d7 YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 
(cherry picked from commit 640ba1d23fe8b8105bae6d342ddc1c839302f8e5)
 2017-03-29 12:38:50 -07:00
Robert Kanter
a65011a128 YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-28 14:33:08 -07:00
Jason Lowe
bea8f8190f YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter 2017-03-28 09:45:18 -05:00
Wangda Tan
c97da481d7 YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 
(cherry picked from commit cd014d57aa8b896da02b5bcadafbd404bca2bc12)
 2017-03-27 13:59:36 -07:00
Akira Ajisaka
e4320305d3 YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki. 
(cherry picked from commit b63ae854f9f5a900ca860dcb26c30785af32bde0)
 2017-03-27 11:26:23 +09:00
Sunil G
f51ee482a7 YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki. 2017-03-26 17:00:06 +05:30
Daniel Templeton
d4bba011cc YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable 
(Contributed by Laura Adams via Daniel Templeton)

(cherry picked from commit 0a05c5c5989edeba2cffe16e80350245778cefce)
 2017-03-21 14:27:27 -07:00
Varun Saxena
e2cabb4d21 YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena) 
(cherry picked from commit 35034653d02ac8156338d7267e5975d2d66272d5)
 2017-03-20 23:51:15 +05:30
Sunil G
caaa6fa96b YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202 
(cherry picked from commit 2d0e24eb01cec6f4554671b54729717510711c63)
 2017-03-15 14:58:20 +05:30
Ray Chiang
fa59f4e490 YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang) 2017-03-14 14:45:13 -07:00
Jason Lowe
ef95e74eb1 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 
(cherry picked from commit 9649c27864a23ea156bae904368c1d3cf94c6e9d)
 2017-03-10 13:07:20 -06:00
Karthik Kambatla
4218671f3c YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 
(cherry picked from commit a96afae125ba02fb4480542d3fb0891623ee4c37)
 2017-03-09 23:16:19 -08:00
Daniel Templeton
c60cd88ad1 YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton) 2017-03-09 18:18:03 -08:00
Daniel Templeton
6774f9c3c2 YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton) 
(cherry picked from commit 8bc8804be1b96f8a4fc33b5d24dd7f7af93a437e)
 2017-03-09 17:53:24 -08:00
Jason Lowe
4a1187238a YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)
 2017-03-08 16:47:31 -06:00
Sunil G
ec4c2d42fe YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt. 
(cherry picked from commit 1eb81867032b016a59662043cbae50daa52dafa9)
 2017-03-08 12:13:14 +05:30
Daniel Templeton
83fbb051bb YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton) 
(cherry picked from commit e0c239cdbda336e09a35d112d451c2e17d74a3fc)
 2017-03-07 12:01:38 -08:00
Junping Du
a58dfcb421 YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He. 
(cherry picked from commit 8db7a8c3aea3d989361f32cca5b271e9653773b6)
 2017-03-03 14:40:32 -08:00
Ray Chiang
303ee13e3c YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang) 
(cherry picked from commit 30b91ff9540e35195af834d1bd5983114a556c6d)
 2017-03-03 12:57:17 -08:00
Jason Lowe
5eca427da2 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc7603053e3ac1bc2464f9622995017db5245)
 2017-03-03 08:24:05 -06:00
Jian He
c7ddf95d4a YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam 
(cherry picked from commit 235203dffda1482fb38762fde544c4dd9c3e1fa8)
 2017-02-28 13:26:55 -08:00
Xuan
95bd3c3d55 YARN-6189: Improve application status log message when RM restarted when 
app is in NEW state. Contributed by Junping Du

(cherry picked from commit e0bb867c3fa638c9f689ee0b044b400481cf02b5)
 2017-02-28 11:06:07 -08:00
Karthik Kambatla
58eefc71c7 YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha) 
(cherry picked from commit f187d63816584b783fbfe238475c8f37decdb6dc)
 2017-02-27 22:16:32 -08:00
Naganarasimha
1eec911cd9 YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R. 
(cherry picked from commit 5f5b031d1f20cb7f621db41979e963eaa42cf52f)
 2017-02-27 20:42:10 +05:30
Sunil G
68b08e96a0 YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan. 2017-02-27 17:54:47 +05:30
Karthik Kambatla
c24325bbcb YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha) 
(cherry picked from commit fbfe86deea5f2aa857cd13fee913b7becee57f93)
 2017-02-26 20:37:18 -08:00
Karthik Kambatla
a95d3e1fe7 YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha) 
(cherry picked from commit 815d53506fb0c5ca029c993d6b094db2ac0ca6eb)
 2017-02-26 20:31:03 -08:00
Wangda Tan
650ff95e00 YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 
(cherry picked from commit 0a55bd841ec0f2eb89a0383f4c589526e8b138d4)
 2017-02-23 17:22:25 -08:00
Karthik Kambatla
40bc9e7ddb YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha) 
(cherry picked from commit 694e680d20dc07f634b539537021b09d9316601c)
 2017-02-23 15:30:29 -08:00
Wangda Tan
a82480c7d4 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-23 13:59:20 -08:00
Sunil G
801357726d YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt. 
(cherry picked from commit a207aa9930e7ee4f10228e2db4b4e733794eb8ea)
 2017-02-23 14:25:14 +05:30
Karthik Kambatla
37edbd35f1 YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha) 
(cherry picked from commit b10e962224a8ae1c6031a05322b0cc5e564bd078)
 2017-02-22 15:59:11 -08:00
Karthik Kambatla
ca7a6a7365 YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 
(cherry picked from commit 718ad9f6ee93d4145f2bb19b7582ce4e1174feaf)
 2017-02-22 15:52:44 -08:00
Daniel Templeton
93219b6454 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)

(cherry picked from commit c7a36e613053ec8b111146004b887c2f13535469)
 2017-02-17 14:38:06 -08:00
Karthik Kambatla
5575328bb6 YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha) 
(cherry picked from commit dbbfcf74ab44d7bfdc805b63affd0defc57182b8)
 2017-02-17 14:07:47 -08:00