Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,321 Commits 326 Branches 371 Tags
Commit Graph

2385 Commits

Author SHA1 Message Date
Wangda Tan
70a810adc0 YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda) 
(cherry picked from commit 3fdae0a2b66c5fb6853875b66fcf50bc96d6e2e9)
 2017-04-04 14:39:38 -07:00
Eric Payne
4396c8e938 YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe 
(cherry picked from commit 318bfb01bc6793da09e32e9cc292eb63224b6ca2)
 2017-03-31 12:36:01 -05:00
Jason Lowe
3ae529ba54 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 
(cherry picked from commit 4d1fac5df2508011adfc4c5d683beb00fd5ced45)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClientOnRMRestart.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2017-03-31 10:21:44 -05:00
Varun Saxena
3fe7d36e72 YARN-6352. Header injections are possible in application proxy servlet (Naganarasimha G R via Varun Saxena) 2017-03-30 03:49:32 +05:30
Daniel Templeton
c2636468d7 YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 
(cherry picked from commit 640ba1d23fe8b8105bae6d342ddc1c839302f8e5)
 2017-03-29 12:38:50 -07:00
Akira Ajisaka
554bc8342b
YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger. 
(cherry picked from commit 82fb9ce8df7a534a4cbcec624d7b6c0b33e79123)
 2017-03-29 10:37:52 +09:00
Robert Kanter
a65011a128 YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-28 14:33:08 -07:00
Varun Saxena
97c83f2498 YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena) 
(cherry picked from commit 01aca54a22c8586d232a8f79fe9977aeb8d09b83)
 2017-03-29 01:56:59 +05:30
Jason Lowe
bea8f8190f YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter 2017-03-28 09:45:18 -05:00
Wangda Tan
c97da481d7 YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 
(cherry picked from commit cd014d57aa8b896da02b5bcadafbd404bca2bc12)
 2017-03-27 13:59:36 -07:00
Akira Ajisaka
e4320305d3 YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki. 
(cherry picked from commit b63ae854f9f5a900ca860dcb26c30785af32bde0)
 2017-03-27 11:26:23 +09:00
Sunil G
f51ee482a7 YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki. 2017-03-26 17:00:06 +05:30
Daniel Templeton
d4bba011cc YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable 
(Contributed by Laura Adams via Daniel Templeton)

(cherry picked from commit 0a05c5c5989edeba2cffe16e80350245778cefce)
 2017-03-21 14:27:27 -07:00
Varun Saxena
e2cabb4d21 YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena) 
(cherry picked from commit 35034653d02ac8156338d7267e5975d2d66272d5)
 2017-03-20 23:51:15 +05:30
Jason Lowe
75e6b7428e YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 
(cherry picked from commit 4a8e3045027036afebbcb80f23b7a2886e56c255)
 2017-03-17 14:07:48 -05:00
Jason Lowe
a16ba4296e YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 2017-03-16 09:31:20 -05:00
Sunil G
caaa6fa96b YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202 
(cherry picked from commit 2d0e24eb01cec6f4554671b54729717510711c63)
 2017-03-15 14:58:20 +05:30
Ray Chiang
fa59f4e490 YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang) 2017-03-14 14:45:13 -07:00
Junping Du
f254002f1d YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong. 
(cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)
 2017-03-14 02:58:44 -07:00
Jason Lowe
ef95e74eb1 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 
(cherry picked from commit 9649c27864a23ea156bae904368c1d3cf94c6e9d)
 2017-03-10 13:07:20 -06:00
Karthik Kambatla
4218671f3c YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 
(cherry picked from commit a96afae125ba02fb4480542d3fb0891623ee4c37)
 2017-03-09 23:16:19 -08:00
Daniel Templeton
c60cd88ad1 YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton) 2017-03-09 18:18:03 -08:00
Daniel Templeton
6774f9c3c2 YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton) 
(cherry picked from commit 8bc8804be1b96f8a4fc33b5d24dd7f7af93a437e)
 2017-03-09 17:53:24 -08:00
Sidharta S
3b6e5ef917 YARN-5669. Add support for docker pull command (Contribtued by luhuichun) 2017-03-09 16:23:21 -08:00
Jason Lowe
4a1187238a YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)
 2017-03-08 16:47:31 -06:00
Robert Kanter
11d4cdc3a6 YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter) 
(cherry picked from commit 287ba4ffa66212c02e1b1edc8fca53f6368a9efc)
 2017-03-08 10:45:59 -08:00
Sunil G
ec4c2d42fe YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt. 
(cherry picked from commit 1eb81867032b016a59662043cbae50daa52dafa9)
 2017-03-08 12:13:14 +05:30
Daniel Templeton
83fbb051bb YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton) 
(cherry picked from commit e0c239cdbda336e09a35d112d451c2e17d74a3fc)
 2017-03-07 12:01:38 -08:00
Junping Du
a58dfcb421 YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He. 
(cherry picked from commit 8db7a8c3aea3d989361f32cca5b271e9653773b6)
 2017-03-03 14:40:32 -08:00
Ray Chiang
303ee13e3c YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang) 
(cherry picked from commit 30b91ff9540e35195af834d1bd5983114a556c6d)
 2017-03-03 12:57:17 -08:00
Jason Lowe
5eca427da2 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc7603053e3ac1bc2464f9622995017db5245)
 2017-03-03 08:24:05 -06:00
Jian He
c7ddf95d4a YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam 
(cherry picked from commit 235203dffda1482fb38762fde544c4dd9c3e1fa8)
 2017-02-28 13:26:55 -08:00
Xuan
95bd3c3d55 YARN-6189: Improve application status log message when RM restarted when 
app is in NEW state. Contributed by Junping Du

(cherry picked from commit e0bb867c3fa638c9f689ee0b044b400481cf02b5)
 2017-02-28 11:06:07 -08:00
Karthik Kambatla
58eefc71c7 YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha) 
(cherry picked from commit f187d63816584b783fbfe238475c8f37decdb6dc)
 2017-02-27 22:16:32 -08:00
Naganarasimha
1eec911cd9 YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R. 
(cherry picked from commit 5f5b031d1f20cb7f621db41979e963eaa42cf52f)
 2017-02-27 20:42:10 +05:30
Sunil G
68b08e96a0 YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan. 2017-02-27 17:54:47 +05:30
Karthik Kambatla
c24325bbcb YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha) 
(cherry picked from commit fbfe86deea5f2aa857cd13fee913b7becee57f93)
 2017-02-26 20:37:18 -08:00
Karthik Kambatla
a95d3e1fe7 YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha) 
(cherry picked from commit 815d53506fb0c5ca029c993d6b094db2ac0ca6eb)
 2017-02-26 20:31:03 -08:00
Xuan
19c5b3d622 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache 
stores. Contributed by Li Lu

(cherry picked from commit 53d372a2550c970f3dd3c49738af3c1789ae589b)
 2017-02-24 11:00:16 -08:00
Wangda Tan
650ff95e00 YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 
(cherry picked from commit 0a55bd841ec0f2eb89a0383f4c589526e8b138d4)
 2017-02-23 17:22:25 -08:00
Karthik Kambatla
40bc9e7ddb YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha) 
(cherry picked from commit 694e680d20dc07f634b539537021b09d9316601c)
 2017-02-23 15:30:29 -08:00
Wangda Tan
a82480c7d4 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-23 13:59:20 -08:00
Sunil G
801357726d YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt. 
(cherry picked from commit a207aa9930e7ee4f10228e2db4b4e733794eb8ea)
 2017-02-23 14:25:14 +05:30
Karthik Kambatla
37edbd35f1 YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha) 
(cherry picked from commit b10e962224a8ae1c6031a05322b0cc5e564bd078)
 2017-02-22 15:59:11 -08:00
Karthik Kambatla
ca7a6a7365 YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 
(cherry picked from commit 718ad9f6ee93d4145f2bb19b7582ce4e1174feaf)
 2017-02-22 15:52:44 -08:00
Daniel Templeton
93219b6454 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)

(cherry picked from commit c7a36e613053ec8b111146004b887c2f13535469)
 2017-02-17 14:38:06 -08:00
Karthik Kambatla
5575328bb6 YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha) 
(cherry picked from commit dbbfcf74ab44d7bfdc805b63affd0defc57182b8)
 2017-02-17 14:07:47 -08:00
Daniel Templeton
10fd5627bb YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with 
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)

(cherry picked from commit 585168cc1bdb87398d4f388f3eeccd1c71114982)
 2017-02-17 13:14:37 -08:00
Karthik Kambatla
afc8124ff6 YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha) 
(cherry picked from commit a77f432449aad67da31bd8bf8644b71def741bde)
 2017-02-16 15:03:35 -08:00
Rohith Sharma K S
f07b52dc5a YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit a37b3694ead2ab4e8753e68213f31868b68376b8.
 2017-02-16 17:58:38 +05:30