Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,828 Commits 326 Branches 371 Tags
Commit Graph

3150 Commits

Author SHA1 Message Date
Jason Lowe
4a1187238a YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)
 2017-03-08 16:47:31 -06:00
Jian He
ea8d53de6a Revert "HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand" 
This reverts commit 36eda5af75a1c98b507ea497fce51aa936b2c699.
 2017-03-08 13:20:45 -08:00
Jian He
36eda5af75 HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:49:07 -08:00
Robert Kanter
11d4cdc3a6 YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter) 
(cherry picked from commit 287ba4ffa66212c02e1b1edc8fca53f6368a9efc)
 2017-03-08 10:45:59 -08:00
Sunil G
ec4c2d42fe YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt. 
(cherry picked from commit 1eb81867032b016a59662043cbae50daa52dafa9)
 2017-03-08 12:13:14 +05:30
Daniel Templeton
83fbb051bb YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton) 
(cherry picked from commit e0c239cdbda336e09a35d112d451c2e17d74a3fc)
 2017-03-07 12:01:38 -08:00
Jian He
712434e744 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:23 -08:00
Junping Du
a58dfcb421 YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He. 
(cherry picked from commit 8db7a8c3aea3d989361f32cca5b271e9653773b6)
 2017-03-03 14:40:32 -08:00
Ray Chiang
303ee13e3c YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang) 
(cherry picked from commit 30b91ff9540e35195af834d1bd5983114a556c6d)
 2017-03-03 12:57:17 -08:00
Jason Lowe
5eca427da2 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc7603053e3ac1bc2464f9622995017db5245)
 2017-03-03 08:24:05 -06:00
Jian He
c7ddf95d4a YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam 
(cherry picked from commit 235203dffda1482fb38762fde544c4dd9c3e1fa8)
 2017-02-28 13:26:55 -08:00
Xuan
95bd3c3d55 YARN-6189: Improve application status log message when RM restarted when 
app is in NEW state. Contributed by Junping Du

(cherry picked from commit e0bb867c3fa638c9f689ee0b044b400481cf02b5)
 2017-02-28 11:06:07 -08:00
Gera Shegalov
df35ba81fe YARN-1728. Workaround guice3x-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera) 2017-02-28 09:46:57 -08:00
Karthik Kambatla
58eefc71c7 YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha) 
(cherry picked from commit f187d63816584b783fbfe238475c8f37decdb6dc)
 2017-02-27 22:16:32 -08:00
Naganarasimha
1eec911cd9 YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R. 
(cherry picked from commit 5f5b031d1f20cb7f621db41979e963eaa42cf52f)
 2017-02-27 20:42:10 +05:30
Sunil G
68b08e96a0 YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan. 2017-02-27 17:54:47 +05:30
Karthik Kambatla
c24325bbcb YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha) 
(cherry picked from commit fbfe86deea5f2aa857cd13fee913b7becee57f93)
 2017-02-26 20:37:18 -08:00
Karthik Kambatla
a95d3e1fe7 YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha) 
(cherry picked from commit 815d53506fb0c5ca029c993d6b094db2ac0ca6eb)
 2017-02-26 20:31:03 -08:00
Xuan
19c5b3d622 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache 
stores. Contributed by Li Lu

(cherry picked from commit 53d372a2550c970f3dd3c49738af3c1789ae589b)
 2017-02-24 11:00:16 -08:00
Wangda Tan
650ff95e00 YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 
(cherry picked from commit 0a55bd841ec0f2eb89a0383f4c589526e8b138d4)
 2017-02-23 17:22:25 -08:00
Karthik Kambatla
40bc9e7ddb YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha) 
(cherry picked from commit 694e680d20dc07f634b539537021b09d9316601c)
 2017-02-23 15:30:29 -08:00
Wangda Tan
a82480c7d4 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-23 13:59:20 -08:00
Sunil G
801357726d YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt. 
(cherry picked from commit a207aa9930e7ee4f10228e2db4b4e733794eb8ea)
 2017-02-23 14:25:14 +05:30
Karthik Kambatla
37edbd35f1 YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha) 
(cherry picked from commit b10e962224a8ae1c6031a05322b0cc5e564bd078)
 2017-02-22 15:59:11 -08:00
Karthik Kambatla
ca7a6a7365 YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 
(cherry picked from commit 718ad9f6ee93d4145f2bb19b7582ce4e1174feaf)
 2017-02-22 15:52:44 -08:00
Daniel Templeton
93219b6454 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)

(cherry picked from commit c7a36e613053ec8b111146004b887c2f13535469)
 2017-02-17 14:38:06 -08:00
Karthik Kambatla
5575328bb6 YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha) 
(cherry picked from commit dbbfcf74ab44d7bfdc805b63affd0defc57182b8)
 2017-02-17 14:07:47 -08:00
Daniel Templeton
10fd5627bb YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with 
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)

(cherry picked from commit 585168cc1bdb87398d4f388f3eeccd1c71114982)
 2017-02-17 13:14:37 -08:00
Karthik Kambatla
afc8124ff6 YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha) 
(cherry picked from commit a77f432449aad67da31bd8bf8644b71def741bde)
 2017-02-16 15:03:35 -08:00
Li Lu
8fc67e5973 YARN-6177. Yarn client should exit with an informative error message if an incompatible Jersey library is used at client. Contributed by Weiwei Yang. 
(cherry picked from commit 5d339c46f5b16b951afd82afd9e907b9aa2ded9a)
 2017-02-16 13:46:30 -08:00
Rohith Sharma K S
f07b52dc5a YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit a37b3694ead2ab4e8753e68213f31868b68376b8.
 2017-02-16 17:58:38 +05:30
Karthik Kambatla
9fecd5063f YARN-5798. Set UncaughtExceptionHandler for all FairScheduler threads. (Yufei Gu via kasha) 
(cherry picked from commit 74dd14225059322825f706120aa57cf673820daf)
 2017-02-16 00:03:26 -08:00
Karthik Kambatla
fefac1276a YARN-4212. FairScheduler: Can't create a DRF queue under a FAIR policy queue. (Yufei Gu via kasha) 
(cherry picked from commit 11be3f70e029c2324b167563168c8a254d234aef)
 2017-02-15 23:51:51 -08:00
Karthik Kambatla
46b6c95e0a YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha) 
(cherry picked from commit 6c25dbcdc0517a825b92fb16444aa1d3761e160c)
 2017-02-15 23:23:18 -08:00
Junping Du
323782b389 Addendum patch for YARN-6174 - remove deleted file. 
(cherry picked from commit 0fc6f38379f0047afd23ac14abcb5086d65a7f67)
 2017-02-15 09:09:27 -08:00
Junping Du
2aa6f317f6 YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong. 
(cherry picked from commit ce2d5bfa5f84e7e563980796549b56ef1e4bbf1e)
 2017-02-15 09:08:54 -08:00
Varun Saxena
9cab25a63d YARN-6156. AM blacklisting to consider node label partition (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit b7613e0f406fb2b9bd5b1b3c79658e801f63c587)
 2017-02-15 14:58:39 +05:30
Masatake Iwasaki
cba7e2c346 YARN-4753. Use doxia macro to generate in-page TOC of YARN site documentation. (iwasakims) 2017-02-15 13:37:25 +09:00
Karthik Kambatla
58acb676a2 YARN-6061. Addendum. Remove extraneous change. 
(cherry picked from commit 353a9b2d9165a221491395edbadf8acc3a39990b)
 2017-02-14 15:20:11 -08:00
Karthik Kambatla
37921b3fef YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha) 
(cherry picked from commit 652679aa8ad6f9e61b8ed8e2b04b3e0332025e94)
 2017-02-14 13:40:06 -08:00
Karthik Kambatla
30ff5bff1a YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha) 
(cherry picked from commit 646c6d6509f515b1373288869fb92807fa2ddc9b)
 2017-02-13 11:27:02 -08:00
Junping Du
205fc1f10c YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 
(cherry picked from commit 464ff479ceec76609dca3539081de6b503b17325)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestNMWebServices.java
 2017-02-13 06:16:04 -08:00
Daniel Templeton
7c3578ca17 YARN-6166. Unnecessary INFO logs in AMRMClientAsyncImpl$CallbackHandlerThread.run (Contributed by Grant Whiteheart via Daniel Templeton) 
(cherry picked from commit 1238fa71f8e62cecde6e2437d1b83ad3922295f8)
 2017-02-10 09:48:50 -08:00
Sunil G
7b2b6caf12 YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres Perez. 
(cherry picked from commit 3a0a0a485a4145e3bd2e5946731ee1d901b51ead)
 2017-02-10 15:54:59 +05:30
Karthik Kambatla
e259a818ad YARN-6144. FairScheduler: preempted resources can become negative. (Miklos Szegedi via kasha) 
(cherry picked from commit 08f93978f3ec724b24a93d7ef538f158da75802f)
 2017-02-09 14:45:21 -08:00
Karthik Kambatla
41361ec39e YARN-6112. UpdateCallDuration is calculated only when debug logging is enabled. (Yufei Gu via kasha) 2017-02-09 13:12:21 -08:00
Jason Lowe
e330c3806c YARN-6137. Yarn client implicitly invoke ATS client which accesses HDFS. Contributed by Li Lu 
(cherry picked from commit 37b4acf7cee1f05599a84bbb1ebf07979a71f82f)
 2017-02-08 15:00:50 -06:00
Junping Du
33e7c3044f YARN-6145. Improve log message on fail over. Contributed by Jian He. 
(cherry picked from commit eec52e158b7bc14b2d3d53512323ba05e15e09e3)
 2017-02-08 02:15:11 -08:00
Sidharta S
127ed38d13 YARN-5258. Document Use of Docker with LinuxContainerExecutor (Contributed by Daniel Templeton) 2017-02-07 16:35:56 -08:00
Junping Du
ab41da7e64 YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong. 
(cherry picked from commit 9dbfab1284015af6983b4a55b79c9e65bc0194e5)
 2017-02-07 02:14:09 -08:00