Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,667 Commits 326 Branches 371 Tags
Commit Graph

2554 Commits

Author SHA1 Message Date
Jian He
b7604a22e6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 
(cherry picked from commit 8a9d293dd60f6d51e1574e412d40746ba8175fe1)
 2016-07-06 16:45:32 -07:00
Wangda Tan
e246cf74c9 YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 16:42:35 -07:00
Vinod Kumar Vavilapalli
c3d9ac82af YARN-5224. Added new web-services /containers/{containerid}/logs & /containers/{containerid}/logs/{filename} and using them in "yarn logs" CLI to get logs of finished containers of a running application. Contributed by Xuan Gong. 2016-07-06 14:19:16 -07:00
Jian He
ccb0791904 YARN-5227. Yarn logs command: no need to specify applicationId when 
specifying containerId. Contributed by Gergely Novák

(cherry picked from commit d169f5052fe83debcea7cf2f317dcd990890a857)
 2016-07-06 13:29:12 -07:00
Jian He
46f8204992 YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit d792a90206e940c31d1048e53dc24ded605788bf)
 2016-07-05 18:06:03 -07:00
Vinod Kumar Vavilapalli
56a08983e2 YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 
(cherry picked from commit ce9c006430d13a28bc1ca57c5c70cc1b7cba1692)
 2016-07-05 16:08:57 -07:00
Varun Saxena
516aa58c1b YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-05 23:10:08 +05:30
Ray Chiang
fb9327edae YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang) 
(cherry picked from commit 0a5def155eff4564b5dc7685e7460952f51bbd24)
 2016-07-01 15:55:56 -07:00
Jian He
d2559ca307 YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 
(cherry picked from commit c35a5a7a8d85b42498e6981a6b1f09f2bdd56459)
 2016-07-01 14:29:28 -07:00
Varun Saxena
4837aebca4 YARN-5282. Fix typos in CapacityScheduler documentation. (Ray Chiang via Varun Saxena). 2016-07-01 11:39:26 +05:30
Varun Saxena
527cd9852a YARN-5182. MockNodes.newNodes creates one more node per rack than requested. (Karthik Kambatla via Varun Saxena). 2016-06-30 00:06:17 +05:30
Rohith Sharma K S
d3fc1bddea YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. 
(cherry picked from commit 26b5e6116f392b6be91dca57968259f87554ce33)
 2016-06-29 10:13:17 +05:30
Xuan
b7f18f2815 YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong 
(cherry picked from commit 6ab5aa1c1f82f81726c6daa38b3db90d8c3ad856)
 2016-06-22 21:50:09 -07:00
Arun Suresh
e64d21b297 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 
(cherry picked from commit 99e5dd68d0f44109c169d74824fa45a7396a5990)
 2016-06-22 19:05:28 -07:00
Sangjin Lee
2a79910c22 YARN-4958. The file localization process should allow for wildcards to reduce the application footprint in the state store (Daniel Templeton via sjlee) 
(cherry picked from commit 5107a967fa2558deba11c33a326d4d2e5748f452)
 2016-06-20 10:00:41 -07:00
Junping Du
ea4cab2330 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040a4d98fbac51527dfc9447888cb32ab)
 2016-06-19 17:45:57 -07:00
Karthik Kambatla
25be8105cd YARN-5077. Fix FSLeafQueue#getFairShare() for queues with zero fairshare. (Yufei Gu via kasha) 
(cherry picked from commit 20f2799938c5cad59f59617cdb0f86e137c3c2c0)
 2016-06-17 22:25:04 -07:00
Karthik Kambatla
e9c1155233 YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit. Addendum to fix javac warning. (Arun Suresh via kasha) 
(cherry picked from commit fbbe0bb627c1b5f333eb5d5390e709cc43517420)
 2016-06-17 22:13:18 -07:00
Junping Du
6afa044dda YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 
(cherry picked from commit c35fa4a0e57b99415e284c7c2f3b1a0bebc610c2)
 2016-06-17 08:25:33 -07:00
Wangda Tan
7eb524ddee YARN-4844. Addendum patch, revert few changes to public and evolving to avoid breaking downstream projects as much as possible. 2016-06-16 11:09:56 -07:00
Junping Du
eec703f36f YARN-5083. YARN CLI for AM logs does not give any error message if entered invalid am value. Contributed by Jian He. 
(cherry picked from commit e14ee0d3b55816bed1d27a8caf78001985119e3c)
 2016-06-16 08:56:46 -07:00
Junping Du
6dd34baf3c YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 
(cherry picked from commit 5dfc38ff57669cba9078146e91ed990a1d25a3f0)
 2016-06-15 16:20:07 -07:00
Varun Vasudev
75235149af YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong. 
(cherry picked from commit 25064fb2fb79751cedbb8019900d811e07549ecf)
 2016-06-15 12:43:26 +05:30
Varun Vasudev
5bd39d2f3c YARN-5122. "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete. Contributed by Jian He. 
(cherry picked from commit d6284428e8bd09a06cf18ff81a7064d744ef47d0)
 2016-06-15 12:37:51 +05:30
Wangda Tan
863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Wangda Tan
cd3bdbc70a YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 
(cherry picked from commit eec835ec17f77bf713d90dbc2f77dea45f3ee660)
 2016-06-13 21:57:55 -07:00
Rohith Sharma K S
c540b9bd7b YARN-4989. TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently. Contributed by Ajith S. 
(cherry picked from commit 28b66ae919e348123f4c05a4787c9ec56c087c25)
 2016-06-13 11:23:10 +05:30
Karthik Kambatla
a92be02c2e YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. Addendum to fix compilation. (Arun Suresh via kasha) 2016-06-12 14:28:14 -07:00
Arun Suresh
f3c7a99e33 YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 7dae2b3bc4cebbb186b3edd14e31074be02af329)
 2016-06-12 10:06:18 -07:00
Arun Suresh
fccb641942 YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh) 
(cherry picked from commit 51432779588fdd741b4840601f5db637ec783d92)
 2016-06-12 09:43:19 -07:00
Arun Suresh
5985221b46 YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit (sandflee via asuresh) 
(cherry picked from commit 5279af7cd4afb090da742a96b5786d9dee6224bc)
 2016-06-10 22:34:33 -07:00
Rohith Sharma K S
efdfa68911 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 
(cherry picked from commit e0f4620cc7db3db4b781e6042ab7dd754af28f18)
 2016-06-11 10:31:00 +05:30
Wangda Tan
e91e9e596d YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 
(cherry picked from commit 03fc6b1bb0f5c0844cd5477ffba43de8a14d4d60)
 2016-06-10 09:53:40 -07:00
Wangda Tan
a4094ddb5a YARN-5208. Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled. (Rohith Sharma K S via wangda) 
(cherry picked from commit 244506f9c8953029283aa8e0eb2006ae2f30025e)
 2016-06-10 09:37:35 -07:00
Vinod Kumar Vavilapalli
f6cd1bcf89 YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 
(cherry picked from commit 9378d9428f127eff7acd6c13544016cdbf2d65fb)
 2016-06-09 12:32:53 -07:00
Wangda Tan
aef6e455b1 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-09 10:08:31 -07:00
Naganarasimha
27fc225558 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 
(cherry picked from commit 1500a0a3009e453c9f05a93df7a78b4e185eef30)
 2016-06-09 05:44:41 +05:30
Vinod Kumar Vavilapalli
67dbd4c466 YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 
(cherry picked from commit 5a43583c0bbb9650ea6a9f48d9544ec3ec24b580)
 2016-06-08 09:51:36 -07:00
Arun Suresh
c29fbdd8ac YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 3344ba70e027c929e07bad5e6877c796d41181e9)
 2016-06-08 08:32:17 -07:00
Jason Lowe
c6f33e096e YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 
(cherry picked from commit 8c8a377cac10b086a7ff37ee366b79e6b04d2738)
 2016-06-08 14:13:19 +00:00
Arun Suresh
154c7c343b YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 76f0800c21f49fba01694cbdc870103053da802c)
 2016-06-07 17:17:22 -07:00
Xuan
10f0c0475e YARN-5199. Close LogReader in in AHSWebServices#getStreamingOutput and 
FileInputStream in NMWebServices#getLogs. Contributed by Xuan Gong

(cherry picked from commit 58be55b6e07b94aa55ed87c461f3e5c04cc61630)
 2016-06-07 16:08:54 -07:00
Arun Suresh
6a9f38ebaf Addendum patch for YARN-5180 updating findbugs-exclude.xml 
(cherry picked from commit 8554aee1bef5aff9e49e5e9119d6a7a4abf1c432)
 2016-06-07 16:00:59 -07:00
Rohith Sharma K S
a29f8bdc3d YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit bddea5fe5fe72eee8e2ecfcec616bd8ceb4d72e7)
 2016-06-07 11:23:02 +05:30
Arun Suresh
d3655f4123 YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 
(cherry picked from commit 3a154f75ed85d864b3ffd35818992418f2b6aa59)
 2016-06-06 21:18:53 -07:00
Arun Suresh
f0a869b52a YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 
(cherry picked from commit 7a9b7372a1a917c7b5e1beca7e13c0419e3dbfef)
 2016-06-06 21:07:24 -07:00
Ming Ma
74e2b5efa2 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
 2016-06-06 14:34:47 -07:00
Arun Suresh
8ebd89dcfb YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 
(cherry picked from commit db54670e83a84c1d7deff2c225725687cf9e5f14)
 2016-06-03 14:50:51 -07:00
Vinod Kumar Vavilapalli
029888871f YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 
Made one minor edit for branch-2 patch.

(cherry picked from commit f10ebc67f57a4a2e3cc916c41154ab9b6a4635c9)
 2016-06-03 13:11:17 -07:00
Jian He
7f865bf438 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29794f8e61bebe03b2a7ca4b6743ec92)
 2016-06-03 11:11:20 -07:00