Commit Graph

3583 Commits

Author SHA1 Message Date
Chris Nauroth 725e013359 YARN-1331. yarn.cmd exits with NoClassDefFoundError trying to run rmadmin or logs. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534499 13f79535-47bb-0310-9956-ffa450edef68
2013-10-22 05:38:42 +00:00
Sanford Ryza dc523bd182 YARN-1288. Make Fair Scheduler ACLs more user friendly (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534315 13f79535-47bb-0310-9956-ffa450edef68
2013-10-21 18:45:38 +00:00
Sanford Ryza 65a9e3aa00 YARN-1258: Move to 2.2.1 in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534310 13f79535-47bb-0310-9956-ffa450edef68
2013-10-21 18:39:02 +00:00
Sanford Ryza 881d1d5c7e Reverting "YARN-1258: Move to 2.2.1 in CHANGES.txt" because it contained unintended changes
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534308 13f79535-47bb-0310-9956-ffa450edef68
2013-10-21 18:37:32 +00:00
Sanford Ryza 65bda6d168 YARN-1258: Move to 2.2.1 in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534307 13f79535-47bb-0310-9956-ffa450edef68
2013-10-21 18:31:57 +00:00
Vinod Kumar Vavilapalli 7f97fd1319 YARN-1185. Fixed FileSystemRMStateStore to not leave partial files that prevent subsequent ResourceManager recovery. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533803 13f79535-47bb-0310-9956-ffa450edef68
2013-10-19 18:14:14 +00:00
Sanford Ryza 6a55ed333a YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text file busy errors. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532532 13f79535-47bb-0310-9956-ffa450edef68
2013-10-15 21:14:47 +00:00
Sanford Ryza 989db9ee55 YARN-1182. MiniYARNCluster creates and inits the RM/NM only on start() (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532109 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 21:42:31 +00:00
Sanford Ryza cd7e7c3cdd YARN-1259. In Fair Scheduler web UI, queue num pending and num active apps switched. (Robert Kanter via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532094 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 21:06:01 +00:00
Chris Nauroth 11716cf390 HADOOP-10040. Revert svn propset for CRLF line endings on Windows files. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531977 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 17:01:33 +00:00
Sanford Ryza 2caa899882 YARN-305. Fair scheduler logs too many "Node offered to app" messages. (Lohit Vijayarenu via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531817 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 08:51:34 +00:00
Chris Nauroth bbac0cf05e HADOOP-10040. hadoop.cmd in UNIX format and would not run by default on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531491 13f79535-47bb-0310-9956-ffa450edef68
2013-10-12 03:14:24 +00:00
Sanford Ryza 5bed5bb829 YARN-1044. used/min/max resources do not display info in the scheduler page (Sangjin Lee via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531447 13f79535-47bb-0310-9956-ffa450edef68
2013-10-11 22:13:57 +00:00
Sanford Ryza 156b9f2e88 YARN-1300. SLS tests fail because conf puts YARN properties in fair-scheduler.xml (Ted Yu via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531423 13f79535-47bb-0310-9956-ffa450edef68
2013-10-11 21:18:48 +00:00
Luke Lu 735d8b27f7 YARN-7. Support CPU resource for DistributedShell. (Junping Du via llu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531222 13f79535-47bb-0310-9956-ffa450edef68
2013-10-11 08:18:57 +00:00
Sanford Ryza f218527fff YARN-1265. Fair Scheduler chokes on unhealthy node reconnect (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531146 13f79535-47bb-0310-9956-ffa450edef68
2013-10-10 22:49:56 +00:00
Devarajulu K 22b332ff46 YARN-879. Fixed tests w.r.t o.a.h.y.server.resourcemanager.Application. Contributed by Junping Du.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530902 13f79535-47bb-0310-9956-ffa450edef68
2013-10-10 09:47:11 +00:00
Vinod Kumar Vavilapalli 9b9ddf29e2 YARN-1283. Fixed RM to give a fully-qualified proxy URL for an application so that clients don't need to do scheme-mangling. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530819 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 22:49:06 +00:00
Alejandro Abdelnur a184fceddc Amending yarn CHANGES.txt moving YARN-1284 to 2.2.1
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530716 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 17:11:37 +00:00
Daryn Sharp b858b812b7 HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules (Ivan A. Veselovsky via daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530667 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 15:09:37 +00:00
Sanford Ryza 643b1a4019 YARN-1258. Allow configuring the Fair Scheduler root queue (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530542 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 09:19:14 +00:00
Sanford Ryza 389d846e77 Fix position of YARN-461 in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530505 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 06:49:24 +00:00
Sanford Ryza 00388655e5 YARN-976. Document the meaning of a virtual core. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530500 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 05:56:23 +00:00
Sanford Ryza 5a87d89366 Add missing file TestCgroupsLCEResourcesHandler for YARN-1284.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530493 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 05:07:41 +00:00
Sanford Ryza 726c3538a7 YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed containers. (Alejandro Abdelnur via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530492 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 05:05:17 +00:00
Arun Murthy 418773f44e Moving recent bug-fixes to hadoop-2.2.1.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530128 13f79535-47bb-0310-9956-ffa450edef68
2013-10-08 02:03:29 +00:00
Jason Darrell Lowe 3de419a478 YARN-465. fix coverage org.apache.hadoop.yarn.server.webproxy. Contributed by Aleksey Gorshkov and Andrey Klochkov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530095 13f79535-47bb-0310-9956-ffa450edef68
2013-10-07 22:21:38 +00:00
Arun Murthy 43c7a834dc Preparing for hadoop-2.2.0 release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529750 13f79535-47bb-0310-9956-ffa450edef68
2013-10-07 05:17:00 +00:00
Vinod Kumar Vavilapalli 21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 18:43:36 +00:00
Vinod Kumar Vavilapalli 44f7ee7192 YARN-1278. Fixed NodeManager to not delete local resources for apps on resync command from RM - a bug caused by YARN-1149. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529657 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 18:32:16 +00:00
Vinod Kumar Vavilapalli 52b0ce3565 YARN-1274. Fixed NodeManager's LinuxContainerExecutor to create user, app-dir and log-dirs correctly even when there are no resources to localize for the container. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529555 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 00:14:55 +00:00
Arun Murthy 5e2d93b4d3 YARN-1090. Fixed CS UI to better reflect applications as non-schedulable and not as pending. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529538 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 22:26:42 +00:00
Arun Murthy d55f3780fb YARN-1032. Fixed NPE in RackResolver. Contributed by Lohit Vijayarenu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529534 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 22:20:18 +00:00
Sanford Ryza 25361d56cf Fix location of YARN-1268 in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529531 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 22:10:14 +00:00
Sanford Ryza 54e54d45b3 YARN-1268. TestFairScheduer.testContinuousScheduling is flaky (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529529 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 22:04:53 +00:00
Vinod Kumar Vavilapalli be3edccf0a YARN-1273. Fixed Distributed-shell to account for containers that failed to start. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529389 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 06:08:48 +00:00
Vinod Kumar Vavilapalli 0a887a0910 YARN-1254. Fixed NodeManager to not pollute container's credentials. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529382 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 04:25:44 +00:00
Vinod Kumar Vavilapalli fc23fd3121 YARN-1167. Fixed Distributed Shell to not incorrectly show empty hostname on RM UI. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529376 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 01:43:29 +00:00
Hitesh Shah f104665f7e YARN-1251. TestDistributedShell#TestDSShell failed with timeout. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529369 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 00:48:03 +00:00
Alejandro Abdelnur 80c5bffc4b YARN-1253. Changes to LinuxContainerExecutor to run containers as a single dedicated user in non-secure mode. (rvs via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529325 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 21:59:43 +00:00
Bikas Saha cbab04727b YARN-1232. Configuration to support multiple RMs (Karthik Kambatla via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529251 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 18:40:18 +00:00
Chris Nauroth 6be30a7799 YARN-1219. FSDownload changes file suffix making FileUtil.unTar() throw exception. Contributed by Shanyu Zhao.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529084 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 05:59:07 +00:00
Hitesh Shah 6ff600d9e3 YARN-1131. logs command should return an appropriate error message if YARN application is still running. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529068 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 02:36:17 +00:00
Sanford Ryza d8ea364d07 YARN-1271. Text file busy errors launching containers again (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529058 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 01:30:19 +00:00
Bikas Saha cc76f70fe3 Addendum for missing file YARN-1256. NM silently ignores non-existent service in StartContainerRequest (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529048 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 00:57:18 +00:00
Hitesh Shah d3841bd499 YARN-1149. NM throws InvalidStateTransitonException: Invalid event: APPLICATION_LOG_HANDLING_FINISHED at RUNNING. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529043 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 00:43:26 +00:00
Bikas Saha 8ebf37f369 YARN-1256. NM silently ignores non-existent service in StartContainerRequest (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529039 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 00:23:35 +00:00
Sanford Ryza ac2cdb5f65 YARN-1236. FairScheduler setting queue name in RMApp is not working. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529034 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 23:29:39 +00:00
Vinod Kumar Vavilapalli 1608d8b527 YARN-621. Changed YARN web app to not add paths that can cause duplicate additions of authenticated filters there by causing kerberos replay errors. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529030 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 23:21:32 +00:00
Arun Murthy 79a11ce09d YARN-890. Ensure CapacityScheduler doesn't round-up metric for available resources. Contributed by Xuan Gong & Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529015 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:54:35 +00:00
Jonathan Turner Eagles 7b687dda09 YARN-1199. Make NM/RM Versions Available (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529003 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:04:12 +00:00
Jonathan Turner Eagles dbdb8c6f1f Revert YARN-677. Increase coverage to FairScheduler (Vadim Bondarev and Dennis Y via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528914 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 15:52:30 +00:00
Sanford Ryza 2db1d4d4ee YARN-1213. Restore config to ban submitting to undeclared pools in the Fair Scheduler. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528696 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 01:17:48 +00:00
Sanford Ryza 86cc60c43b YARN-876. Node resource is added twice when node comes back from unhealthy. (Peng Zhang via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528660 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 22:33:47 +00:00
Jonathan Turner Eagles ed7278c59a YARN-425. coverage fix for yarn api (Aleksey Gorshkov via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528641 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 21:51:05 +00:00
Bikas Saha 0a6e275ee3 YARN-1141. Updating resource requests should be decoupled with updating blacklist (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528632 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 21:31:47 +00:00
Jonathan Turner Eagles d14584dec5 YARN-677. Increase coverage to FairScheduler (Vadim Bondarev and Dennis Y via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528524 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 15:45:11 +00:00
Vinod Kumar Vavilapalli cb44a4b850 YARN-1260. Added webapp.http.address to yarn-default.xml so that default install with https enabled doesn't have broken link on NM UI. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528312 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 06:31:27 +00:00
Sanford Ryza 4b9fa6af9d YARN-1262. TestApplicationCleanup relies on all containers assigned in a single heartbeat (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528243 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:42:50 +00:00
Chris Nauroth 85a1bd107a YARN-1215. Correct CHANGES.txt.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528239 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:37:42 +00:00
Chris Nauroth b2a7811fe7 YARN-1215. Yarn URL should include userinfo. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528233 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:28:02 +00:00
Sanford Ryza aa2745abe5 YARN-1228. Clean up Fair Scheduler configuration loading. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528201 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 20:14:18 +00:00
Sanford Ryza ae05623a75 YARN-1010. FairScheduler: decouple container scheduling from nodemanager heartbeats. (Wei Yan via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528192 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 19:54:50 +00:00
Vinod Kumar Vavilapalli 74d20250ff YARN-1070. Fixed race conditions in NodeManager during container-kill. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527827 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 00:18:09 +00:00
Alejandro Abdelnur 84e4e4b143 YARN-1247. test-container-executor has gotten out of sync with the changes to container-executor. (rvs via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527813 13f79535-47bb-0310-9956-ffa450edef68
2013-09-30 22:55:20 +00:00
Sanford Ryza 09bb12c394 YARN-1221. With Fair Scheduler, reserved MB reported in RM web UI increases indefinitely (Siqi Li via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527794 13f79535-47bb-0310-9956-ffa450edef68
2013-09-30 21:59:21 +00:00
Vinod Kumar Vavilapalli 1c5b49eeaf YARN-899. Added back queue level administrator-acls so that there is no regression w.r.t 1.x. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527282 13f79535-47bb-0310-9956-ffa450edef68
2013-09-29 08:21:36 +00:00
Vinod Kumar Vavilapalli 4ce930ea44 YARN-1246. Added application finish-status to ApplicationSummary for the sake of testing given ApplicationHistoryServer is not yet ready. Contributed by Arpit Gupta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527205 13f79535-47bb-0310-9956-ffa450edef68
2013-09-28 16:29:58 +00:00
Alejandro Abdelnur 58b08e11b9 YARN-1021. Yarn Scheduler Load Simulator. (ywskycn via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527059 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 20:23:19 +00:00
Arun Murthy 96c11fe602 MAPREDUCE-5514. Fix TestRMContainerAllocator. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526735 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 23:02:51 +00:00
Jonathan Turner Eagles 5a15c392a1 YARN-819. ResourceManager and NodeManager should check for a minimum allowed version (Robert Parker via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526660 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 19:55:26 +00:00
Colin McCabe 8a66e493ba HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping. (Junping Du via Colin Patrick McCabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526567 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 15:24:30 +00:00
Vinod Kumar Vavilapalli db5322695a YARN-1157. Fixed ResourceManager UI to behave correctly when apps like distributed-shell do not set tracking urls. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526371 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 04:18:00 +00:00
Hitesh Shah 42c3cd3d13 YARN-49. Improve distributed shell application to work on a secure cluster. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526330 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 23:39:17 +00:00
Bikas Saha 929f96ee14 YARN-1214. Register ClientToken MasterKey in SecretManager after it is saved (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526078 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 01:07:59 +00:00
Siddharth Seth 10a4289ebf YARN-1229. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526065 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:35:03 +00:00
Vinod Kumar Vavilapalli 78e995a37d YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525947 13f79535-47bb-0310-9956-ffa450edef68
2013-09-24 17:20:18 +00:00
Sanford Ryza 3175cf17c7 Fix credit for YARN-1188
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525518 13f79535-47bb-0310-9956-ffa450edef68
2013-09-23 07:11:01 +00:00
Sanford Ryza 6d030abe8c YARN-1188. The context of QueueMetrics becomes default when using FairScheduler (Akira Ajisaka via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525516 13f79535-47bb-0310-9956-ffa450edef68
2013-09-23 07:06:05 +00:00
Sanford Ryza 9f6cf60ae0 YARN-1128. FifoPolicy.computeShares throws NPE on empty list of Schedulables (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525313 13f79535-47bb-0310-9956-ffa450edef68
2013-09-22 01:46:58 +00:00
Jing Zhao ab22245dc8 HDFS-5231. Fix broken links in the document of HDFS Federation. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525133 13f79535-47bb-0310-9956-ffa450edef68
2013-09-20 20:34:55 +00:00
Vinod Kumar Vavilapalli 13420d01f2 YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524864 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 23:35:58 +00:00
Hitesh Shah 8628c1704b YARN-353. Add Zookeeper-based store implementation for RMStateStore. Contributed by Bikas Saha, Jian He and Karthik Kambatla.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524829 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 20:39:10 +00:00
Arun Murthy 492ecd34c4 Preparing for hadoop-2.1.1-beta release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523890 13f79535-47bb-0310-9956-ffa450edef68
2013-09-17 05:16:21 +00:00
Christopher Douglas 90f43b1c18 YARN-1184. ClassCastException during preemption enforcement.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523863 13f79535-47bb-0310-9956-ffa450edef68
2013-09-17 01:31:09 +00:00
Vinod Kumar Vavilapalli 4ea295016e YARN-1001. Added a web-service to get statistics about per application-type per state for consumption by downstream projects. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523855 13f79535-47bb-0310-9956-ffa450edef68
2013-09-17 00:24:17 +00:00
Bikas Saha ead3dcc3de YARN-1027. Implement RMHAProtocolService (Karthik Kambatla via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523750 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 18:33:59 +00:00
Arun Murthy 60a1a70197 YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contibuted by Binglin Chang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523652 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:04:04 +00:00
Alejandro Abdelnur 89c7ee9245 YARN-1137. Add support whitelist for system users to Yarn container-executor.c. (rvs via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523587 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 11:01:28 +00:00
Bikas Saha ec010a2936 YARN-540. Race condition causing RM to potentially relaunch already unregistered AMs on RM restart (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523376 13f79535-47bb-0310-9956-ffa450edef68
2013-09-15 01:01:06 +00:00
Jason Darrell Lowe baea09be61 YARN-1189. NMTokenSecretManagerInNM is not being told when applications have finished. Contributed by Omkar Vinit Joshi
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523158 13f79535-47bb-0310-9956-ffa450edef68
2013-09-14 00:18:29 +00:00
Bikas Saha f4951e0708 YARN-1116. Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523146 13f79535-47bb-0310-9956-ffa450edef68
2013-09-13 23:51:50 +00:00
Jason Darrell Lowe 2950c9ff6e YARN-1194. TestContainerLogsPage fails with native builds. Contributed by Roman Shaposhnik
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522968 13f79535-47bb-0310-9956-ffa450edef68
2013-09-13 15:14:07 +00:00
Chris Nauroth f152a7e788 YARN-1078. TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522644 13f79535-47bb-0310-9956-ffa450edef68
2013-09-12 15:58:34 +00:00
Thomas Graves c22091aecb YARN-1176. RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes (Jonathan Eagles via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1522062 13f79535-47bb-0310-9956-ffa450edef68
2013-09-11 22:00:08 +00:00
Chris Nauroth b5c6ff164a YARN-1025. ResourceManager and NodeManager do not load native libraries on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521670 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 22:51:27 +00:00
Jonathan Turner Eagles 220b21732b YARN-1119. Add ClusterMetrics checks to tho TestRMNodeTransitions tests (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521611 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 19:53:49 +00:00
Bikas Saha bcb865314f YARN-1098. Separate out RM services into Always On and Active (Karthik Kambatla via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521560 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 17:43:51 +00:00
Alejandro Abdelnur 11944e52fd Amending yarn CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521454 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 12:14:49 +00:00
Vinod Kumar Vavilapalli 59b5490989 YARN-292. Fixed FifoScheduler and FairScheduler to make their applications data structures thread safe to avoid RM crashing with ArrayIndexOutOfBoundsException. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521328 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 01:24:45 +00:00
Vinod Kumar Vavilapalli 0f91d8485a YARN-910. Augmented auxiliary services to listen for container starts and completions in addition to application events. Contributed by Alejandro Abdelnur.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521298 13f79535-47bb-0310-9956-ffa450edef68
2013-09-09 21:48:09 +00:00
Vinod Kumar Vavilapalli 1a649aa51a YARN-1152. Fixed a bug in ResourceManager that was causing clients to get invalid client token key errors when an appliation is about to finish. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521292 13f79535-47bb-0310-9956-ffa450edef68
2013-09-09 21:19:07 +00:00
Alejandro Abdelnur f2e0a125f4 YARN-1144. Unmanaged AMs registering a tracking URI should not be proxy-fied. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521039 13f79535-47bb-0310-9956-ffa450edef68
2013-09-09 11:12:45 +00:00
Alejandro Abdelnur 1cd7b067f7 YARN-1049. ContainerExistStatus should define a status for preempted containers. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1521036 13f79535-47bb-0310-9956-ffa450edef68
2013-09-09 11:07:49 +00:00
Vinod Kumar Vavilapalli 03ae82a311 YARN-696. Changed RMWebservice apps call to take in multiple application states. Contributed by Trevor Lorimer.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520736 13f79535-47bb-0310-9956-ffa450edef68
2013-09-07 07:28:19 +00:00
Vinod Kumar Vavilapalli f35983b805 YARN-1107. Fixed a bug in ResourceManager because of which RM in secure mode fails to restart. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520726 13f79535-47bb-0310-9956-ffa450edef68
2013-09-07 05:43:04 +00:00
Vinod Kumar Vavilapalli fc95998117 Fixing CHANGES.txt for YARN-758 as it is now merged into branch-2.1-beta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520659 13f79535-47bb-0310-9956-ffa450edef68
2013-09-06 18:30:00 +00:00
Vinod Kumar Vavilapalli 1e513bfc68 YARN-957. Fixed a bug in CapacityScheduler because of which requests that need more than a node's total capability were incorrectly allocated on that node causing apps to hang. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520187 13f79535-47bb-0310-9956-ffa450edef68
2013-09-05 01:20:03 +00:00
Bikas Saha 49afc64cd4 YARN-1065. NM should provide AuxillaryService data to the container (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1520135 13f79535-47bb-0310-9956-ffa450edef68
2013-09-04 20:42:11 +00:00
Vinod Kumar Vavilapalli 2d525510b4 YARN-1124. Modified YARN CLI application list to display new and submitted applications together with running apps by default, following up YARN-1074. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519869 13f79535-47bb-0310-9956-ffa450edef68
2013-09-03 22:18:46 +00:00
Vinod Kumar Vavilapalli ff69557040 YARN-1077. Fixed TestContainerLaunch test failure on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519333 13f79535-47bb-0310-9956-ffa450edef68
2013-09-02 03:10:39 +00:00
Vinod Kumar Vavilapalli 66841c29db YARN-1120. Made ApplicationConstants.Environment.USER definition OS neutral as the corresponding value is now set correctly end-to-end. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519330 13f79535-47bb-0310-9956-ffa450edef68
2013-09-02 02:31:27 +00:00
Vinod Kumar Vavilapalli a759abcd89 YARN-649. Added a new NM web-service to serve container logs in plain text over HTTP. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519326 13f79535-47bb-0310-9956-ffa450edef68
2013-09-02 00:09:17 +00:00
Vinod Kumar Vavilapalli 76cb07ee20 YARN-981. Fixed YARN webapp so that /logs servlet works like before. Addendum patch to fix bugs in the first patch. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519208 13f79535-47bb-0310-9956-ffa450edef68
2013-08-31 19:55:37 +00:00
Sanford Ryza 236b8530bd HADOOP-9918. Add addIfService to CompositeService (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519129 13f79535-47bb-0310-9956-ffa450edef68
2013-08-31 02:09:45 +00:00
Vinod Kumar Vavilapalli 6d100eb794 YARN-1117. Improved help messages for "yarn application" and "yarn node" commands. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519117 13f79535-47bb-0310-9956-ffa450edef68
2013-08-30 23:26:01 +00:00
Bikas Saha bafd302208 YARN-771. AMRMClient support for resource blacklisting (Junping Du via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1519107 13f79535-47bb-0310-9956-ffa450edef68
2013-08-30 22:58:41 +00:00
Vinod Kumar Vavilapalli 89fb4d8ffd YARN-707. Added user information also in the YARN ClientToken so that AMs can implement authorization based on incoming users. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518868 13f79535-47bb-0310-9956-ffa450edef68
2013-08-30 01:12:40 +00:00
Vinod Kumar Vavilapalli febedd64e9 YARN-1080. Improved help message for "yarn logs" command. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518731 13f79535-47bb-0310-9956-ffa450edef68
2013-08-29 17:55:30 +00:00
Sanford Ryza c13893d5d9 YARN-1034. Remove "experimental" in the Fair Scheduler documentation. (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518444 13f79535-47bb-0310-9956-ffa450edef68
2013-08-29 02:22:54 +00:00
Thomas Graves 56c1b9de0c YARN-1101. Active nodes can be decremented below 0 (Robert Parker via tgraves_
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518384 13f79535-47bb-0310-9956-ffa450edef68
2013-08-28 21:09:08 +00:00
Vinod Kumar Vavilapalli 53f559dd79 YARN-1081. Made a trivial change to YARN node CLI header to avoid potential confusion. Contributed by Akira AJISAKA.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518080 13f79535-47bb-0310-9956-ffa450edef68
2013-08-28 05:46:09 +00:00
Vinod Kumar Vavilapalli 4047ad72cf YARN-602. Fixed NodeManager to not let users override some mandatory environmental variables. Contributed by Kenji Kikushima.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518077 13f79535-47bb-0310-9956-ffa450edef68
2013-08-28 05:13:21 +00:00
Vinod Kumar Vavilapalli ca5de53bc6 YARN-1083. Changed ResourceManager to fail when the expiry interval is less than the configured node-heartbeat interval. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518036 13f79535-47bb-0310-9956-ffa450edef68
2013-08-27 23:54:02 +00:00
Vinod Kumar Vavilapalli 914a0e5172 YARN-981. Fixed YARN webapp so that /logs servlet works like before. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1518030 13f79535-47bb-0310-9956-ffa450edef68
2013-08-27 23:36:42 +00:00
Vinod Kumar Vavilapalli 5adba5597c YARN-1085. Addendum patch to address issues with the earlier patch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517721 13f79535-47bb-0310-9956-ffa450edef68
2013-08-26 22:59:09 +00:00
Sanford Ryza f9f7b792fb YARN-942. In Fair Scheduler documentation, inconsistency on which properties have prefix (Akira Ajisaka via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517691 13f79535-47bb-0310-9956-ffa450edef68
2013-08-26 20:47:54 +00:00
Sanford Ryza e7154d7855 YARN-1093. Corrections to Fair Scheduler documentation (Wing Yew Poon via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517658 13f79535-47bb-0310-9956-ffa450edef68
2013-08-26 19:52:10 +00:00
Alejandro Abdelnur 942e2ebaa5 YARN-1008. MiniYARNCluster with multiple nodemanagers, all nodes have same key for allocations. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517563 13f79535-47bb-0310-9956-ffa450edef68
2013-08-26 15:39:11 +00:00
Vinod Kumar Vavilapalli 18e805677d YARN-1094. Fixed a blocker with RM restart code because of which RM crashes when try to recover an existing app. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517215 13f79535-47bb-0310-9956-ffa450edef68
2013-08-24 23:32:41 +00:00
Vinod Kumar Vavilapalli 962da4dcc7 YARN-1074. Cleaned up YARN CLI application list to only display running applications by default. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517196 13f79535-47bb-0310-9956-ffa450edef68
2013-08-24 21:16:40 +00:00
Vinod Kumar Vavilapalli e86036662c YARN-1085. Modified YARN and MR2 web-apps to do HTTP authentication in secure setup with kerberos. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517101 13f79535-47bb-0310-9956-ffa450edef68
2013-08-24 02:47:47 +00:00
Jason Darrell Lowe c660339c09 Revert MAPREDUCE-5475 and YARN-707
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517097 13f79535-47bb-0310-9956-ffa450edef68
2013-08-24 01:15:37 +00:00
Sanford Ryza d912eea822 YARN-905. Add state filters to nodes CLI (Wei Yan via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517083 13f79535-47bb-0310-9956-ffa450edef68
2013-08-23 22:46:08 +00:00
Jason Darrell Lowe f1638fdf94 YARN-707. Add user info in the YARN ClientToken. Contributed by Vinod Kumar Vavilapalli
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1517073 13f79535-47bb-0310-9956-ffa450edef68
2013-08-23 22:11:54 +00:00
Arun Murthy 487ce6c7bc YARN-1082. Addendum patch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516352 13f79535-47bb-0310-9956-ffa450edef68
2013-08-22 01:19:26 +00:00
Arun Murthy ad0011a340 YARN-1082. Create base directories on HDFS after RM login to ensure RM recovery doesn't fail in secure mode. Contributed by Vinod K V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516337 13f79535-47bb-0310-9956-ffa450edef68
2013-08-21 23:55:04 +00:00
Bikas Saha 42a2846b3b YARN-881. Priority#compareTo method seems to be wrong. (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516331 13f79535-47bb-0310-9956-ffa450edef68
2013-08-21 23:33:59 +00:00
Luke Lu 782191f1ba HADOOP-9784. Add a builder for HttpServer. (Junping Du via llu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1516128 13f79535-47bb-0310-9956-ffa450edef68
2013-08-21 10:12:13 +00:00
Vinod Kumar Vavilapalli 8f343e684c YARN-1006. Fixed broken rendering in the Nodes list web page on the RM web UI. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1515629 13f79535-47bb-0310-9956-ffa450edef68
2013-08-19 22:04:18 +00:00
Vinod Kumar Vavilapalli 79a184505d YARN-643. Fixed ResourceManager to remove all tokens consistently on app finish. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1515256 13f79535-47bb-0310-9956-ffa450edef68
2013-08-19 01:53:34 +00:00
Vinod Kumar Vavilapalli 1ad3fe4633 YARN-107. Fixed ResourceManager and clients to better handle forceKillApplication on non-running and finished applications. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514918 13f79535-47bb-0310-9956-ffa450edef68
2013-08-16 23:00:54 +00:00
Arun Murthy a37d2fc89d Preparing for hadoop-2.1.0-beta rc2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514469 13f79535-47bb-0310-9956-ffa450edef68
2013-08-15 20:29:43 +00:00
Siddharth Seth d62bd71d65 Update CHANGES.txt to move YARN-1045 and MAPREDUCE-5352 to the correct version.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514432 13f79535-47bb-0310-9956-ffa450edef68
2013-08-15 18:44:52 +00:00
Siddharth Seth b776bd46ae YARN-1045. Improve toString implementation for PBImpls. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514185 13f79535-47bb-0310-9956-ffa450edef68
2013-08-15 07:20:14 +00:00
Arun Murthy 472d8bc983 YARN-1056. Remove dual use of string 'resourcemanager' in yarn.resourcemanager.connect.{max.wait.secs|retry_interval.secs}. Contributed by Karthik Kambatla.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514135 13f79535-47bb-0310-9956-ffa450edef68
2013-08-15 02:35:48 +00:00
Alejandro Abdelnur 8451ab5c01 HADOOP-9872. Improve protoc version handling and detection. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514068 13f79535-47bb-0310-9956-ffa450edef68
2013-08-14 22:15:04 +00:00
Jason Darrell Lowe 8937fd537a YARN-337. RM handles killed application tracking URL poorly. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513888 13f79535-47bb-0310-9956-ffa450edef68
2013-08-14 14:18:16 +00:00
Sanford Ryza 88f6837fa5 YARN-1060. Two tests in TestFairScheduler are missing @Test annotation (Niranjan Singh via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513724 13f79535-47bb-0310-9956-ffa450edef68
2013-08-14 04:37:06 +00:00
Alejandro Abdelnur 19b36f0900 HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513281 13f79535-47bb-0310-9956-ffa450edef68
2013-08-12 22:15:10 +00:00
Sanford Ryza 109705f077 YARN-1046. Disable mem monitoring my default in MiniYARNCluster (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512493 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 21:10:03 +00:00
Sanford Ryza 49f2a440c3 YARN-656. In scheduler UI, including reserved memory in Memory Total can make it exceed cluster capacity. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512480 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 20:24:36 +00:00
Sanford Ryza 796ed241bb Reverting 1512475 (labeled as YARN-656). Accidentally committed two patches together
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512478 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 20:21:58 +00:00
Sanford Ryza 6a0ba85292 YARN-656. In scheduler UI, including reserved memory in Memory Total can make it exceed cluster capacity. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512475 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 20:17:04 +00:00
Sanford Ryza 1784601bb9 Amending YARN-589. Adding missing file from patch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512112 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 00:33:51 +00:00
Sanford Ryza 5b001e1a8f YARN-589. Expose a REST API for monitoring the fair scheduler (Sandy Ryza).
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512111 13f79535-47bb-0310-9956-ffa450edef68
2013-08-09 00:26:45 +00:00
Arun Murthy e90afcc971 YARN-1043. Push all metrics consistently. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1512081 13f79535-47bb-0310-9956-ffa450edef68
2013-08-08 22:54:31 +00:00
Jonathan Turner Eagles 3e0203a3a4 YARN-985. Nodemanager should log where a resource was localized (Ravi Prakash via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1511100 13f79535-47bb-0310-9956-ffa450edef68
2013-08-06 19:51:33 +00:00
Bikas Saha 41e2518e0c YARN-994. HeartBeat thread in AMRMClientAsync does not handle runtime exception correctly (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1510070 13f79535-47bb-0310-9956-ffa450edef68
2013-08-03 22:31:50 +00:00
Vinod Kumar Vavilapalli ac933234ac YARN-906. Fixed a bug in NodeManager where cancelling ContainerLaunch at KILLING state causes that the container to hang. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509924 13f79535-47bb-0310-9956-ffa450edef68
2013-08-03 00:49:09 +00:00
Vinod Kumar Vavilapalli 5b7889f9a7 YARN-903. Changed ContainerManager to suppress unnecessary warnings when stopping already stopped containers. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509560 13f79535-47bb-0310-9956-ffa450edef68
2013-08-02 06:54:22 +00:00
Jason Darrell Lowe ef4f9b09ac Correct fix versions for MAPREDUCE-5428 and YARN-573.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509475 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 23:54:19 +00:00
Jason Darrell Lowe 4757963d32 YARN-573. Shared data structures in Public Localizer and Private Localizer are not Thread safe. Contributed by Omkar Vinit Joshi
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509389 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 19:34:55 +00:00
Sanford Ryza 1ff7043a1f YARN-758. Augment MockNM to use multiple cores (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509086 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 06:10:48 +00:00
Vinod Kumar Vavilapalli ba5925c96a YARN-502. Fixed a state machine issue with RMNode inside ResourceManager which was crashing scheduler. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1509060 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 00:55:08 +00:00
Vinod Kumar Vavilapalli 813efd25a1 YARN-966. Fixed ContainerLaunch to not fail quietly when there are no localized resources due to some other failure. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508688 13f79535-47bb-0310-9956-ffa450edef68
2013-07-30 23:58:17 +00:00
Vinod Kumar Vavilapalli 48264f1994 YARN-948. Changed ResourceManager to validate the release container list before actually releasing them. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508609 13f79535-47bb-0310-9956-ffa450edef68
2013-07-30 20:30:54 +00:00
Arun Murthy c70178edca Updating release date for hadoop-2.1.0-beta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508402 13f79535-47bb-0310-9956-ffa450edef68
2013-07-30 12:19:35 +00:00
Vinod Kumar Vavilapalli 3ca892dc0a Reverting YARN-245 to fix a critical bug.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508277 13f79535-47bb-0310-9956-ffa450edef68
2013-07-30 03:06:54 +00:00
Vinod Kumar Vavilapalli 817a654346 YARN-945. Removed setting of AMRMToken's service from ResourceManager and changed client libraries do it all the time and correctly. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508232 13f79535-47bb-0310-9956-ffa450edef68
2013-07-29 23:17:39 +00:00
Vinod Kumar Vavilapalli c23cf3cddf YARN-961. Changed ContainerManager to enforce Token auth irrespective of security. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508216 13f79535-47bb-0310-9956-ffa450edef68
2013-07-29 22:23:29 +00:00
Sanford Ryza 9eb06b31d6 YARN-932. TestResourceLocalizationService.testLocalizationInit can fail on JDK7. (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508209 13f79535-47bb-0310-9956-ffa450edef68
2013-07-29 22:06:28 +00:00
Vinod Kumar Vavilapalli 412e19f8e3 YARN-245. Fixed NodeManager to handle duplicate responses from ResourceManager. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508157 13f79535-47bb-0310-9956-ffa450edef68
2013-07-29 18:15:05 +00:00
Luke Lu 13b526b15f YARN-347. Node CLI should show CPU info besides memory in node status. (Junping Du via llu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507959 13f79535-47bb-0310-9956-ffa450edef68
2013-07-29 07:34:19 +00:00
Alejandro Abdelnur f179afc68d YARN-937. Fix unmanaged AM in non-secure/secure setup post YARN-701. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507706 13f79535-47bb-0310-9956-ffa450edef68
2013-07-27 20:04:38 +00:00
Vinod Kumar Vavilapalli ae8f07d47b YARN-960. Fixed ResourceManager to propagate client-submitted credentials irrespective of security. Contributed by Daryn Sharp.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507700 13f79535-47bb-0310-9956-ffa450edef68
2013-07-27 18:50:27 +00:00
Tsz-wo Sze 49d8dceb1f Move YARN-84 to section 2.1.0-beta in CHANGES.txt.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507587 13f79535-47bb-0310-9956-ffa450edef68
2013-07-27 07:03:25 +00:00
Vinod Kumar Vavilapalli db1e0c1eef YARN-688. Fixed NodeManager to properly cleanup containers when it is shut down. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506814 13f79535-47bb-0310-9956-ffa450edef68
2013-07-25 04:14:25 +00:00
Bikas Saha 9c453d4432 YARN-875. Application can hang if AMRMClientAsync callback thread has exception (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506750 13f79535-47bb-0310-9956-ffa450edef68
2013-07-24 22:13:17 +00:00
Bikas Saha ed6598791e YARN-873. YARNClient.getApplicationReport(unknownAppId) returns a null report (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506732 13f79535-47bb-0310-9956-ffa450edef68
2013-07-24 21:34:47 +00:00
Kihwal Lee 605fafb4c0 YARN-968. RM admin commands don't work. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506675 13f79535-47bb-0310-9956-ffa450edef68
2013-07-24 19:43:48 +00:00
Vinod Kumar Vavilapalli 3ca3137179 YARN-926. Modified ContainerManagerProtcol APIs to take in requests for multiple containers. Contributed by Jian He.
MAPREDUCE-5412. Update MR app to use multiple containers API of ContainerManager after YARN-926. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506391 13f79535-47bb-0310-9956-ffa450edef68
2013-07-24 03:41:03 +00:00
Jason Darrell Lowe a42e459b9e MAPREDUCE-5356. Ability to refresh aggregated log retention period and check interval. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506226 13f79535-47bb-0310-9956-ffa450edef68
2013-07-23 19:20:57 +00:00
Alejandro Abdelnur ca35ed13bf YARN-461. Fair scheduler should not accept apps with empty string queue name. (ywskycn via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1505858 13f79535-47bb-0310-9956-ffa450edef68
2013-07-23 00:00:11 +00:00
Vinod Kumar Vavilapalli 5b3bb05fbe YARN-853. Fixed CapacityScheduler's maximum-am-resource-percent to properly work beyond refreshing queues. Contributed by Devaraj K.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1505855 13f79535-47bb-0310-9956-ffa450edef68
2013-07-22 23:49:20 +00:00
Vinod Kumar Vavilapalli 65250d0a62 Fixing YARN CHANGES.txt. The general bug list got mixed up with HADOOP-8562 sub-tasks.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1505847 13f79535-47bb-0310-9956-ffa450edef68
2013-07-22 23:31:40 +00:00
Arun Murthy 5b99672658 YARN-897. Ensure child queues are ordered correctly to account for completed containers. Contributed by Djellel Eddine Difallah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1505146 13f79535-47bb-0310-9956-ffa450edef68
2013-07-20 14:59:41 +00:00
Hitesh Shah 3c6e5b900f YARN-919. Document setting default heap sizes in yarn env.sh Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504970 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 19:23:12 +00:00
Arun Murthy 31f29de05b Fixed CHANGES.txt to reflect that new content in branch-2 goes to hadoop-2.3.0.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504742 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 01:29:07 +00:00
Arun Murthy 87a02553ee Fixed CHANGES.txt to reflect that all fixes in 2.1.1-beta are now in 2.1.0-beta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504738 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 01:03:19 +00:00
Arun Murthy ac914f79bc YARN-918. Remove ApplicationAttemptId from RegisterApplicationMasterRequestProto. Contributed by Vinod K V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504735 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 00:57:40 +00:00
Vinod Kumar Vavilapalli 7a29bccd7a YARN-814. Improving diagnostics when containers fail during launch due to various reasons like invalid env etc. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504732 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 00:28:18 +00:00
Arun Murthy 32bc200d54 YARN-701. Use application tokens irrespective of secure or non-secure mode. Contributed by Vinod K V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504604 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 19:03:48 +00:00
Hitesh Shah 1c133ffb72 YARN-865. RM webservices can't query based on application Types. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504288 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 21:42:24 +00:00
Bikas Saha 9f3e488936 YARN-922. Change FileSystemRMStateStore to use directories (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504261 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 20:19:49 +00:00
Vinod Kumar Vavilapalli c6facfb10d YARN-912. Move client facing exceptions to yarn-api module. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504032 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 07:31:33 +00:00
Vinod Kumar Vavilapalli 8b9c1e68ab YARN-62. Modified NodeManagers to avoid AMs from abusing container tokens for repetitive container launches. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503986 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 04:24:44 +00:00
Bikas Saha 3520039979 YARN-927. Change ContainerRequest to not have more than 1 container count and remove StoreContainerRequest (bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503960 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 00:47:46 +00:00
Vinod Kumar Vavilapalli 6d12709b19 YARN-820. Fixed an invalid state transition in NodeManager caused by failing resource localization. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503947 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 23:44:56 +00:00
Vinod Kumar Vavilapalli af0d2fc3e3 YARN-661. Fixed NM to cleanup users' local directories correctly when starting up. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503942 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 23:31:02 +00:00
Bikas Saha 33b5a81509 YARN-513. Create common proxy client for communicating with RM (Xuan Gong & Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503933 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 22:47:40 +00:00
Bikas Saha 8f8be404a7 Trivial fix for minor refactor error for YARN-521
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503543 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 02:40:57 +00:00
Vinod Kumar Vavilapalli 4466d8653f YARN-523. Modified a test-case to validate container diagnostics on localization failures. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503532 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 00:57:41 +00:00
Bikas Saha 1fe8e2d7ee YARN-521. Augment AM - RM client module to be able to request containers only at specific locations (Sandy Ryza via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503526 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 00:23:18 +00:00
Chris Nauroth 1db4319072 YARN-909. Disable TestLinuxContainerExecutorWithMocks on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503357 13f79535-47bb-0310-9956-ffa450edef68
2013-07-15 17:27:31 +00:00
Bikas Saha fe735f237c YARN-654. AMRMClient: Perform sanity checks for parameters of public methods (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503353 13f79535-47bb-0310-9956-ffa450edef68
2013-07-15 17:13:05 +00:00
Bikas Saha b980f2aa4e YARN-763. AMRMClientAsync should stop heartbeating after receiving shutdown from RM (Xuan Gong via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502914 13f79535-47bb-0310-9956-ffa450edef68
2013-07-13 23:30:20 +00:00
Bikas Saha 2adee63426 YARN-541. getAllocatedContainers() is not returning all the allocated containers (bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502906 13f79535-47bb-0310-9956-ffa450edef68
2013-07-13 22:31:02 +00:00
Alejandro Abdelnur 1714932158 YARN-333. Schedulers cannot control the queue-name of an application. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502374 13f79535-47bb-0310-9956-ffa450edef68
2013-07-11 21:39:28 +00:00
Christopher Douglas 85f0efb68f YARN-569. Add support for requesting and enforcing preemption requests via
a capacity monitor. Contributed by Carlo Curino, Chris Douglas


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502083 13f79535-47bb-0310-9956-ffa450edef68
2013-07-11 01:20:23 +00:00
Alejandro Abdelnur 59a5ab9d1c updating CHANGES.txt after committing MAPREDUCE-5333,HADOOP-9661,HADOOP-9355,HADOOP-9673,HADOOP-9414,HADOOP-9416,HDFS-4797,YARN-866,YARN-736,YARN-883 to 2.1-beta branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502075 13f79535-47bb-0310-9956-ffa450edef68
2013-07-11 00:22:14 +00:00
Chris Nauroth 413bddf596 MAPREDUCE-4374. Fix child task environment variable config and add support for Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502046 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 22:59:08 +00:00
Vinod Kumar Vavilapalli dfe70f6a03 YARN-295. Fixed a race condition in ResourceManager RMAppAttempt state machine. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501856 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:08:43 +00:00
Vinod Kumar Vavilapalli 3ee5949912 YARN-368. Fixed a typo in error message in Auxiliary services. Contributed by Albert Chu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501852 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 17:02:08 +00:00
Bikas Saha 15ce82b9c5 YARN-369. Handle ( or throw a proper error when receiving) status updates from application masters that have not registered (Mayank Bansal & Abhishek Kapoor via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501605 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 23:14:27 +00:00
Hitesh Shah e2cce7bd26 YARN-727, MAPREDUCE-5325. ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501599 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 23:01:01 +00:00
Chris Nauroth 31fff48ec4 YARN-894. NodeHealthScriptRunner timeout checking is inaccurate on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501016 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 23:40:43 +00:00
Vinod Kumar Vavilapalli 5e4f6ad1d9 YARN-791. Changed RM APIs and web-services related to nodes to ensure that both are consistent with each other. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500994 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 22:30:52 +00:00
Bikas Saha eff5d9b17e YARN-845. RM crash with NPE on NODE_UPDATE (Mayank Bansal via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499886 13f79535-47bb-0310-9956-ffa450edef68
2013-07-04 23:31:26 +00:00
Thomas Graves aa7e148d1f Preparing for 0.23.9 release
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1498480 13f79535-47bb-0310-9956-ffa450edef68
2013-07-01 14:29:45 +00:00
Luke Lu 8eb3be63f5 YARN-877. Support resource blacklisting for FifoScheduler. (Junping Du via llu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1498021 13f79535-47bb-0310-9956-ffa450edef68
2013-06-29 20:18:57 +00:00
Alejandro Abdelnur c221204cca YARN-883. Expose Fair Scheduler-specific queue metrics. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1497884 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 18:59:47 +00:00
Arun Murthy c918b5c02c Preparing for hadoop-2.1.0-beta release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496791 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 06:51:50 +00:00
Arun Murthy f3a893b696 YARN-869. Move ResourceManagerAdministrationProtocol out of main YARN api. Contributed by Vinod K V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496751 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 05:57:05 +00:00
Vinod Kumar Vavilapalli b3a8d99817 YARN-874. Making common RPC to switch to not switch to simple when other mechanisms are enabled and thus fix YARN/MR test failures after HADOOP-9421. Contributed by Daryn Sharp and Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496692 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 00:42:26 +00:00
Alejandro Abdelnur e60fbbcc2e YARN-736. Add a multi-resource fair sharing metric. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496153 13f79535-47bb-0310-9956-ffa450edef68
2013-06-24 18:33:45 +00:00
Bikas Saha 87d67c8c55 MiniYARNCluster broken after YARN-827 (bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495684 13f79535-47bb-0310-9956-ffa450edef68
2013-06-22 08:08:21 +00:00
Bikas Saha ba44df9166 Deleting files missed for YARN-827
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495633 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 22:54:04 +00:00
Sanjay Radia 6cb5ad16d0 HADOOP-9619 Mark stability of .proto files (sanjay Radia)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495564 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 19:53:21 +00:00
Bikas Saha 37d7935a9d YARN-827. Need to make Resource arithmetic methods accessible^CJian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495533 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 18:28:57 +00:00
Alejandro Abdelnur 066f970457 YARN-866. Add test for class ResourceWeights. (ywskycn via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495494 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 16:24:53 +00:00
Vinod Kumar Vavilapalli 1a06175440 YARN-851. Share NMTokens using NMTokenCache (api-based) between AMRMClient and NMClient instead of memory based approach which is used currently. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495247 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 00:08:13 +00:00
Hitesh Shah 575495b5b1 YARN-861. TestContainerManager is failing. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495160 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 20:18:04 +00:00
Jonathan Turner Eagles 90fd863cb8 YARN-478. fix coverage org.apache.hadoop.yarn.webapp.log (Aleksey Gorshkov via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495129 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 18:56:12 +00:00
Vinod Kumar Vavilapalli 3f809667bc YARN-854. Fixing YARN bugs that are failing applications in secure environment. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494845 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 03:41:36 +00:00
Chris Nauroth 51c85ca067 YARN-852. TestAggregatedLogFormat.testContainerLogsFileAccess fails on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494733 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 18:45:37 +00:00
Chris Nauroth ed2c56421c YARN-597. Change attribution of YARN-597 from trunk to release 2.1.0-beta in CHANGES.txt. (cnauroth)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494717 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 18:11:03 +00:00
Arun Murthy d3198dddc8 YARN-553. Replaced YarnClient.getNewApplication with YarnClient.createApplication which provides a directly usable ApplicationSubmissionContext to simplify the api. Contributed by Karthik Kambatla.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494476 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 07:06:54 +00:00
Siddharth Seth ab647cfcdb YARN-848. Fix NodeManager to register with RM using the fully qualified hostname. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494385 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 23:49:23 +00:00
Vinod Kumar Vavilapalli 243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 23:19:49 +00:00
Bikas Saha 40cab68409 YARN-850. Rename getClusterAvailableResources to getAvailableResources in AMRMClients (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494309 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 21:14:28 +00:00
Vinod Kumar Vavilapalli 2fd7f28230 YARN-846. Move pb Impl classes from yarn-api to yarn-common. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494052 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 07:36:43 +00:00
Arun Murthy 6451288704 YARN-799. Fix CgroupsLCEResourcesHandler to use /tasks instead of /cgroup.procs. Contributed by Chris Riccomini.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494035 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 06:34:10 +00:00
Vinod Kumar Vavilapalli f4d80e91ae YARN-841. Move Auxiliary service to yarn-api, annotate and document it. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494031 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 06:20:37 +00:00
Arun Murthy acc0d3eb52 YARN-840. Moved ProtoUtils to yarn.api.records.pb.impl. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494027 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 05:47:54 +00:00
Vinod Kumar Vavilapalli 7ef54faad4 YARN-834. Fixed annotations for yarn-client module, reorganized packages and clearly differentiated *Async apis. Contributed by Arun C Murthy and Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494017 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 04:02:47 +00:00
Siddharth Seth 2b14656ab5 YARN-805. Fix javadoc and annotations on classes in the yarn-api package. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493992 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 03:10:11 +00:00
Vinod Kumar Vavilapalli 5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 23:13:17 +00:00
Chris Nauroth b02dc5b464 YARN-839. TestContainerLaunch.testContainerEnvVariables fails on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493937 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 21:05:00 +00:00
Vinod Kumar Vavilapalli 7d0c43d7cf YARN-822. Renamed ApplicationToken to be AMRMToken, and similarly the corresponding TokenSelector and SecretManager. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493889 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 19:11:29 +00:00
Bikas Saha 7801a730e9 Fix hadoop-yarn-project/CHANGES.txt for YARN-759
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493842 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 16:49:50 +00:00
Vinod Kumar Vavilapalli cd30058193 YARN-837. Moved yarn.ClusterInfo into MapReduce project as it doesn't belong to YARN. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493676 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 08:13:48 +00:00
Vinod Kumar Vavilapalli 4662b8b443 YARN-833. Moved Graph and VisualizeStateMachine into yarn.state package. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493654 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 06:51:51 +00:00
Vinod Kumar Vavilapalli b9efe6bd4a HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 06:39:33 +00:00
Vinod Kumar Vavilapalli d4a811edb2 YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 04:49:13 +00:00
Vinod Kumar Vavilapalli f5f8f3bca4 YARN-825. Fixed javadoc and annotations for yarn-common module. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493634 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 03:32:49 +00:00
Arun Murthy ff70f912f7 YARN-824. Added static factory methods to hadoop-yarn-client interfaces. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493631 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 03:19:21 +00:00
Arun Murthy 6b75a5c3b5 YARN-831. Removed minimum resource from GetNewApplicationResponse as a follow-up to YARN-787. Contributed Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493626 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 02:33:11 +00:00
Arun Murthy 77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 02:27:18 +00:00
Alejandro Abdelnur 02f87683e3 YARN-752. In AMRMClient, automatically add corresponding rack requests for requested nodes. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493599 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 22:11:38 +00:00
Vinod Kumar Vavilapalli ac0bbb750f YARN-823. Moved RMAdmin from yarn.client to yarn.client.cli and renamed it to be RMAdminCLI. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493596 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 21:42:59 +00:00
Vinod Kumar Vavilapalli 4aa7503342 YARN-828. Removed the unsed YarnVersionAnnotation. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493592 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 21:34:53 +00:00
Vinod Kumar Vavilapalli e3769ea1d2 YARN-829. Renamed RMTokenSelector to be RMDelegationTokenSelector. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493577 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 20:33:30 +00:00
Arun Murthy 7f96128e43 MAPREDUCE-5184. Document compatibility for MapReduce applications in hadoop-2 vis-a-vis hadoop-1. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493570 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 19:09:48 +00:00
Arun Murthy 98d97d316c YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 13:59:41 +00:00
Vinod Kumar Vavilapalli 769a0bd831 YARN-693. Modified RM to send NMTokens on allocate call so that AMs can then use them for authentication with NMs. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493448 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 03:11:33 +00:00
Vinod Kumar Vavilapalli 29a3708724 YARN-781. Exposing LOGDIR in all containers' environment which should be used by containers for logging purposes. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493428 13f79535-47bb-0310-9956-ffa450edef68
2013-06-15 23:09:24 +00:00
Vinod Kumar Vavilapalli c2d0cf1b20 YARN-821. Renamed setFinishApplicationStatus to setFinalApplicationStatus in FinishApplicationMasterRequest for consistency. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493315 13f79535-47bb-0310-9956-ffa450edef68
2013-06-15 04:57:04 +00:00
Vinod Kumar Vavilapalli b503b6a07d YARN-639. Modified Distributed Shell application to start using the new NMClient library. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493280 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 23:18:34 +00:00
Alejandro Abdelnur b9753e509c YARN-789. Enable zero capabilities resource requests in fair scheduler. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493219 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 19:16:40 +00:00
Alejandro Abdelnur 334de8d211 ARN-803. factor out scheduler config validation from the ResourceManager to each scheduler implementation. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493160 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 17:07:25 +00:00
Vinod Kumar Vavilapalli fdc9412a81 YARN-806. Moved ContainerExitStatus from yarn.api to yarn.api.records. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493138 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 16:10:37 +00:00
Alejandro Abdelnur 710a5eea62 YARN-782. vcores-pcores ratio functions differently from vmem-pmem ratio in misleading way. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493064 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 13:41:45 +00:00
Arun Murthy aa14698fd6 YARN-812. Set default logger for application summary logger to hadoop.root.logger. Contributed by Siddarth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492958 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 06:04:06 +00:00
Vinod Kumar Vavilapalli e9f753a11e YARN-792. Moved NodeHealthStatus from yarn.api.record to yarn.server.api.record. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492943 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 03:17:03 +00:00
Vinod Kumar Vavilapalli f0eb4bc342 YARN-692. Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492907 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 00:06:42 +00:00
Vinod Kumar Vavilapalli c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 23:25:14 +00:00
Vinod Kumar Vavilapalli 686ac311c9 YARN-746. Renamed Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively. Contributed by Steve Loughran.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492780 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 17:49:14 +00:00
Vinod Kumar Vavilapalli 0928502029 YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran.
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 15:54:38 +00:00
Alejandro Abdelnur 1104c7d166 YARN-648. FS: Add documentation for pluggable policy. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492388 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 19:24:23 +00:00
Chris Nauroth 1ed916f6c6 YARN-700. TestInfoBlock fails on Windows because of line ending missmatch. Contributed by Ivan Mitic.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492383 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 19:18:39 +00:00
Alejandro Abdelnur 80eb92aff0 YARN-600. Hook up cgroups CPU settings to the number of virtual cores allocated. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492365 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 18:57:31 +00:00
Jonathan Turner Eagles b90755c296 YARN-427. Coverage fix for org.apache.hadoop.yarn.server.api.* (Aleksey Gorshkov via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492282 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 16:52:15 +00:00
Alejandro Abdelnur e75ee47fc9 YARN-795. Fair scheduler queue metrics should subtract allocated vCores from available vCores. (ywskycn via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492021 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 00:25:07 +00:00
Siddharth Seth 6d4d87008c YARN-731. RPCUtil.unwrapAndThrowException should unwrap remote RuntimeExceptions. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492000 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 22:34:39 +00:00
Arun Murthy 25946da413 YARN-767. Initialize application metrics at RM bootup. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491989 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 22:15:02 +00:00
Siddharth Seth c6c41abf68 YARN-737. Throw some specific exceptions directly instead of wrapping them in YarnException. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491896 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 18:47:16 +00:00
Konstantin Boudnik f3b322e023 Setting release date for 2.0.5-alpha release
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491139 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 04:55:26 +00:00
Vinod Kumar Vavilapalli 0b0be8d4dd YARN-642. Removed health parameter from ResourceManager /nodes web-service and cleaned the behaviour of the status parameter. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490890 13f79535-47bb-0310-9956-ffa450edef68
2013-06-08 00:54:22 +00:00
Vinod Kumar Vavilapalli 176315d697 YARN-777. Removed unreferenced objects from .proto files. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490882 13f79535-47bb-0310-9956-ffa450edef68
2013-06-08 00:11:12 +00:00
Alejandro Abdelnur af8514eef2 YARN-686. Flatten NodeReport. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490827 13f79535-47bb-0310-9956-ffa450edef68
2013-06-07 20:59:23 +00:00
Jason Darrell Lowe a8c330fd33 YARN-760. NodeManager throws AvroRuntimeException on failed start. Contributed by Niranjan Singh
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490704 13f79535-47bb-0310-9956-ffa450edef68
2013-06-07 16:20:20 +00:00
Bikas Saha 9fcfbf5f51 YARN-759. Create Command enum in AllocateResponse (bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 23:33:48 +00:00
Bikas Saha 91a4da5ebb YARN-761. TestNMClientAsync fails sometimes (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490437 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 20:47:09 +00:00
Bikas Saha 2051fd5ee2 YARN-750. Allow for black-listing resources in YARN API and Impl in CS (acmurthy via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490392 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 18:46:59 +00:00
Thomas Graves 98f55792ff YARN-764. blank Used Resources on Capacity Scheduler page (Nemon Lou via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490387 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 18:25:36 +00:00
Thomas Graves 5fc3c47800 Updating release date for 0.23.8
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489907 13f79535-47bb-0310-9956-ffa450edef68
2013-06-05 15:05:43 +00:00
Vinod Kumar Vavilapalli 66de4ccf60 YARN-724. Moved ProtoBase from api.records to api.records.impl.pb. Contributed by Jian He.
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via YARN-724. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489658 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 22:43:19 +00:00
Vinod Kumar Vavilapalli 707b310c79 YARN-753. Added individual factory methods for all api protocol records and converted the records to be abstract classes. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489644 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 21:52:11 +00:00
Vinod Kumar Vavilapalli af65d6f80e YARN-756. Remnants from moving Preemption* records to yarn.api where they really belong. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489614 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 21:21:09 +00:00
Kihwal Lee 78063b3a76 YARN-742. Log aggregation causes a lot of redundant setPermission calls. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489596 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 20:49:34 +00:00
Vinod Kumar Vavilapalli 978012b9b6 YARN-755. Renamed AllocateResponse.reboot to AllocateResponse.resync. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489295 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 05:53:47 +00:00
Vinod Kumar Vavilapalli 28aabe0b2b YARN-756. Move Preemption* records to yarn.api where they really belong. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489290 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 05:03:51 +00:00
Vinod Kumar Vavilapalli a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth.
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 04:05:50 +00:00
Vinod Kumar Vavilapalli c71fb196b9 YARN-757. Changed TestRMRestart to use the default scheduler to avoid test failures. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489268 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 01:14:22 +00:00
Vinod Kumar Vavilapalli d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He.
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 00:34:18 +00:00
Arun Murthy 5cf48b4612 YARN-726. Fix queue & finish time fields in web-ui for ResourceManager. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489234 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 23:50:19 +00:00
Arun Murthy 4a14264ddb YARN-398. Make it possible to specify hard locality constraints in resource requests for CapacityScheduler. Contributed by Arun C. Murthy.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489087 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 18:13:53 +00:00
Alejandro Abdelnur c1b635ed48 YARN-326. Add multi-resource scheduling to the fair scheduler. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489070 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 17:33:55 +00:00
Arun Murthy 52b189db90 YARN-730. Fix NMClientAsync to remove completed containers. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488840 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 04:11:33 +00:00
Siddharth Seth a2205a3b72 YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 02:35:34 +00:00
Arun Murthy f27e484636 YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
2013-06-02 22:53:58 +00:00
Vinod Kumar Vavilapalli 3dce234ed9 YARN-733. Fixed TestNMClient from failing occasionally. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488618 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 22:01:06 +00:00
Vinod Kumar Vavilapalli a2c4233004 YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He.
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 21:43:17 +00:00
Konstantin Boudnik aba8b6e540 Adding 2.0.5-alpha release to CHANGES.txt files of trunk
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488587 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 20:07:03 +00:00
Bikas Saha 3492f5eff1 YARN-660. Improve AMRMClient with matching requests (bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488485 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 08:23:30 +00:00
Siddharth Seth 39f019f413 YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 00:14:31 +00:00
Alejandro Abdelnur 781e82ca9a YARN-392. Make it possible to specify hard locality constraints in resource requests. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488326 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 17:26:18 +00:00
Siddharth Seth 2692675fc3 YARN-684. ContainerManager.startContainer should use ContainerTokenIdentifier instead of the entire Container. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488085 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 04:13:16 +00:00
Vinod Kumar Vavilapalli 982753dc8e YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth.
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 20:18:39 +00:00
Konstantin Boudnik 2fb2a5ab42 HADOOP-9607. Fixes in Javadoc build. Contributed by Timothy St. Clair
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487979 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 19:06:49 +00:00
Siddharth Seth b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli 29902cd53c YARN-638. Modified ResourceManager to restore RMDelegationTokens after restarting. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487720 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 04:14:26 +00:00
Vinod Kumar Vavilapalli 1858c7e52e YARN-714. Added NMTokens to be sent to AMs as part of heart-beat response. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487704 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 01:57:05 +00:00
Vinod Kumar Vavilapalli 5420f287cc YARN-578. Fixed NM to use SecureIOUtils for reading and aggregating logs. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487672 13f79535-47bb-0310-9956-ffa450edef68
2013-05-29 23:14:59 +00:00
Jason Darrell Lowe 31c96829c7 YARN-512. Log aggregation root directory check is more expensive than it needs to be. Contributed by Maysam Yabandeh
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487498 13f79535-47bb-0310-9956-ffa450edef68
2013-05-29 14:25:04 +00:00
Vinod Kumar Vavilapalli edc6d7d3ab YARN-422. Add a NM Client library to help application-writers. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487184 13f79535-47bb-0310-9956-ffa450edef68
2013-05-29 01:41:41 +00:00
Suresh Srinivas a5d7b6ef6f HADOOP-8562. Moved related jiras to 2.0.5 section in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1486336 13f79535-47bb-0310-9956-ffa450edef68
2013-05-25 15:15:50 +00:00
Vinod Kumar Vavilapalli 259edf8dca YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 03:22:54 +00:00
Vinod Kumar Vavilapalli 61885df2fa YARN-711. Copied BuilderUtil methods in individual API records as BuilderUtils is going to be dismantled. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485888 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 22:47:33 +00:00
Jonathan Turner Eagles 52cbb906d9 YARN-548. Add tests for YarnUncaughtExceptionHandler (Vadim Bondarev via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485877 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 22:21:56 +00:00
Vinod Kumar Vavilapalli 43876770d9 YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal.
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 18:02:04 +00:00
Siddharth Seth d1ff045c41 Fixed incorrect jira in CHANGES.txt (YARN-716 to YARN-715)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485582 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 06:00:59 +00:00
Siddharth Seth 8a430d0643 YARN-716. Fixed unit test failures - TestDistributedShell and TestUnmanagedAMLauncher. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485579 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 05:48:22 +00:00
Vinod Kumar Vavilapalli 132e38ef47 YARN-708. Deleting files that were supposed to removed in the previous commit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485481 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 23:06:58 +00:00
Vinod Kumar Vavilapalli 1fac1ac98c YARN-708. Moved RecordFactory classes to hadoop-yarn-api, and put some miscellaneous fixes to the interfaces. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485478 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 22:51:04 +00:00
Vinod Kumar Vavilapalli c665b0e86b YARN-706. Fixed race conditions in TestFSDownload. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485028 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 01:50:34 +00:00
Vinod Kumar Vavilapalli e5ed91aff7 YARN-695. Remove masterContainer and status unused fields from ApplicationReportProto and fix bugs in ApplicationReportPBImpl. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484612 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 21:50:47 +00:00
Christopher Douglas f9ab715880 YARN-568. Followup; remove an unused field added to Allocation
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484375 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 05:46:43 +00:00
Vinod Kumar Vavilapalli ca80246731 YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 06:36:33 +00:00
Robert Joseph Evans 3641f938f6 YARN-690. RM exits on token cancel/renew problems (daryn via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483578 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 21:41:49 +00:00
Vinod Kumar Vavilapalli 065747efab YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
2013-05-16 06:58:34 +00:00
Siddharth Seth 1a119f87b4 YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 04:11:55 +00:00
Arun Murthy 6f236b28ff YARN-597. TestFSDownload fails on Windows due to dependencies on tar/gzip/jar tools. Contributed by Ivan Mitic.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482149 13f79535-47bb-0310-9956-ffa450edef68
2013-05-13 23:49:05 +00:00
Vinod Kumar Vavilapalli 68148989bf YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli a35c7fd80b YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli 2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth.
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 21:49:28 +00:00
Alejandro Abdelnur f19465f72f YARN-598. Add virtual cores to queue metrics. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480816 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 22:23:50 +00:00
Alejandro Abdelnur 51ccb87031 YARN-655. Fair scheduler metrics should subtract allocated memory from available memory. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480809 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 22:15:58 +00:00
Alejandro Abdelnur e0562e3d07 YARN-637. FS: maxAssign is not honored. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480802 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 21:52:03 +00:00
Alejandro Abdelnur c7bfd17ad1 YARN-507. Add interface visibility and stability annotations to FS interfaces/classes. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480799 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 21:47:21 +00:00
Christopher Douglas 505fe26539 YARN-568. Add support for work preserving preemption to the FairScheduler.
Contributed by Carlo Curino and Sandy Ryza


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480778 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 21:21:42 +00:00
Vinod Kumar Vavilapalli 9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli 0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli e5692ecbe1 Fixing release date for 2.0.4-alpha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480480 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 21:57:44 +00:00
Aaron Myers 8dee15abe3 YARN-646. Fix two typos in Fair Scheduler user guide. Contributed by Dapeng Sun.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480360 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 17:17:43 +00:00
Vinod Kumar Vavilapalli 74231f0276 YARN-582. Changed ResourceManager to recover Application token and client tokens for app attempt so that RM can be restarted while preserving current applications. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480168 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 06:20:22 +00:00
Vinod Kumar Vavilapalli 33ed6e2838 YARN-651. Changed PBClientImpls of ContainerManager and RMAdmin to throw IOExceptions also. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480154 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 04:17:02 +00:00
Christopher Douglas 1f5ac02a0d YARN-45. Add protocol for schedulers to request containers back from
ApplicationMasters. Contributed by Carlo Curino and Chris Douglas.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479771 13f79535-47bb-0310-9956-ffa450edef68
2013-05-07 06:12:36 +00:00
Vinod Kumar Vavilapalli 9eb53b9c84 YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479740 13f79535-47bb-0310-9956-ffa450edef68
2013-05-07 00:42:48 +00:00
Vinod Kumar Vavilapalli 674cf7f742 YARN-633. Changed RMAdminProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479738 13f79535-47bb-0310-9956-ffa450edef68
2013-05-07 00:37:46 +00:00
Vinod Kumar Vavilapalli 92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong.
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli c0fa9d2231 YARN-645. Moved RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager where it really belongs. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479022 13f79535-47bb-0310-9956-ffa450edef68
2013-05-04 01:12:17 +00:00
Vinod Kumar Vavilapalli fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
2013-05-02 22:44:12 +00:00
Vinod Kumar Vavilapalli 46b2898a1a YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478230 13f79535-47bb-0310-9956-ffa450edef68
2013-05-01 23:23:20 +00:00
Vinod Kumar Vavilapalli 6de09af244 YARN-599. Refactoring submitApplication in ClientRMService and RMAppManager to separate out various validation checks depending on whether they rely on RM configuration or not. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477478 13f79535-47bb-0310-9956-ffa450edef68
2013-04-30 05:44:06 +00:00
Suresh Srinivas 27d4592771 YARN-506. Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477408 13f79535-47bb-0310-9956-ffa450edef68
2013-04-29 23:09:28 +00:00
Vinod Kumar Vavilapalli d9ce42479e YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy minimum resource requirements. Contributed by Kenji Kikushima.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476824 13f79535-47bb-0310-9956-ffa450edef68
2013-04-28 20:03:37 +00:00
Suresh Srinivas fe3853484e HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
2013-04-27 14:24:02 +00:00
Vinod Kumar Vavilapalli 95fa3e82e4 YARN-363. Add webapps/proxy directory without which YARN proxy-server fails when started in stand-alone mode. Contributed by Kenji Kikushima.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476330 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 18:24:30 +00:00
Vinod Kumar Vavilapalli f0ac5a849c YARN-591. Moved RM recovery related records out of public API as they do not belong there. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476324 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 18:02:45 +00:00
Vinod Kumar Vavilapalli 635f96e74e YARN-562. Missed files from previous commit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 04:42:34 +00:00
Vinod Kumar Vavilapalli fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He.
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 03:50:20 +00:00
Thomas White d10428cab2 YARN-289. Fair scheduler allows reservations that won't fit on node. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475681 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 09:15:51 +00:00
Thomas White a43c344eee YARN-595. Refactor fair scheduler to use common Resources. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475670 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 08:33:43 +00:00
Vinod Kumar Vavilapalli 448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah.
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 05:42:22 +00:00
Siddharth Seth be256a1819 YARN-579. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471814 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 02:46:47 +00:00
Vinod Kumar Vavilapalli a2f97fd9cb YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471608 13f79535-47bb-0310-9956-ffa450edef68
2013-04-24 19:42:47 +00:00
Vinod Kumar Vavilapalli cd975d4b64 YARN-581. Added a test to verify that app delegation tokens are restored after RM restart. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471187 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 22:49:44 +00:00
Vinod Kumar Vavilapalli 27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong.
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 21:39:51 +00:00
Vinod Kumar Vavilapalli ffd7dbb34a YARN-583. Moved application level local resources to be localized under the filecache sub-directory under application directory. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470812 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 04:37:45 +00:00
Bikas Saha a1ee2145cf YARN-549. YarnClient.submitApplication should wait for application to be accepted by the RM (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470797 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 03:06:41 +00:00
Bikas Saha e24a9b83f1 YARN-594. Update test and add comments in YARN-534 (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470243 13f79535-47bb-0310-9956-ffa450edef68
2013-04-20 22:04:08 +00:00
Vinod Kumar Vavilapalli a91067fc5e YARN-542. Changed the default global AM max-attempts value to be not one. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470094 13f79535-47bb-0310-9956-ffa450edef68
2013-04-20 00:35:29 +00:00
Vinod Kumar Vavilapalli c570309b07 YARN-547. Fixed race conditions in public and private resource localization which used to cause duplicate downloads. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470076 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 22:35:43 +00:00
Vinod Kumar Vavilapalli d5556cb5c4 YARN-586. Fixed a typo in ApplicationSubmissionContext#setApplicationId. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470068 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 22:25:17 +00:00
Bikas Saha edcfd4527c YARN-593. container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources (Chris Nauroth via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469998 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 19:29:22 +00:00
Vinod Kumar Vavilapalli 44bf8525a5 YARN-493. Fixed some shell related flaws in YARN on Windows. Contributed by Chris Nauroth.
HADOOP-9486. Promoted Windows and Shell related utils from YARN to Hadoop Common. Contributed by Chris Nauroth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469667 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 02:14:58 +00:00
Vinod Kumar Vavilapalli 8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong.
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 01:33:13 +00:00
Alejandro Abdelnur df55edd5de YARN-482. FS: Extend SchedulingMode to intermediate queues. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469506 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 18:11:21 +00:00
Vinod Kumar Vavilapalli d219c904d3 YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469492 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 17:49:22 +00:00
Alejandro Abdelnur de6d05c8b0 YARN-476. ProcfsBasedProcessTree info message confuses users. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469093 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 23:11:15 +00:00
Alejandro Abdelnur 014dc3ae9b YARN-518. Fair Scheduler's document link could be added to the hadoop 2.x main doc page. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469091 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 23:05:58 +00:00
Bikas Saha ef9f251679 YARN-514.Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469059 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 20:19:43 +00:00
Vinod Kumar Vavilapalli dbcda89fef YARN-500. Fixed YARN webapps to not roll-over ports when explicitly asked to use non-ephemeral ports. Contributed by Kenji Kikushima.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468739 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 04:16:51 +00:00
Vinod Kumar Vavilapalli 0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza.
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
2013-04-15 23:54:09 +00:00
Arun Murthy 5e94420456 YARN-412. Pushing to 2.0.5-beta only.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467470 13f79535-47bb-0310-9956-ffa450edef68
2013-04-12 21:19:39 +00:00
Arun Murthy c2592021f3 YARN-412. Fixed FifoScheduler to check hostname of a NodeManager rather than its host:port during scheduling which caused incorrect locality for containers. Contributed by Roger Hoover.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467244 13f79535-47bb-0310-9956-ffa450edef68
2013-04-12 12:12:08 +00:00
Vinod Kumar Vavilapalli 529170de44 Fixing CHANGES.txt entry for YARN-319.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467133 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 23:09:43 +00:00
Thomas Graves 6d9c311840 Preparing for release 0.23.7
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467069 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 20:08:00 +00:00
Vinod Kumar Vavilapalli e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong.
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 19:28:51 +00:00
Vinod Kumar Vavilapalli 4234bc87b3 YARN-539. Addressed memory leak of LocalResource objects NM when a resource localization fails. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466756 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 02:08:11 +00:00
Vinod Kumar Vavilapalli 2e3b56f6e9 YARN-495. Changed NM reboot behaviour to be a simple resync - kill all containers and re-register with RM. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466752 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 02:00:47 +00:00
Vinod Kumar Vavilapalli add00d6d74 YARN-487. Modify path manipulation in LocalDirsHandlerService to let TestDiskFailures pass on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466746 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 01:00:38 +00:00
Vinod Kumar Vavilapalli 7d00d3d20f YARN-534. Change RM restart recovery to also account for AM max-attempts configuration after the restart. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466208 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 20:16:19 +00:00
Vinod Kumar Vavilapalli 3a54a5653b YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 19:56:10 +00:00
Vinod Kumar Vavilapalli b1b99bb8bd YARN-524. Moving CHANGES.txt entry to the correct section.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465876 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 04:16:51 +00:00
Vinod Kumar Vavilapalli 55865f42c4 YARN-557. Fix TestUnmanagedAMLauncher failure on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465869 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 03:26:38 +00:00
Vinod Kumar Vavilapalli b96d18bd23 YARN-99. Modify private distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465853 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 01:33:38 +00:00
Bikas Saha 16fedf5473 YARN-479. NM retry behavior for connection to RM should be similar for lost heartbeats (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465731 13f79535-47bb-0310-9956-ffa450edef68
2013-04-08 19:17:16 +00:00
Bikas Saha bc6777dd5b YARN-193. Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465067 13f79535-47bb-0310-9956-ffa450edef68
2013-04-05 17:45:44 +00:00
Jason Darrell Lowe 18757c1267 YARN-525. make CS node-locality-delay refreshable. Contributed by Thomas Graves
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465009 13f79535-47bb-0310-9956-ffa450edef68
2013-04-05 15:30:24 +00:00
Vinod Kumar Vavilapalli 0836d68abf YARN-532. Change RMAdmin and Localization client protocol PB implementations to implement closeable so that they can be stopped when needed via RPC.stopProxy(). Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464788 13f79535-47bb-0310-9956-ffa450edef68
2013-04-04 23:30:15 +00:00
Siddharth Seth 681312d9c4 Updated CHANGES.txt to reflect YARN-470 being merged into branch-2.0.4-alpha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464772 13f79535-47bb-0310-9956-ffa450edef68
2013-04-04 21:25:28 +00:00
Vinod Kumar Vavilapalli 1fd462b118 YARN-536. Removed the unused objects ContainerStatus and ContainerStatus from Container which also don't belong to the container. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464271 13f79535-47bb-0310-9956-ffa450edef68
2013-04-04 03:41:18 +00:00
Alejandro Abdelnur 3a089cf356 YARN-458. YARN daemon addresses must be placed in many different configs. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464204 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 21:04:23 +00:00
Alejandro Abdelnur 2100e93b04 YARN-538. RM address DNS lookup can cause unnecessary slowness on every JHS page load. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464197 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 20:41:57 +00:00
Eli Collins 2d02b7741d Revert YARN-516 per HADOOP-9357.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464181 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 19:42:12 +00:00
Thomas White dd2e87a79f YARN-381. Improve fair scheduler docs. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464130 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 18:01:14 +00:00
Vinod Kumar Vavilapalli 3e9200ddde YARN-101. Fix NodeManager heartbeat processing to not lose track of completed containers in case of dropped heartbeats. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464105 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 16:57:07 +00:00
Vinod Kumar Vavilapalli e67e3ff05d YARN-467. Modify public distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463823 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 05:00:28 +00:00
Bikas Saha d40b154d27 YARN-382. SchedulerUtils improve way normalizeRequest sets the resource capabilities (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463653 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 18:07:07 +00:00
Vinod Kumar Vavilapalli bcce9354bc YARN-447. Move ApplicationComparator in CapacityScheduler to use comparator in ApplicationId. Contributed by Nemon Lou.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463405 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 06:59:13 +00:00
Vinod Kumar Vavilapalli 17be263e9b YARN-516. Fix failure in TestContainerLocalizer caused by HADOOP-9357. Contributed by Andrew Wang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463362 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 02:16:35 +00:00
Vinod Kumar Vavilapalli 7ca9fe73a0 YARN-309. Changed NodeManager to obtain heart-beat interval from the ResourceManager. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463346 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 01:04:07 +00:00
Steve Loughran 7c65dbb938 YARN-524 TestYarnVersionInfo failing if generated properties doesn't include an SVN URL
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463300 13f79535-47bb-0310-9956-ffa450edef68
2013-04-01 20:57:02 +00:00
Vinod Kumar Vavilapalli aa634e0814 YARN-475. Remove a unused constant in the public API - ApplicationConstants.AM_APP_ATTEMPT_ID_ENV. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463033 13f79535-47bb-0310-9956-ffa450edef68
2013-03-31 19:17:44 +00:00
Jason Darrell Lowe 2fd56e33b4 YARN-515. Node Manager not getting the master key. Contributed by Robert Joseph Evans
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462632 13f79535-47bb-0310-9956-ffa450edef68
2013-03-29 20:20:19 +00:00
Thomas Graves 01aabf7363 YARN-460. CS user left in list of active users for the queue even when application finished (tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462486 13f79535-47bb-0310-9956-ffa450edef68
2013-03-29 14:36:53 +00:00
Bikas Saha 520033b1cd YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
2013-03-28 19:44:28 +00:00
Alejandro Abdelnur 967b62f5d2 YARN-24. Nodemanager fails to start if log aggregation enabled and namenode unavailable. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461891 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 23:14:05 +00:00
Vinod Kumar Vavilapalli d0bbff6c32 YARN-209. Fix CapacityScheduler to trigger application-activation when the cluster capacity changes. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461773 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 18:38:28 +00:00
Thomas White 414458ff9d YARN-496. Fair scheduler configs are refreshed inconsistently in reinitialize. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461614 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 14:42:41 +00:00
Vinod Kumar Vavilapalli d60c2fa17f YARN-474. Fix CapacityScheduler to trigger application-activation when am-resource-percent configuration is refreshed. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461402 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 03:44:34 +00:00
Siddharth Seth 66e90b205a YARN-440. Flatten RegisterNodeManagerResponse. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461256 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 18:20:36 +00:00
Alejandro Abdelnur e74d1f0435 YARN-469. Make scheduling mode in FS pluggable. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460961 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 03:25:26 +00:00
Bikas Saha 2aed48a67f YARN-498. Unmanaged AM launcher does not set various constants in env for an AM, also does not handle failed AMs properly (Hitesh Shah via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460954 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 02:34:27 +00:00
Vinod Kumar Vavilapalli 0757364014 YARN-378. Fix RM to make the AM max attempts/retries to be configurable per application by clients. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460895 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 21:11:33 +00:00
Bikas Saha 1acb805058 YARN-497. Yarn unmanaged-am launcher jar does not define a main class in its manifest (Hitesh Shah via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460846 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 18:57:15 +00:00
Siddharth Seth 18e08a8f51 YARN-439. Flatten NodeHeartbeatResponse. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460811 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 18:28:50 +00:00
Siddharth Seth 6a482a88b8 YARN-71. Fix the NodeManager to clean up local-dirs on restart. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460808 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 18:23:59 +00:00
Robert Joseph Evans c02dd9aa4b YARN-109. .tmp file is not deleted for localized archives (Mayank Bansal via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460723 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 15:46:25 +00:00
Bikas Saha 7c325a5d71 YARN-417. Addendum patch to fix broken test (Sandy Ryza via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460064 13f79535-47bb-0310-9956-ffa450edef68
2013-03-23 00:08:24 +00:00
Hitesh Shah b15b44e722 YARN-470. Support a way to disable resource monitoring on the NodeManager. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460001 13f79535-47bb-0310-9956-ffa450edef68
2013-03-22 21:28:31 +00:00
Bikas Saha 28bac40295 YARN-417. Create AMRMClient wrapper that provides asynchronous callbacks. (Sandy Ryza via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459555 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 22:26:06 +00:00
Hitesh Shah 0cda8267d5 YARN-491. TestContainerLogsPage fails on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459526 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 21:04:59 +00:00
Hitesh Shah d6b33ee981 YARN-490. TestDistributedShell fails on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459520 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 20:48:18 +00:00
Hitesh Shah 92fb7cdcde YARN-488. TestContainerManagerSecurity fails on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459514 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 20:38:43 +00:00
Hitesh Shah 2703518240 YARN-297. Improve hashCode implementations for PB records. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459054 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 21:09:33 +00:00
Hitesh Shah 1bd345d6e3 YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 20:44:35 +00:00
Jason Darrell Lowe 67699c2d18 YARN-200. yarn log does not output all needed information, and is in a binary format. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458466 13f79535-47bb-0310-9956-ffa450edef68
2013-03-19 19:29:15 +00:00
Bikas Saha 37eed0a3e1 YARN-481. Add AM Host and RPC Port to ApplicationCLI Status Output (Chris Riccomini via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458065 13f79535-47bb-0310-9956-ffa450edef68
2013-03-18 23:48:58 +00:00
Alejandro Abdelnur d2775d6795 YARN-485. TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the process to die. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457936 13f79535-47bb-0310-9956-ffa450edef68
2013-03-18 19:44:01 +00:00
Aaron Myers 64741f4635 HDFS-4521. Invalid network toploogies should not be cached. Contributed by Colin Patrick McCabe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457878 13f79535-47bb-0310-9956-ffa450edef68
2013-03-18 17:20:13 +00:00
Hitesh Shah 2ba66a54f5 YARN-196. Nodemanager should be more robust in handling connection failure to ResourceManager when a cluster is started. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457038 13f79535-47bb-0310-9956-ffa450edef68
2013-03-15 17:58:58 +00:00
Robert Joseph Evans 1b6f0582fb YARN-237. Refreshing the RM page forgets how many rows I had in my Datatables (jian he via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456536 13f79535-47bb-0310-9956-ffa450edef68
2013-03-14 16:56:10 +00:00
Robert Joseph Evans 9ec88e8e29 YARN-468. coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter (Aleksey Gorshkov via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456458 13f79535-47bb-0310-9956-ffa450edef68
2013-03-14 14:52:33 +00:00
Vinod Kumar Vavilapalli 44eb4c31b8 YARN-198. Added a link to RM pages from the NodeManager web app. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455800 13f79535-47bb-0310-9956-ffa450edef68
2013-03-13 04:24:35 +00:00
Suresh Srinivas 5865ec1c37 HADOOP-9379. capture the ulimit info after printing the log to the console. Contributed by Arpit Gupta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455316 13f79535-47bb-0310-9956-ffa450edef68
2013-03-11 20:31:17 +00:00
Thomas Graves 5f2c518c95 YARN-443. allow OS scheduling priority of NM to be different than the containers it launches (tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454411 13f79535-47bb-0310-9956-ffa450edef68
2013-03-08 14:46:26 +00:00
Arun Murthy 80c2cf57fd Preparing for bug-fix 2.0.4-alpha release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453886 13f79535-47bb-0310-9956-ffa450edef68
2013-03-07 14:21:40 +00:00
Suresh Srinivas 96fc01fbc2 HADOOP-9373. Remove empty files that should have been removed in the commit r1453599
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453645 13f79535-47bb-0310-9956-ffa450edef68
2013-03-07 01:16:02 +00:00
Suresh Srinivas 50d136f9c9 HADOOP-9373. Merge CHANGES.branch-trunk-win.txt to CHANGES.txt. Contributed by Suresh Srinivas.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453599 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 22:37:42 +00:00
Hitesh Shah 2e4382b7c3 YARN-429. capacity-scheduler config missing from yarn-test artifact. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453501 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 19:26:57 +00:00
Suresh Srinivas 638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 19:15:18 +00:00
Jonathan Turner Eagles 21066255da YARN-227. Application expiration difficult to debug for end-users (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453080 13f79535-47bb-0310-9956-ffa450edef68
2013-03-05 23:12:17 +00:00
Jason Darrell Lowe 5889f54ad1 YARN-345. Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452548 13f79535-47bb-0310-9956-ffa450edef68
2013-03-04 21:47:39 +00:00
Robert Joseph Evans 9334dc23f9 YARN-448. Remove unnecessary hflush from log aggregation (Kihwal Lee via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452475 13f79535-47bb-0310-9956-ffa450edef68
2013-03-04 20:08:21 +00:00
Vinod Kumar Vavilapalli 6680ee5f5d YARN-410. Fixed RM UI so that the new lines diagnostics for a failed app on the per-application page are translated to html line breaks. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451808 13f79535-47bb-0310-9956-ffa450edef68
2013-03-02 01:21:53 +00:00
Vinod Kumar Vavilapalli 0b708ab75d YARN-380. Fix yarn node -status output to be better readable. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451711 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 20:39:18 +00:00
Siddharth Seth 83d8065867 YARN-376. Fixes a bug which would prevent the NM knowing about completed containers and applications. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451473 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 05:59:54 +00:00
Alejandro Abdelnur 5d6eca08bd MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does not exist. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451452 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 01:28:18 +00:00
Siddharth Seth 62d70af19a YARN-406. Fix TestRackResolver to function in networks where "host1" resolves to a valid host. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451391 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 22:17:39 +00:00
Kihwal Lee 27c8a87a54 YARN-269. Resource Manager not logging the health_check_script result when taking it out. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451341 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 21:02:06 +00:00
Robert Joseph Evans 0057600a82 YARN-426. Failure to download a public resource prevents further downloads (Jason Lowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450807 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 15:30:10 +00:00
Suresh Srinivas 70b1c5ca75 HADOOP-9334. Upgrade netty version. Contributed by Nicolas Liochon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450463 13f79535-47bb-0310-9956-ffa450edef68
2013-02-26 21:54:25 +00:00
Siddharth Seth 7ea82ca558 YARN-365. Change NM heartbeat handling to not generate a scheduler event on each heartbeat. (Contributed by Xuan Gong)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450007 13f79535-47bb-0310-9956-ffa450edef68
2013-02-26 03:32:10 +00:00
Suresh Srinivas 10e1e314ac YARN-390. ApplicationCLI and NodeCLI hard-coded platform-specific line separator causes test failures on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1449980 13f79535-47bb-0310-9956-ffa450edef68
2013-02-26 00:26:24 +00:00
Aaron Myers 8271fae921 HADOOP-9267. hadoop -help, -h, --help should show usage instructions. Contributed by Andrew Wang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1449161 13f79535-47bb-0310-9956-ffa450edef68
2013-02-22 18:36:24 +00:00
Jonathan Turner Eagles 320f7bdcb8 YARN-236. RM should point tracking URL to RM web page when app fails to start (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448406 13f79535-47bb-0310-9956-ffa450edef68
2013-02-20 20:57:22 +00:00
Thomas Graves efaaf58605 YARN-400. RM can return null application resource usage report leading to NPE in client (Jason Lowe via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448241 13f79535-47bb-0310-9956-ffa450edef68
2013-02-20 15:35:06 +00:00
Alejandro Abdelnur 6bcbf492a2 HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445956 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 22:13:44 +00:00
Siddharth Seth 7bc6040ebb YARN-391. Formatting fixes for LCEResourceHandler classes. Contributed by Steve Loughran)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445882 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 19:40:36 +00:00
Thomas Graves 666667464a YARN-249. Capacity Scheduler web page should show list of active users per queue like it used to (in 1.x) (Ravi Prakash via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445673 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 15:40:01 +00:00
Jason Darrell Lowe d4931241ab YARN-362. Unexpected extra results when using webUI table search. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1444085 13f79535-47bb-0310-9956-ffa450edef68
2013-02-08 15:55:32 +00:00
Tsz-wo Sze a63d50f79a YARN-377. Use the new StringUtils methods added by HADOOP-9252 and fix TestContainersMonitor. Contributed by Chris Nauroth
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443796 13f79535-47bb-0310-9956-ffa450edef68
2013-02-07 23:53:49 +00:00
Siddharth Seth 362da383fd YARN-385. Add missing fields - location and #containers to ResourceRequestPBImpl's toString(). Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443702 13f79535-47bb-0310-9956-ffa450edef68
2013-02-07 20:28:20 +00:00
Siddharth Seth bdfe23235e YARN-383. AMRMClientImpl should handle null rmClient in stop(). Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443699 13f79535-47bb-0310-9956-ffa450edef68
2013-02-07 20:21:38 +00:00
Suresh Srinivas 54a9c6f964 HADOOP-9253. Capture ulimit info in the logs at service start time. Contributed by Arpit Gupta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443517 13f79535-47bb-0310-9956-ffa450edef68
2013-02-07 14:52:58 +00:00
Siddharth Seth ab16a37572 YARN-355. Fixes a bug where RM app submission could jam under load. Contributed by Daryn Sharp.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443131 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 19:03:52 +00:00
Daryn Sharp d272918426 YARN-357. App submission should not be synchronized (daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443016 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 15:26:30 +00:00
Arun Murthy c52a277687 YARN-3. Merged to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443011 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 15:19:24 +00:00
Arun Murthy 7cc2050041 Preparing for hadoop-2.0.3-alpha release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442959 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 13:31:52 +00:00
Arun Murthy 38bb4ad738 YARN-370. Fix SchedulerUtils to correctly round up the resource for containers. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442840 13f79535-47bb-0310-9956-ffa450edef68
2013-02-06 04:24:56 +00:00
Siddharth Seth 5d679c4f43 YARN-360. Allow apps to concurrently register tokens for renewal. Contributed by Daryn Sharp.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1442441 13f79535-47bb-0310-9956-ffa450edef68
2013-02-05 00:26:16 +00:00
Hitesh Shah b1baf11fb2 YARN-372. Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441668 13f79535-47bb-0310-9956-ffa450edef68
2013-02-02 00:43:54 +00:00
Jason Darrell Lowe f811198164 YARN-364. AggregatedLogDeletionService can take too long to delete logs. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441239 13f79535-47bb-0310-9956-ffa450edef68
2013-01-31 22:46:05 +00:00
Thomas Graves 6384684299 YARN-343. Capacity Scheduler maximum-capacity value -1 is invalid (Xuan Gong via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1440457 13f79535-47bb-0310-9956-ffa450edef68
2013-01-30 15:15:23 +00:00
Kihwal Lee df10249669 YARN-133. Update web services docs for RM clusterMetrics. Contributed by Ravi Prakash.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439653 13f79535-47bb-0310-9956-ffa450edef68
2013-01-28 21:51:36 +00:00
Jason Darrell Lowe 1137fb06df HADOOP-9246. Execution phase for hadoop-maven-plugin should be process-resources. Contributed by Karthik Kambatla and Chris Nauroth
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439620 13f79535-47bb-0310-9956-ffa450edef68
2013-01-28 20:52:14 +00:00
Suresh Srinivas e625e6d296 HADOOP-9245. mvn clean without running mvn install before fails. Contributed by Karthik Kambatla.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1438239 13f79535-47bb-0310-9956-ffa450edef68
2013-01-24 22:35:12 +00:00
Jason Darrell Lowe 2e9e7601e1 YARN-354. WebAppProxyServer exits immediately after startup. Contributed by Liang Xie
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437775 13f79535-47bb-0310-9956-ffa450edef68
2013-01-23 22:05:55 +00:00
Thomas White 345bcee066 YARN-319. Submitting a job to a fair scheduler queue for which the user does not have permission causes the client to wait forever. Contributed by shenhong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437336 13f79535-47bb-0310-9956-ffa450edef68
2013-01-23 10:53:05 +00:00
Hitesh Shah 29ddce96a2 YARN-231. RM Restart - Add FS-based persistent store implementation for RMStateStore. Contributed by Bikas Saha
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437245 13f79535-47bb-0310-9956-ffa450edef68
2013-01-23 00:57:37 +00:00
Hitesh Shah f5551bf809 YARN-277. Use AMRMClient in DistributedShell to exemplify the approach. Contributed by Bikas Saha
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1437156 13f79535-47bb-0310-9956-ffa450edef68
2013-01-22 20:49:09 +00:00
Suresh Srinivas ae270e72cf HADOOP-8924. Add maven plugin alternative to shell script to save package-info.java. Contributed by Alejandro Abdelnur and Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435380 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 22:35:58 +00:00
Suresh Srinivas f8cca2df8a HADOOP-8924. Revert r1435372 that missed some files
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435379 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 22:32:49 +00:00
Suresh Srinivas 98db9f5930 HADOOP-8924. Add maven plugin alternative to shell script to save package-info.java. Contributed by Alejandro Abdelnur and Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435372 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 22:18:12 +00:00
Alejandro Abdelnur f85b583731 YARN-302. Fair scheduler assignmultiple should default to false. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434996 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 00:48:22 +00:00
Alejandro Abdelnur c9dd4fa67f YARN-331. Fill in missing fair scheduler documentation. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434990 13f79535-47bb-0310-9956-ffa450edef68
2013-01-18 00:37:23 +00:00
Todd Lipcon e322e0a05f HADOOP-9215. when using cmake-2.6, libhadoop.so doesn't get created (only libhadoop.so.1.0.0). Contributed by Colin Patrick McCabe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434530 13f79535-47bb-0310-9956-ffa450edef68
2013-01-17 01:37:41 +00:00
Thomas Graves f266efcd10 Preparing for release 0.23.6
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1434030 13f79535-47bb-0310-9956-ffa450edef68
2013-01-16 16:43:12 +00:00
Siddharth Seth 2cee917f53 YARN-135. Add missing files from last commit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433594 13f79535-47bb-0310-9956-ffa450edef68
2013-01-15 19:29:34 +00:00
Siddharth Seth 6a2f2551fd YARN-135. Client tokens should be per app-attempt, and should be unregistered on App-finish. Contributed by Vinod Kumar Vavilapalli
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433570 13f79535-47bb-0310-9956-ffa450edef68
2013-01-15 18:33:38 +00:00
Thomas White 72e631098d YARN-336. Fair scheduler FIFO scheduling within a queue only allows 1 app at a time. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433526 13f79535-47bb-0310-9956-ffa450edef68
2013-01-15 17:09:48 +00:00
Thomas White 8447557333 YARN-335. Fair scheduler doesn't check whether rack needs containers before assigning to node. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433484 13f79535-47bb-0310-9956-ffa450edef68
2013-01-15 16:12:33 +00:00
Suresh Srinivas bbfcc709a2 YARN-328. Use token request messages defined in hadoop common. Contributed by Suresh Srinivas.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433231 13f79535-47bb-0310-9956-ffa450edef68
2013-01-15 00:16:57 +00:00
Hitesh Shah d7f9f9d177 YARN-330. Fix flakey test: TestNodeManagerShutdown#testKillContainersOnShutdown. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433138 13f79535-47bb-0310-9956-ffa450edef68
2013-01-14 21:27:39 +00:00
Thomas Graves ce9b0a05b0 YARN-334. Maven RAT plugin is not checking all source files (tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432931 13f79535-47bb-0310-9956-ffa450edef68
2013-01-14 14:34:51 +00:00
Thomas Graves 106e2e27ff YARN-325. RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completing (Arun C Murthy via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431070 13f79535-47bb-0310-9956-ffa450edef68
2013-01-09 21:00:47 +00:00
Siddharth Seth ba5b19fb5d YARN-320. RM should always be able to renew its own tokens. Contributed by Daryn Sharp
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1431020 13f79535-47bb-0310-9956-ffa450edef68
2013-01-09 19:33:06 +00:00
Arun Murthy 4539263971 YARN-2. Enhanced CapacityScheduler to account for CPU alongwith memory for multi-dimensional resource scheduling. Contributed by Arun C. Murthy.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430682 13f79535-47bb-0310-9956-ffa450edef68
2013-01-09 05:08:23 +00:00
Vinod Kumar Vavilapalli 9385dd50c7 YARN-253. Fixed container-launch to not fail when there are no local resources to localize. Contributed by Tom White.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430269 13f79535-47bb-0310-9956-ffa450edef68
2013-01-08 13:20:07 +00:00
Vinod Kumar Vavilapalli 2cd41855d5 YARN-170. Change NodeManager stop to be reentrant. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429796 13f79535-47bb-0310-9956-ffa450edef68
2013-01-07 13:43:52 +00:00
Vinod Kumar Vavilapalli 917ff03b2c YARN-315. Using the common security token protobuf definition from hadoop common. Contributed by Suresh Srinivas.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429748 13f79535-47bb-0310-9956-ffa450edef68
2013-01-07 11:08:21 +00:00
Vinod Kumar Vavilapalli 81222adc11 YARN-217. Fix RMAdmin protocol description to make it work in secure mode also. Contributed by Devaraj K.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429683 13f79535-47bb-0310-9956-ffa450edef68
2013-01-07 06:36:20 +00:00
Robert Joseph Evans 64e4fb983e MAPREDUCE-4819. AM can rerun job after reporting final job status to the client (bobby and Bikas Saha via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429114 13f79535-47bb-0310-9956-ffa450edef68
2013-01-04 20:35:56 +00:00
Thomas Graves dd8fc7e499 YARN-50. Implement renewal / cancellation of Delegation Tokens(Siddharth Seth via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429085 13f79535-47bb-0310-9956-ffa450edef68
2013-01-04 20:15:32 +00:00
Thomas White d6143a5b31 YARN-286. Add a YARN ApplicationClassLoader.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428781 13f79535-47bb-0310-9956-ffa450edef68
2013-01-04 11:00:55 +00:00
Siddharth Seth 0a61990855 YARN-103. Add a yarn AM-RM client module. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428554 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 19:11:37 +00:00
Thomas White 8cd026d365 YARN-301. Fair scheduler throws ConcurrentModificationException when iterating over app's priorities. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428400 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 15:26:55 +00:00
Thomas White 0768f96a83 YARN-300. After YARN-271, fair scheduler can infinite loop and not schedule any application. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428387 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 15:07:22 +00:00
Thomas White 19a291a0d6 YARN-288. Fair scheduler queue doesn't accept any jobs when ACLs are configured. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428362 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 14:32:01 +00:00
Thomas White ba90c9c867 YARN-192. Node update causes NPE in the fair scheduler. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428314 13f79535-47bb-0310-9956-ffa450edef68
2013-01-03 12:29:13 +00:00
Jason Darrell Lowe 16cf201042 YARN-293. Node Manager leaks LocalizerRunner object for every Container. Contributed by Robert Joseph Evans
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1428095 13f79535-47bb-0310-9956-ffa450edef68
2013-01-02 22:41:03 +00:00
Robert Joseph Evans 4b9f0443cb YARN-225. Proxy Link in RM UI thows NPE in Secure mode (Devaraj K via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1426515 13f79535-47bb-0310-9956-ffa450edef68
2012-12-28 14:21:07 +00:00
Vinod Kumar Vavilapalli 47c59570c1 Fixing CHANGES.txt for YARN-285.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425229 13f79535-47bb-0310-9956-ffa450edef68
2012-12-22 01:57:18 +00:00
Vinod Kumar Vavilapalli 5a408bdeac YARN-285. Added a temporary plugin interface for RM to be able to redirect to JobHistory server for apps that it no longer tracks. Contributed by Derek Dagit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425210 13f79535-47bb-0310-9956-ffa450edef68
2012-12-22 00:26:29 +00:00
Thomas Graves a806a2aac6 YARN-280. RM does not reject app submission with invalid tokens (Daryn Sharp via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1425079 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 18:49:47 +00:00
Alejandro Abdelnur a276f68b92 YARN-283. Fair scheduler fails to get queue info without root prefix. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424998 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 16:20:12 +00:00
Alejandro Abdelnur 5014a4b133 YARN-282. Fair scheduler web UI double counts Apps Submitted. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424995 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 16:12:20 +00:00
Alejandro Abdelnur f121d1e333 YARN-278. Fair scheduler maxRunningApps config causes no apps to make progress. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424989 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 16:07:11 +00:00
Alejandro Abdelnur 39ec437268 YARN-272. Fair scheduler log messages try to print objects without overridden toString methods. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424984 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 16:01:48 +00:00
Thomas White ee007d3f38 YARN-271. Fair scheduler hits IllegalStateException trying to reserve different apps on same node. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424945 13f79535-47bb-0310-9956-ffa450edef68
2012-12-21 15:10:24 +00:00
Luke Lu a28fac41b2 Update CHANGES.txt separately for clean cherry-picks.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424245 13f79535-47bb-0310-9956-ffa450edef68
2012-12-20 00:21:00 +00:00
Luke Lu 1a49c85438 YARN-223. Update process tree instead of getting new process trees. (Radim Kolar via llu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424244 13f79535-47bb-0310-9956-ffa450edef68
2012-12-20 00:20:53 +00:00
Arun Murthy 6cd0736cc5 YARN-230. RM Restart phase 1 - includes support for saving/restarting all applications on an RM bounce. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423758 13f79535-47bb-0310-9956-ffa450edef68
2012-12-19 04:21:18 +00:00
Alejandro Abdelnur 595d52cf37 YARN-254. Update fair scheduler web UI for hierarchical queues. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423742 13f79535-47bb-0310-9956-ffa450edef68
2012-12-19 03:04:39 +00:00
Alejandro Abdelnur 5032a694ed YARN-3. Add support for CPU isolation/monitoring of containers. (adferguson via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1423706 13f79535-47bb-0310-9956-ffa450edef68
2012-12-18 22:58:32 +00:00
Alejandro Abdelnur be8ef2ec33 YARN-264. y.s.rm.DelegationTokenRenewer attempts to renew token even after removing an app. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1421575 13f79535-47bb-0310-9956-ffa450edef68
2012-12-13 22:16:29 +00:00
Thomas White 618fa6cd59 YARN-267. Fix fair scheduler web UI. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420704 13f79535-47bb-0310-9956-ffa450edef68
2012-12-12 14:35:01 +00:00
Jason Darrell Lowe b72b36009a YARN-266. RM and JHS Web UIs are blank because AppsBlock is not escaping string properly. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420232 13f79535-47bb-0310-9956-ffa450edef68
2012-12-11 15:45:10 +00:00
Jason Darrell Lowe 2d1237cbb0 YARN-258. RM web page UI shows Invalid Date for start and finish times. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1417948 13f79535-47bb-0310-9956-ffa450edef68
2012-12-06 15:39:30 +00:00
Thomas White 235749a8ab YARN-72. NM should handle cleaning up containers when it shuts down. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1416484 13f79535-47bb-0310-9956-ffa450edef68
2012-12-03 12:12:11 +00:00
Jason Darrell Lowe 1573c1159a YARN-251. Proxy URI generation fails for blank tracking URIs. Contributed by Tom White
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415742 13f79535-47bb-0310-9956-ffa450edef68
2012-11-30 17:07:40 +00:00
Thomas White ae6f1123f5 YARN-187. Add hierarchical queues to the fair scheduler. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415592 13f79535-47bb-0310-9956-ffa450edef68
2012-11-30 12:03:25 +00:00
Arun Murthy 1943fdbec6 YARN-229. Remove old unused RM recovery code. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1415029 13f79535-47bb-0310-9956-ffa450edef68
2012-11-29 01:56:33 +00:00
Thomas White dd0ba9c0cc MAPREDUCE-4778. Fair scheduler event log is only written if directory exists on HDFS. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414729 13f79535-47bb-0310-9956-ffa450edef68
2012-11-28 14:40:40 +00:00
Thomas White fcbd55d092 YARN-222. Fair scheduler should create queue for each user by default. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414655 13f79535-47bb-0310-9956-ffa450edef68
2012-11-28 12:53:29 +00:00
Thomas White d9050e1208 YARN-224. Fair scheduler logs too many nodeUpdate INFO messages. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414647 13f79535-47bb-0310-9956-ffa450edef68
2012-11-28 12:27:30 +00:00
Robert Joseph Evans 966e20fa3d YARN-204. test coverage for org.apache.hadoop.tools (Aleksey Gorshkov via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1414239 13f79535-47bb-0310-9956-ffa450edef68
2012-11-27 16:05:16 +00:00
Jason Darrell Lowe 6c5f37e46d MAPREDUCE-4811. JobHistoryServer should show when it was started in WebUI About page. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1412377 13f79535-47bb-0310-9956-ffa450edef68
2012-11-22 00:05:26 +00:00
Robert Joseph Evans f0a6d06485 YARN-219. NM should aggregate logs when application finishes. (bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411289 13f79535-47bb-0310-9956-ffa450edef68
2012-11-19 16:41:11 +00:00
Thomas White e464607b7d YARN-129. Simplify classpath construction for mini YARN tests.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411235 13f79535-47bb-0310-9956-ffa450edef68
2012-11-19 15:12:22 +00:00
Alejandro Abdelnur 576f96f0ac YARN-184. Remove unnecessary locking in fair scheduler, and address findbugs excludes. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410826 13f79535-47bb-0310-9956-ffa450edef68
2012-11-18 05:00:56 +00:00
Robert Joseph Evans a23e8b0929 YARN-151. Browser thinks RM main page JS is taking too long (Ravi Prakash via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410564 13f79535-47bb-0310-9956-ffa450edef68
2012-11-16 20:56:08 +00:00
Robert Joseph Evans ab5039d4f1 YARN-214. RMContainerImpl does not handle event EXPIRE at state RUNNING (jeagles via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410522 13f79535-47bb-0310-9956-ffa450edef68
2012-11-16 18:26:19 +00:00
Robert Joseph Evans d2b9a604bd MAPREDUCE-4802. Takes a long time to load the task list on the AM for large jobs (Ravi Prakash via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1410498 13f79535-47bb-0310-9956-ffa450edef68
2012-11-16 17:30:22 +00:00
Robert Joseph Evans 1dedca8c9a YARN-188. Coverage fixing for CapacityScheduler (Aleksey Gorshkov via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409827 13f79535-47bb-0310-9956-ffa450edef68
2012-11-15 15:15:51 +00:00
Robert Joseph Evans d158daecb5 Updated CHANGES.txt in preparation for 0.23.5 release
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409818 13f79535-47bb-0310-9956-ffa450edef68
2012-11-15 14:57:23 +00:00
Robert Joseph Evans 7e2ecffd88 MAPREDUCE-4720. Browser thinks History Server main page JS is taking too long (Ravi Prakash via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409580 13f79535-47bb-0310-9956-ffa450edef68
2012-11-15 00:01:07 +00:00
Jason Darrell Lowe 397c261433 YARN-216. Remove jquery theming support. Contributed by Robert Joseph Evans.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1409313 13f79535-47bb-0310-9956-ffa450edef68
2012-11-14 19:22:11 +00:00
Jason Darrell Lowe 6db6e00649 YARN-212. NM state machine ignores an APPLICATION_CONTAINER_FINISHED event when it shouldn't. Contributed by Nathan Roberts
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1408812 13f79535-47bb-0310-9956-ffa450edef68
2012-11-13 15:59:13 +00:00
Alejandro Abdelnur 9d5205aadd HADOOP-8860. Split MapReduce and YARN sections in documentation navigation. (tomwhite via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407658 13f79535-47bb-0310-9956-ffa450edef68
2012-11-09 21:44:00 +00:00
Jonathan Turner Eagles 9e3af0ccd2 YARN-206. TestApplicationCleanup.testContainerCleanup occasionally fails. (jlowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407606 13f79535-47bb-0310-9956-ffa450edef68
2012-11-09 19:52:28 +00:00
Thomas White 933a6d2c1e YARN-183. Clean up fair scheduler code. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407433 13f79535-47bb-0310-9956-ffa450edef68
2012-11-09 12:38:10 +00:00
Robert Joseph Evans 8a1f333707 YARN-186. Coverage fixing LinuxContainerExecutor (Aleksey Gorshkov via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407171 13f79535-47bb-0310-9956-ffa450edef68
2012-11-08 17:22:52 +00:00
Thomas White 5a324b18ed YARN-169. Update log4j.appender.EventCounter to use org.apache.hadoop.log.metrics.EventCounter. Contributed by Anthony Rojas.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406999 13f79535-47bb-0310-9956-ffa450edef68
2012-11-08 10:33:29 +00:00
Arun Murthy fb5b96dfc3 YARN-201. Fix CapacityScheduler to be less conservative for starved off-switch requests. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406834 13f79535-47bb-0310-9956-ffa450edef68
2012-11-07 21:56:18 +00:00
Robert Joseph Evans b68bd472dc YARN-202. Log Aggregation generates a storm of fsync() for namenode (Kihwal Lee via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406269 13f79535-47bb-0310-9956-ffa450edef68
2012-11-06 18:48:49 +00:00
Robert Joseph Evans 05b6dc647e Localization update
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1405846 13f79535-47bb-0310-9956-ffa450edef68
2012-11-05 15:43:54 +00:00
Vinod Kumar Vavilapalli a124297cf0 YARN-189. Fixed a deadlock between RM's ApplicationMasterService and the dispatcher. Contributed by Thomas Graves.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404431 13f79535-47bb-0310-9956-ffa450edef68
2012-11-01 00:32:37 +00:00
Robert Joseph Evans ab8fa651f7 YARN-166. capacity scheduler doesn't allow capacity < 1.0 (tgraves via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404247 13f79535-47bb-0310-9956-ffa450edef68
2012-10-31 16:56:23 +00:00
Robert Joseph Evans 9f55dd3a54 YARN-159. RM web ui applications page should be sorted to display last app first (tgraves via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404228 13f79535-47bb-0310-9956-ffa450edef68
2012-10-31 16:22:34 +00:00
Robert Joseph Evans c15c192ad0 YARN-165. RM should point tracking URL to RM web page for app when AM fails (jlowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404211 13f79535-47bb-0310-9956-ffa450edef68
2012-10-31 15:47:11 +00:00
Thomas Graves aac5c149c7 MAPREDUCE-4752. Reduce MR AM memory usage through String Interning (Robert Evans via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404177 13f79535-47bb-0310-9956-ffa450edef68
2012-10-31 14:57:12 +00:00
Thomas White b54e794fb0 YARN-145. Add a Web UI to the fair share scheduler. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1402657 13f79535-47bb-0310-9956-ffa450edef68
2012-10-26 20:55:57 +00:00
Jason Darrell Lowe 614a743fea YARN-139. Interrupted Exception within AsyncDispatcher leads to user confusion. Contributed by Vinod Kumar Vavilapalli
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401726 13f79535-47bb-0310-9956-ffa450edef68
2012-10-24 15:20:58 +00:00
Robert Joseph Evans 90ba993bc7 YARN-180. Capacity scheduler - containers that get reserved create container token to early (acmurthy and bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401703 13f79535-47bb-0310-9956-ffa450edef68
2012-10-24 14:16:49 +00:00
Robert Joseph Evans 244ec4a5d1 YARN-178. Fix custom ProcessTree instance creation (Radim Kolar via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401698 13f79535-47bb-0310-9956-ffa450edef68
2012-10-24 14:07:16 +00:00
Thomas Graves cc523683cf YARN-177. CapacityScheduler - adding a queue while the RM is running has wacky results (acmurthy vai tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401668 13f79535-47bb-0310-9956-ffa450edef68
2012-10-24 13:21:09 +00:00
Vinod Kumar Vavilapalli ca4bc1a417 YARN-181. Fixed eclipse settings broken by capacity-scheduler.xml move via YARN-140. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401504 13f79535-47bb-0310-9956-ffa450edef68
2012-10-23 22:48:49 +00:00
Siddharth Seth 88d326f0a4 YARN-179. Fix some unit test failures. (Contributed by Vinod Kumar Vavilapalli)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401481 13f79535-47bb-0310-9956-ffa450edef68
2012-10-23 21:06:36 +00:00
Vinod Kumar Vavilapalli 4da8e2613e YARN-174. Modify NodeManager to pass the user's configuration even when rebooting. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1401086 13f79535-47bb-0310-9956-ffa450edef68
2012-10-22 21:25:12 +00:00
Alejandro Abdelnur d0a2d55b1a YARN-140. Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler. (ahmed via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400335 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 23:16:07 +00:00
Siddharth Seth f79ae91414 YARN-136. Make ClientToAMTokenSecretManager part of RMContext (Contributed by Vinod Kumar Vavilapalli)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400278 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 20:37:30 +00:00
Robert Joseph Evans ae15ad6202 YARN-163. Retrieving container log via NM webapp can hang with multibyte characters in log (jlowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400189 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 17:34:15 +00:00
Alejandro Abdelnur a4f30ff53c reverting commit 1399898 (HADOOP-8887)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399972 13f79535-47bb-0310-9956-ffa450edef68
2012-10-19 05:38:26 +00:00
Alejandro Abdelnur 53c598ab11 HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399898 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 22:40:41 +00:00
Vinod Kumar Vavilapalli 6cfa710bad YARN-32. Fix TestApplicationTokens to not depend on test order and thus pass on JDK7. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399795 13f79535-47bb-0310-9956-ffa450edef68
2012-10-18 19:12:13 +00:00
Robert Joseph Evans c63811790a MAPREDUCE-4721. Task startup time in JHS is same as job startup time. (Ravi Prakash via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399289 13f79535-47bb-0310-9956-ffa450edef68
2012-10-17 14:58:08 +00:00
Vinod Kumar Vavilapalli ab727f4966 YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. Contributed by Thomas Graves.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399066 13f79535-47bb-0310-9956-ffa450edef68
2012-10-17 01:24:50 +00:00
Vinod Kumar Vavilapalli 21b8d7b1fd YARN-161. Fix multiple compiler warnings for unchecked operations in YARN common. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399056 13f79535-47bb-0310-9956-ffa450edef68
2012-10-17 00:31:51 +00:00
Suresh Srinivas 94ed14c48c HADOOP-8911. CRLF characters in source and text files. Contributed Raja Aluri.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397432 13f79535-47bb-0310-9956-ffa450edef68
2012-10-12 04:35:42 +00:00
Thomas White 3d76be910d YARN-146. Add unit tests for computing fair share in the fair scheduler. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396972 13f79535-47bb-0310-9956-ffa450edef68
2012-10-11 10:07:01 +00:00
Todd Lipcon ff8f84ccf6 Merge trunk into branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1396918 13f79535-47bb-0310-9956-ffa450edef68
2012-10-11 06:14:26 +00:00
Siddharth Seth 0e2d93557e YARN-150. Fixes AppRejectedTransition does not unregister a rejected app-attempt from the ApplicationMasterService (Contributed by Bikas Saha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396429 13f79535-47bb-0310-9956-ffa450edef68
2012-10-10 00:08:25 +00:00
Vinod Kumar Vavilapalli a5ebccd7bc YARN-30. Fixed tests verifying web-services to work on JDK7. Contributed by Thomas Graves.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396391 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 22:54:11 +00:00
Vinod Kumar Vavilapalli bca57d471f YARN-94. Modify DistributedShell to point to main-class by default, clean up the help message, and hard-code the AM class. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1396226 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 19:40:31 +00:00
Siddharth Seth 1e30e49bf6 YARN-33. Change LocalDirsHandlerService to validate the configured local and log dirs. (Contributed by Mayank Bansal)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395844 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 02:26:24 +00:00
Siddharth Seth c074cfd6f0 YARN-134. Fixes ClientToAMSecretManager creates keys without checking for validity of the appID. (Contributed by Vinod Kumar Vavilapalli)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395841 13f79535-47bb-0310-9956-ffa450edef68
2012-10-09 01:56:05 +00:00
Vinod Kumar Vavilapalli 38e1a5a5d6 YARN-40. Fixed CHANGES.txt from previous commit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395799 13f79535-47bb-0310-9956-ffa450edef68
2012-10-08 22:35:50 +00:00
Vinod Kumar Vavilapalli 331b1123fb YARN-40. Provided support for missing YARN commands Contributed by Devaraj K and Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395793 13f79535-47bb-0310-9956-ffa450edef68
2012-10-08 22:18:42 +00:00
Vinod Kumar Vavilapalli 494e210521 YARN-127. Move RMAdmin tool to its correct location - the client module. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395781 13f79535-47bb-0310-9956-ffa450edef68
2012-10-08 21:21:38 +00:00
Alejandro Abdelnur b8f81d74c8 YARN-23. FairScheduler: FSQueueSchedulable#updateDemand() - potential redundant aggregation. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1394321 13f79535-47bb-0310-9956-ffa450edef68
2012-10-04 23:00:41 +00:00
Siddharth Seth 247a5ee2f9 YARN-102. Move the apache header to the top of the file in MemStore.java. (Contributed by Devaraj K)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1393769 13f79535-47bb-0310-9956-ffa450edef68
2012-10-03 21:19:11 +00:00
Siddharth Seth db51b03d72 YARN-131. Fix incorrect ACL properties in capacity scheduler documentation. (Contributed by Ahmed Radwan)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1393262 13f79535-47bb-0310-9956-ffa450edef68
2012-10-03 02:56:20 +00:00
Arun Murthy b146f9f9ca HADOOP-8738. Reverted since it broke MR based system tests.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1392675 13f79535-47bb-0310-9956-ffa450edef68
2012-10-01 22:42:46 +00:00
Arun Murthy 92f1f2c2a6 YARN-138. Ensure default values for minimum/maximum container sizes is sane. Contributed by Harsh J and Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391944 13f79535-47bb-0310-9956-ffa450edef68
2012-09-30 03:32:07 +00:00
Arun Murthy 70c295d90d YARN-137. Change the default YARN scheduler to be the CapacityScheduler. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391929 13f79535-47bb-0310-9956-ffa450edef68
2012-09-30 00:41:24 +00:00
Siddharth Seth 39f15579b8 YARN-116. Add the ability to change the RM include/exclude file without a restart. (Contributed by xieguiming and Harsh J)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391912 13f79535-47bb-0310-9956-ffa450edef68
2012-09-29 22:39:03 +00:00
Robert Joseph Evans af647bb3ce Preparing for release 0.23.4
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391805 13f79535-47bb-0310-9956-ffa450edef68
2012-09-29 14:15:39 +00:00
Harsh J 34becbb019 HADOOP-8386. hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu). Contributed by Christopher Berner and Andy Isaacson. (harsh)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391780 13f79535-47bb-0310-9956-ffa450edef68
2012-09-29 10:57:53 +00:00
Robert Joseph Evans 41ebf4fb24 YARN-106. Nodemanager needs to set permissions of local directories (jlowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1391649 13f79535-47bb-0310-9956-ffa450edef68
2012-09-28 20:50:44 +00:00
Vinod Kumar Vavilapalli 40062e1aaa Fix NodeManager to verify the application's user-name.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390825 13f79535-47bb-0310-9956-ffa450edef68
2012-09-27 03:43:57 +00:00
Thomas Graves 050fd3a117 MAPREDUCE-4647. We should only unjar jobjar if there is a lib directory in it. (Robert Evans via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390557 13f79535-47bb-0310-9956-ffa450edef68
2012-09-26 15:22:21 +00:00
Arun Murthy 8912aac81e YARN-9. Rename YARN_HOME to HADOOP_YARN_HOME. Contributed by Vinod K V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390218 13f79535-47bb-0310-9956-ffa450edef68
2012-09-25 23:37:32 +00:00
Todd Lipcon b1ceaac3e6 Merge trunk into branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1390199 13f79535-47bb-0310-9956-ffa450edef68
2012-09-25 22:43:04 +00:00
Vinod Kumar Vavilapalli d524942289 YARN-53. Added the missing getGroups API to ResourceManager. Contributed by Bo Wang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389176 13f79535-47bb-0310-9956-ffa450edef68
2012-09-24 00:51:51 +00:00
Siddharth Seth bfe3816c66 YARN-88. DefaultContainerExecutor can fail to set proper permissions. (Contributed by Jason Lowe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388580 13f79535-47bb-0310-9956-ffa450edef68
2012-09-21 17:29:33 +00:00
Siddharth Seth ff7b818432 YARN-82. Correct the fix version in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388340 13f79535-47bb-0310-9956-ffa450edef68
2012-09-21 06:34:06 +00:00
Siddharth Seth f4c336d547 YARN-82. Change the default local and log dirs to be based on hadoop.tmp.dir and yarn.log.dir. (Contributed by Hemanth Yamijala)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388339 13f79535-47bb-0310-9956-ffa450edef68
2012-09-21 06:28:47 +00:00
Todd Lipcon e9f4de5ced Merge trunk into branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1387449 13f79535-47bb-0310-9956-ffa450edef68
2012-09-19 04:34:55 +00:00
Vinod Kumar Vavilapalli 37d5618742 YARN-28. Fixed TestCompositeService to not depend on test-order and thus made it pass on JDK7. Contributed by Thomas Graves.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1386907 13f79535-47bb-0310-9956-ffa450edef68
2012-09-17 23:30:27 +00:00
Robert Joseph Evans 2526a96aaa YARN-108. FSDownload can create cache directories with the wrong permissions (Jason Lowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1386838 13f79535-47bb-0310-9956-ffa450edef68
2012-09-17 21:05:11 +00:00
Aaron Myers d8e386f01a HADOOP-8755. Print thread dump when tests fail due to timeout. Contributed by Andrey Klochkov.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384627 13f79535-47bb-0310-9956-ffa450edef68
2012-09-14 01:44:23 +00:00
Aaron Myers 853db9ec24 HDFS-3926. QJM: Add user documentation for QJM. Contributed by Aaron T. Myers.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/HDFS-3077@1384595 13f79535-47bb-0310-9956-ffa450edef68
2012-09-13 23:02:32 +00:00
Vinod Kumar Vavilapalli 7c491887e5 YARN-93. Fixed RM to propagate diagnostics from applications that have finished but failed Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1384169 13f79535-47bb-0310-9956-ffa450edef68
2012-09-13 00:46:39 +00:00
Vinod Kumar Vavilapalli 11cf00b08a YARN-78. Changed UnManagedAM application to use YarnClient. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1383705 13f79535-47bb-0310-9956-ffa450edef68
2012-09-12 00:49:17 +00:00
Arun Murthy 681408f898 Set date for hadoop-2.0.2-alpha
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382199 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 22:39:45 +00:00
Arun Murthy 7851676f22 Preparing for hadoop-2.0.2-alpha release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382191 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 22:09:48 +00:00
Arun Murthy a1aa1b9349 YARN-57. Allow process-tree based resource calculation et al. to be pluggable to support it on multiple platforms. Contributed by Radim Kolar.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1382072 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 15:48:39 +00:00
Arun Murthy 56d72dfdf1 YARN-80. Add support for delaying rack-local containers in CapacityScheduler. Contributed by Arun C. Murthy.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381872 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 05:10:51 +00:00
Vinod Kumar Vavilapalli e4679b5caf YARN-75. Missing test. Ugh.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381811 13f79535-47bb-0310-9956-ffa450edef68
2012-09-07 00:04:40 +00:00
Vinod Kumar Vavilapalli e9411ae5e3 YARN-75. Added file missing from previous commit. Bad day with svn.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381809 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 23:55:10 +00:00
Vinod Kumar Vavilapalli 68ad1716b2 YARN-75. Removing spurious file from the previous commit.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381807 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 23:47:41 +00:00
Vinod Kumar Vavilapalli 3a4e87e813 YARN-75. Modified ResourceManager's RMContainer to handle a valid RELEASE event at RUNNING state. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381806 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 23:46:31 +00:00
Vinod Kumar Vavilapalli d3224aff28 YARN-15. Fixing CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381801 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 23:36:27 +00:00
Vinod Kumar Vavilapalli d093dcce45 YARN-15. Updated default classpath for YARN applications to reflect split of YARN into a sub-project. Contributed by Arun C Murthy.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381794 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 23:06:35 +00:00
Thomas Graves 02c0317eff YARN-87. NM ResourceLocalizationService does not set permissions of local cache directories (Jason Lowe via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381459 13f79535-47bb-0310-9956-ffa450edef68
2012-09-06 02:08:37 +00:00
Robert Joseph Evans 726b48f51a YARN-68. NodeManager will refuse to shutdown indefinitely due to container log aggregation (daryn via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381317 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 19:39:53 +00:00
Vinod Kumar Vavilapalli 87c5d243db YARN-83. Change package of YarnClient to org.apache.hadoop. Contributed by Bikas Saha.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1381273 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 17:54:32 +00:00
Vinod Kumar Vavilapalli 65b308f783 YARN-42. Modify NM's non-aggregating logs' handler to stop properly so that NMs don't get NPEs on startup errors. Contributed by Devaraj K.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380954 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 02:46:07 +00:00
Vinod Kumar Vavilapalli ab74b1adde YARN-79. Implement close on all clients to YARN so that RPC clients don't throw exceptions on shut-down. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380942 13f79535-47bb-0310-9956-ffa450edef68
2012-09-05 00:14:35 +00:00
Suresh Srinivas 60af999c73 YARN-84. Use Builder to build RPC server. Contributed by Brandon Li
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380921 13f79535-47bb-0310-9956-ffa450edef68
2012-09-04 22:24:04 +00:00
Robert Joseph Evans f5943c1eef Updated CHANGES.txt for 0.23.4
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380906 13f79535-47bb-0310-9956-ffa450edef68
2012-09-04 21:45:33 +00:00
Vinod Kumar Vavilapalli b1632b4a77 YARN-10. Fix DistributedShell module to not have a dependency on hadoop-mapreduce-client-core. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1380879 13f79535-47bb-0310-9956-ffa450edef68
2012-09-04 20:56:53 +00:00
Robert Joseph Evans 60b8c6e1e1 YARN-66. aggregated logs permissions not set properly (tgraves via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379565 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 19:57:34 +00:00
Robert Joseph Evans 27cfde7f4d YARN-60: Fix minor merge conflict (bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379555 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 19:24:31 +00:00
Vinod Kumar Vavilapalli 45a8e8c5a4 YARN-60. Fixed a bug in ResourceManager which causes all NMs to get NPEs and thus causes all containers to be rejected. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379550 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 19:11:05 +00:00
Robert Joseph Evans d8ce3bee58 YARN-63. RMNodeImpl is missing valid transitions from the UNHEALTHY state (Jason Lowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1379498 13f79535-47bb-0310-9956-ffa450edef68
2012-08-31 16:33:21 +00:00
Arun Murthy 8449526f0c Moved MAPREDUCE-4323 to YARN-58 and updated CHANGES.txt to reflect this.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378637 13f79535-47bb-0310-9956-ffa450edef68
2012-08-29 16:51:33 +00:00
Alejandro Abdelnur 54d4630d9b HADOOP-8738. junit JAR is showing up in the distro (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1378175 13f79535-47bb-0310-9956-ffa450edef68
2012-08-28 15:30:43 +00:00
Vinod Kumar Vavilapalli 7fff7b42bf YARN-31. Fix TestDelegationTokenRenewer to not depend on test order so as to pass tests on jdk7. Contributed by Thomas Graves.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377961 13f79535-47bb-0310-9956-ffa450edef68
2012-08-28 03:31:11 +00:00
Harsh J cb540cda69 HADOOP-8719. Workaround for kerberos-related log errors upon running any hadoop command on OSX. Contributed by Jianbin Wei. (harsh)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377821 13f79535-47bb-0310-9956-ffa450edef68
2012-08-27 20:16:16 +00:00
Siddharth Seth 0ec6cda4b7 YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Contributed by Mayank Bansal)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377803 13f79535-47bb-0310-9956-ffa450edef68
2012-08-27 19:35:45 +00:00
Siddharth Seth 10e704c50b YARN-29. Add a yarn-client module. (Contributed by Vinod Kumar Vavilapalli)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377781 13f79535-47bb-0310-9956-ffa450edef68
2012-08-27 18:27:18 +00:00
Siddharth Seth ffd2e01604 YARN-39. RM-NM secret-keys should be randomly generated and rolled every so often. (Contributed by Vinod Kumar Vavilapalli and Siddharth Seth)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1377180 13f79535-47bb-0310-9956-ffa450edef68
2012-08-25 02:18:49 +00:00
Aaron Myers 9ef3b2eb60 MAPREDUCE-2374. "Text File Busy" errors launching MR tasks. Contributed by Andy Isaacson.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1376632 13f79535-47bb-0310-9956-ffa450edef68
2012-08-23 18:24:13 +00:00
Siddharth Seth 67673227f2 YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Contributed by Mayank Bansal)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375829 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 22:27:11 +00:00
Robert Joseph Evans ca2dc3e785 MAPREDUCE-4570. ProcfsBasedProcessTree#constructProcessInfo() prints a warning if procfsDir/<pid>/stat is not found. (Ahmed Radwan via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375687 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 17:46:07 +00:00
Jonathan Turner Eagles df5e2b8352 MAPREDUCE-4323. NM leaks filesystems (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375164 13f79535-47bb-0310-9956-ffa450edef68
2012-08-20 18:32:13 +00:00
Thomas Graves ea7af8f76e YARN-27. Fixed location of ms.start call
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375083 13f79535-47bb-0310-9956-ffa450edef68
2012-08-20 16:04:03 +00:00
Thomas Graves bc538e100a YARN-27. Failed refreshQueues due to misconfiguration prevents further refreshing of queues (Arun Murthy via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1375066 13f79535-47bb-0310-9956-ffa450edef68
2012-08-20 15:36:51 +00:00
Tsz-wo Sze b0ea77303b HADOOP-8240. Add a new API to allow users to specify a checksum type on FileSystem.create(..). Contributed by Kihwal Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374696 13f79535-47bb-0310-9956-ffa450edef68
2012-08-19 05:46:57 +00:00
Thomas Graves 7fc6ad661d YARN-25. remove old aggregated logs (Robert Evans via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1374424 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 20:29:38 +00:00
Thomas White 4e09b481e7 HADOOP-8278. Make sure components declare correct set of dependencies.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1373574 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 19:10:52 +00:00
Alejandro Abdelnur 4bca22005f HADOOP-8681. add support for HTTPS to the web UIs. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371525 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 22:52:56 +00:00
Todd Lipcon 0733adf670 HADOOP-8659. Native libraries must build with soft-float ABI for Oracle JVM on ARM. Contributed by Trevor Robinson.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371507 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 21:58:21 +00:00
Robert Joseph Evans 82910ecaa3 YARN-14. Symlinks to peer distributed cache files no longer work (Jason Lowe via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1371390 13f79535-47bb-0310-9956-ffa450edef68
2012-08-09 19:15:56 +00:00
Arun Murthy 4920387b8a YARN-12. Fix findbugs warnings in FairScheduler. Contributed by Junping Du.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370889 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 18:53:29 +00:00
Arun Murthy b5683210b6 YARN-1. Fixed message in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370668 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 05:36:50 +00:00
Arun Murthy e1fdf62123 YARN-1. Promote YARN to be a sub-project of Apache Hadoop.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1370666 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 05:22:27 +00:00