Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,111 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

304 Commits

Author SHA1 Message Date
Christopher Douglas e8585afa03 YARN-45. Add protocol for schedulers to request containers back from 
ApplicationMasters. Contributed by Carlo Curino and Chris Douglas.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1479773 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 06:18:25 +00:00
Vinod Kumar Vavilapalli 2136542f1b YARN-632. Changed ContainerManager api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1479740 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1479741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:43:32 +00:00
Vinod Kumar Vavilapalli c114469fbd YARN-633. Changed RMAdminProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1479738 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1479739 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-07 00:38:57 +00:00
Vinod Kumar Vavilapalli 486391f766 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1479680 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1479684 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:20:02 +00:00
Vinod Kumar Vavilapalli 8b755237f8 YARN-645. Moved RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager where it really belongs. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1479022 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1479026 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 01:13:43 +00:00
Vinod Kumar Vavilapalli 0b64df0630 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
svn merge --ignore-ancestry -c 1478594 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1478595 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:45:30 +00:00
Vinod Kumar Vavilapalli fbdc7ed67d YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1478230 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1478235 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 23:48:32 +00:00
Vinod Kumar Vavilapalli e2a1f994a1 YARN-599. Refactoring submitApplication in ClientRMService and RMAppManager to separate out various validation checks depending on whether they rely on RM configuration or not. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1477478 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1477479 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-30 05:44:45 +00:00
Vinod Kumar Vavilapalli 373db7be2a YARN-516. Reverting the patch from branch-2 as HADOOP-9357 also got reverted. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1476839 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 21:15:39 +00:00
Vinod Kumar Vavilapalli d5331b18a9 YARN-576. Modified ResourceManager to reject NodeManagers that don't satisy minimum resource requirements. Contributed by Kenji Kikushima. 
svn merge --ignore-ancestry -c 1476824 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1476825 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 20:04:47 +00:00
Vinod Kumar Vavilapalli ffe3368d94 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1476346 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 19:08:44 +00:00
Vinod Kumar Vavilapalli 141fe2f5e5 YARN-363. Add webapps/proxy directory without which YARN proxy-server fails when started in stand-alone mode. Contributed by Kenji Kikushima. 
svn merge --ignore-ancestry -c 1476330 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1476332 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:31:04 +00:00
Vinod Kumar Vavilapalli 27404b1190 YARN-591. Moved RM recovery related records out of public API as they do not belong there. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1476324 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1476327 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 18:04:24 +00:00
Thomas White 07ab02a982 Merge -r 1475680:1475681 from trunk to branch-2. Fixes: YARN-289. Fair scheduler allows reservations that won't fit on node. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1475682 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 09:18:24 +00:00
Thomas White 3bed8dc7b9 Merge -r 1475669:1475670 from trunk to branch-2. Fixes: YARN-595. Refactor fair scheduler to use common Resources. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1475672 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 08:36:22 +00:00
Vinod Kumar Vavilapalli a1fe9a0c9f YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.
svn merge --ignore-ancestry -c 1475636 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1475637 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:45:15 +00:00
Siddharth Seth c4feeefb2b merge YARN-579 from trunk. Stop setting the Application Token in the AppMaster env, in favour of the copy present in the container token field. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1471815 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:48:18 +00:00
Vinod Kumar Vavilapalli 909e52274c YARN-605. Fix failing unit test in TestNMWebServices when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
svn merge --ignore-ancestry -c 1471608 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1471609 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 19:43:23 +00:00
Vinod Kumar Vavilapalli 2dfecb78e5 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1471207 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 00:01:08 +00:00
Vinod Kumar Vavilapalli 700a6aa0a0 YARN-581. Added a test to verify that app delegation tokens are restored after RM restart. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1471187 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1471188 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 22:51:19 +00:00
Vinod Kumar Vavilapalli 23f099da0b YARN-583. Moved application level local resources to be localized under the filecache sub-directory under application directory. Contributed by Omkar Vinit Joshi. 
svn merge --ignore-ancestry -c 1470812 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1470813 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 04:38:45 +00:00
Bikas Saha 778b64e29f Merge -c r1470797 from trunk to branch-2 for YARN-549. YarnClient.submitApplication should wait for application to be accepted by the RM (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1470798 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 03:12:54 +00:00
Bikas Saha 66beaf2c7d Merge -c r1470243 from trunk to branch-2 for YARN-594. Update test and add comments in YARN-534 (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1470245 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 22:06:41 +00:00
Vinod Kumar Vavilapalli e2b5e3c2b4 YARN-542. Changed the default global AM max-attempts value to be not one. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1470094 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1470095 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 00:36:07 +00:00
Vinod Kumar Vavilapalli a02788f864 YARN-547. Fixed race conditions in public and private resource localization which used to cause duplicate downloads. Contributed by Omkar Vinit Joshi. 
svn merge --ignore-ancestry -c 1470076 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1470078 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 22:36:52 +00:00
Vinod Kumar Vavilapalli 6c37becf87 YARN-586. Fixed a typo in ApplicationSubmissionContext#setApplicationId. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1470068 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1470069 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 22:25:53 +00:00
Vinod Kumar Vavilapalli 1bfb6af49f YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1469657 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1469658 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:34:04 +00:00
Alejandro Abdelnur 15d5394221 YARN-482. FS: Extend SchedulingMode to intermediate queues. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1469511 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 18:13:36 +00:00
Vinod Kumar Vavilapalli 27bef76501 YARN-585. Fix failure in TestFairScheduler#testNotAllowSubmitApplication caused by YARN-514. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1469492 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1469494 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 17:49:57 +00:00
Alejandro Abdelnur 4198eaf00d YARN-476. ProcfsBasedProcessTree info message confuses users. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1469094 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 23:12:37 +00:00
Alejandro Abdelnur dc21be9026 YARN-518. Fair Scheduler's document link could be added to the hadoop 2.x main doc page. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1469092 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 23:06:30 +00:00
Bikas Saha 039d673794 Merge -c r1469059 from trunk to branch-2 for YARN-514. Delayed store operations should not result in RM unavailability for app submission (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1469062 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 20:30:48 +00:00
Vinod Kumar Vavilapalli 3d2430fad1 YARN-500. Fixed YARN webapps to not roll-over ports when explicitly asked to use non-ephemeral ports. Contributed by Kenji Kikushima. 
svn merge --ignore-ancestry -c 1468739 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1468740 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 04:17:49 +00:00
Vinod Kumar Vavilapalli 8cc3b081fa YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.
svn merge --ignore-ancestry -c 1468276 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1468277 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:55:26 +00:00
Arun Murthy 8acaea0028 YARN-412. Pushing to 2.0.5-beta only. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1467471 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-12 21:20:45 +00:00
Arun Murthy ac7d455a0f Merge -c 1467244 from trunk to branch-2 to fix YARN-412. Fixed FifoScheduler to check hostname of a NodeManager rather than its host:port during scheduling which caused incorrect locality for containers. Contributed by Roger Hoover. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1467245 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-12 12:15:17 +00:00
Vinod Kumar Vavilapalli 3a1ab1bc98 (Missing commit message from the previous commit) 
YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong.
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1467131 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 23:05:24 +00:00
Vinod Kumar Vavilapalli a84c8bcbd4 git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1467130 13f79535-47bb-0310-9956-ffa450edef68 2013-04-11 23:04:09 +00:00
Vinod Kumar Vavilapalli 51dc407f30 YARN-319. Submitting a job to a fair scheduler queue for which the user does not have permission causes the client to wait forever. Contributed by Shenhong. 
svn merge --ignore-ancestry -c 1437336 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1467110 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:00:10 +00:00
Thomas Graves 61ce78582f Preparing for release 0.23.7 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1467070 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 20:09:26 +00:00
Vinod Kumar Vavilapalli d6f8a7f7ee YARN-539. Addressed memory leak of LocalResource objects NM when a resource localization fails. Contributed by Omkar Vinit Joshi. 
svn merge --ignore-ancestry -c 1466756 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1466757 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:08:49 +00:00
Vinod Kumar Vavilapalli 81a684ebd3 YARN-495. Changed NM reboot behaviour to be a simple resync - kill all containers and re-register with RM. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1466752 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1466753 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 02:01:44 +00:00
Arun Murthy 91a676fc41 Setting version in branch-2 to be 2.0.5-SNAPSHOT. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1466320 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-10 03:16:30 +00:00
Vinod Kumar Vavilapalli fd5224e62c YARN-534. Change RM restart recovery to also account for AM max-attempts configuration after the restart. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1466208 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1466209 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 20:16:56 +00:00
Vinod Kumar Vavilapalli 76d72db5eb YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1466196 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1466197 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 19:57:10 +00:00
Vinod Kumar Vavilapalli 335e02124b YARN-99. Modify private distributed cache to localize files such that no local directory hits unix file count limits and thus prevent job failures. Contributed by Omkar Vinit Joshi. 
svn merge --ignore-ancestry -c 1465853 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1465854 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 01:35:05 +00:00
Bikas Saha 0aa44dc7ac Merge -c 1465731 from trunk to branch-2 for YARN-479. NM retry behavior for connection to RM should be similar for lost heartbeats (Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1465733 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-08 19:19:33 +00:00
Bikas Saha e97e0eaec7 Merge -c 1465067 from trunk to branch-2 for YARN-193. Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1465069 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 17:48:35 +00:00
Jason Darrell Lowe d002c038b2 svn merge -c 1465009 FIXES: YARN-525. make CS node-locality-delay refreshable. Contributed by Thomas Graves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1465012 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 15:34:40 +00:00
Vinod Kumar Vavilapalli 53b3dd18a5 YARN-532. Change RMAdmin and Localization client protocol PB implementations to implement closeable so that they can be stopped when needed via RPC.stopProxy(). Contributed by Siddharth Seth. 
svn merge --ignore-ancestry -c 1464788 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1464789 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 23:31:12 +00:00