Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,087 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

699 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 47d1ca402f MAPREDUCE-5208. Modified ShuffleHandler to use SecureIOUtils for reading local files. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481657 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-12 21:59:16 +00:00
Vinod Kumar Vavilapalli 2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Alejandro Abdelnur d303f01297 MAPREDUCE-4942. mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480748 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 20:04:32 +00:00
Vinod Kumar Vavilapalli 9c4f86879c YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-09 05:32:04 +00:00
Vinod Kumar Vavilapalli 0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli f6167ef6c6 MAPREDUCE-5157. Bring back old sampler related code so that we can support binary compatibility with hadoop-1 sorter example. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480474 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 21:42:47 +00:00
Christopher Douglas 67d7d8f4d3 MAPREDUCE-5209. Fix units in a ShuffleScheduler log message. 
Contributed by Tsuyoshi OZAWA


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480464 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 21:12:27 +00:00
Vinod Kumar Vavilapalli 8888d3fc49 MAPREDUCE-5159. Change ValueAggregatorJob to add APIs which can support binary compatibility with hadoop-1 examples. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480394 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 18:28:22 +00:00
Vinod Kumar Vavilapalli 92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Vinod Kumar Vavilapalli 6e37a7e5e1 MAPREDUCE-5205. Fixed MR App to load tokens correctly. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479016 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 00:27:47 +00:00
Vinod Kumar Vavilapalli fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:44:12 +00:00
Thomas White 9b97df9abe MAPREDUCE-5036. Default shuffle handler port should not be 8080. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478422 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 15:58:53 +00:00
Aaron Myers ca2c38fba1 MAPREDUCE-5193. A few MR tests use block sizes which are smaller than the default minimum block size. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478250 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 01:59:36 +00:00
Vinod Kumar Vavilapalli 71be8ad325 MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs inline with YARN. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477889 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 00:42:32 +00:00
Suresh Srinivas 8f7ce62085 MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable & FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477403 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-29 23:00:39 +00:00
Vinod Kumar Vavilapalli ae393ab26e MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476835 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-28 21:02:25 +00:00
Suresh Srinivas fe3853484e HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-27 14:24:02 +00:00
Vinod Kumar Vavilapalli fbb55784d9 YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He. 
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 03:50:20 +00:00
Vinod Kumar Vavilapalli 448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Siddharth Seth e96077aaf4 MAPREDUCE-5181. RMCommunicator should not use AMToken from the env. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471816 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 02:50:16 +00:00
Arun Murthy daaa73b657 MAPREDUCE-5166. Fix ConcurrentModificationException due to insufficient synchronization on updates to task Counters. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471796 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 00:54:04 +00:00
Arun Murthy 40e78c2ca2 MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471556 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 17:38:43 +00:00
Robert Joseph Evans 0777474cc2 MAPREDUCE-5069. add concrete common implementations of CombineFileInputFormat (Sangjin Lee via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471424 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 14:11:50 +00:00
Vinod Kumar Vavilapalli 27e8c86999 YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong. 
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-23 21:39:51 +00:00
Thomas White 009af54d51 MAPREDUCE-5146. application classloader may be used too early to load classes. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470694 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-22 20:07:06 +00:00
Arun Murthy 794f9bb3e4 MAPREDUCE-5066. Added a timeout for the job.end.notification.url. Contributed by Ivan Mitic. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470216 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-20 19:18:24 +00:00
Thomas Graves febc52be37 MAPREDUCE-5147. Maven build should create hadoop-mapreduce-client-app-VERSION.jar directly (Robert Parker via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470035 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 20:50:48 +00:00
Bikas Saha c1ce3c5ad2 MAPREDUCE-4987. TestMRJobs#testDistributedCache fails on Windows due to classpath problems and unexpected behavior of symlinks (Chris Nauroth via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470003 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 19:32:33 +00:00
Vinod Kumar Vavilapalli 8e1c2823fc YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong. 
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 01:33:13 +00:00
Alejandro Abdelnur e222d1e783 MAPREDUCE-4932. mapreduce.job#getTaskCompletionEvents incompatible with Hadoop 1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469587 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 21:42:03 +00:00
Vinod Kumar Vavilapalli cb78a65a15 MAPREDUCE-5152. Make MR App to simply pass through the container from RM instead of extracting and populating information itself to start any container. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469544 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 20:13:40 +00:00
Alejandro Abdelnur d12c463121 MAPREDUCE-4898. FileOutputFormat.checkOutputSpecs and FileOutputFormat.setOutputPath incompatible with MR1. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469523 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-18 18:51:45 +00:00
Alejandro Abdelnur cd27afa71a MAPREDUCE-5128. mapred-default.xml is missing a bunch of history server configs. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469095 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 23:14:34 +00:00
Bikas Saha 9e650d36b7 MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469060 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-17 20:25:21 +00:00
Thomas Graves 9ea20fc537 MAPREDUCE-5015. Coverage fix for org.apache.hadoop.mapreduce.tools.CLI (Aleksey Gorshkov via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468483 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-16 16:11:35 +00:00
Vinod Kumar Vavilapalli 0e01f26821 YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza. 
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 23:54:09 +00:00
Robert Joseph Evans 947e97f354 MAPREDUCE-4974. Optimising the LineRecordReader initialize() method (Gelesh via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468232 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-15 21:33:47 +00:00
Konstantin Shvachko 8e838bd03c MAPREDUCE-4985. Add compression option to TestDFSIO usage. Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467145 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 23:49:16 +00:00
Vinod Kumar Vavilapalli 3c8bd60616 MAPREDUCE-5094. Disabled memory monitoring by default in MiniMRYarnCluster to avoid some downstream tests failing. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467124 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:54:02 +00:00
Vinod Kumar Vavilapalli 5fd460e688 MAPREUDUCE-5059. Change average merge time on Job overview page to be the time delta between the end of the shuffle and the start of the reduce. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467120 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:27:45 +00:00
Vinod Kumar Vavilapalli e4c55e17fe YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong. 
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:28:51 +00:00
Jason Darrell Lowe 8e86ffd91e MAPREDUCE-5136. TestJobImpl->testJobNoTasks fails with IBM JAVA. Contributed by Amir Sanjar 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467061 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 19:12:59 +00:00
Jason Darrell Lowe d04e65b797 MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466948 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 16:27:24 +00:00
Siddharth Seth 6a1c41111e MAPREDUCE-5079. Changes job recovery to restore state directly from job history, instaed of simulating state machine events. Contributed by Jason Lowe and Robert Parker. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466767 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 04:52:38 +00:00
Vinod Kumar Vavilapalli 3a54a5653b YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi. 
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-09 19:56:10 +00:00
Arun Murthy 7f13207ed1 MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. Contributed by Billie Rinaldi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465188 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-06 05:30:56 +00:00
Thomas Graves a5734cd38a MAPREDUCE-5007. fix coverage org.apache.hadoop.mapreduce.v2.hs (Aleksey Gorshkov via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465017 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-05 15:43:53 +00:00
Jason Darrell Lowe fc75d3f3dc MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot command from the RM. Contributed by Jian He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464255 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-04 01:56:49 +00:00
Konstantin Boudnik d18cc69d4e MAPREDUCE-5088. MR Client gets an renewer token exception while Oozie is submitting a job (daryn) 
merge -c1463804 from branch-2.0.4-alpha



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464153 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 18:45:15 +00:00
Vinod Kumar Vavilapalli 38678cc3d9 MAPREDUCE-5117. Changed MRClientProtocolPBClientImpl to be closeable and thus fix failures in renewal of HistoryServer's delegations tokens. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463828 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 05:37:10 +00:00