Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,087 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

172 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Vinod Kumar Vavilapalli 0727ecaf50 YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong. 
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 22:53:01 +00:00
Vinod Kumar Vavilapalli f6167ef6c6 MAPREDUCE-5157. Bring back old sampler related code so that we can support binary compatibility with hadoop-1 sorter example. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480474 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-08 21:42:47 +00:00
Vinod Kumar Vavilapalli 92b7165a71 YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong. 
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-06 19:04:31 +00:00
Aaron Myers ca2c38fba1 MAPREDUCE-5193. A few MR tests use block sizes which are smaller than the default minimum block size. Contributed by Andrew Wang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478250 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 01:59:36 +00:00
Vinod Kumar Vavilapalli 448df18646 YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah. 
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-25 05:42:22 +00:00
Arun Murthy 40e78c2ca2 MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. Contributed by Arun C. Murthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471556 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 17:38:43 +00:00
Robert Joseph Evans 0777474cc2 MAPREDUCE-5069. add concrete common implementations of CombineFileInputFormat (Sangjin Lee via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471424 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-24 14:11:50 +00:00
Bikas Saha c1ce3c5ad2 MAPREDUCE-4987. TestMRJobs#testDistributedCache fails on Windows due to classpath problems and unexpected behavior of symlinks (Chris Nauroth via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470003 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-19 19:32:33 +00:00
Thomas Graves 9ea20fc537 MAPREDUCE-5015. Coverage fix for org.apache.hadoop.mapreduce.tools.CLI (Aleksey Gorshkov via tgraves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468483 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-16 16:11:35 +00:00
Konstantin Shvachko 8e838bd03c MAPREDUCE-4985. Add compression option to TestDFSIO usage. Contributed by Plamen Jeliazkov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467145 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 23:49:16 +00:00
Vinod Kumar Vavilapalli 3c8bd60616 MAPREDUCE-5094. Disabled memory monitoring by default in MiniMRYarnCluster to avoid some downstream tests failing. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467124 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-11 22:54:02 +00:00
Konstantin Boudnik d18cc69d4e MAPREDUCE-5088. MR Client gets an renewer token exception while Oozie is submitting a job (daryn) 
merge -c1463804 from branch-2.0.4-alpha



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464153 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-03 18:45:15 +00:00
Siddharth Seth d41e67b966 MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related code. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461251 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 18:15:40 +00:00
Robert Joseph Evans a0a844222d MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461146 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-26 14:16:13 +00:00
Hitesh Shah b8c74a8a25 MAPREDUCE-5083. MiniMRCluster should use a random component when creating an actual cluster. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459932 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-22 19:11:01 +00:00
Robert Joseph Evans 7d7553c4eb MAPREDUCE-5042. Reducer unable to fetch for a map task that was recovered (Jason Lowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457119 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-15 21:09:25 +00:00
Suresh Srinivas 6942fd1db5 HADOOP-9372. Fix bad timeout annotations on tests. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453637 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-07 01:09:53 +00:00
Suresh Srinivas 638801cce1 HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-06 19:15:18 +00:00
Alejandro Abdelnur 892e088773 HADOOP-9349. Confusing output when running hadoop version from one hadoop installation when HADOOP_HOME points to another. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451448 13f79535-47bb-0310-9956-ffa450edef68
 2013-03-01 01:20:53 +00:00
Vinod Kumar Vavilapalli 0b9ed2364a MAPREDUCE-4892. Modify CombineFileInputFormat to not skew input slits' allocation on small clusters. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450912 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-27 18:49:37 +00:00
Alejandro Abdelnur ab330b33bb MAPREDUCE-4994. Addendum fixing testcases failures. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1446042 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-14 04:22:04 +00:00
Robert Joseph Evans 79b12f6f4c MAPREDUCE-4905. test org.apache.hadoop.mapred.pipes (Aleksey Gorshkov via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443027 13f79535-47bb-0310-9956-ffa450edef68
 2013-02-06 15:50:23 +00:00
Suresh Srinivas ac21e15a39 MAPREDUCE-4969. TestKeyValueTextInputFormat test fails with Open JDK 7. Contributed by Arpit Agarwal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1441069 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-31 17:11:36 +00:00
Siddharth Seth 8a5fadc5a0 MAPREDUCE-4803. Remove duplicate copy of TestIndexCache. Contributed by Mariappan Asokan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1439590 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-28 19:18:35 +00:00
Jonathan Turner Eagles 88c7fdd065 MAPREDUCE-4458. Warn if java.library.path is used for AM or Task (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435386 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 22:41:19 +00:00
Siddharth Seth 76f9af4ccf MAPREDUCE-4948. Fix a failing unit test - TestYARNRunner.testHistoryServerToken. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1435312 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-18 19:30:00 +00:00
Alejandro Abdelnur 26275e5323 MAPREDUCE-4924. flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR. (rkanter via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1433753 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-16 00:08:25 +00:00
Robert Joseph Evans 12293f8a13 MAPREDUCE-4921. JobClient should acquire HS token with RM principal (daryn via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1432230 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-11 19:00:51 +00:00
Vinod Kumar Vavilapalli 5f7d4d2b45 MAPREDUCE-4810. Added new admin command options for MR AM. Contributed by Jerry Chen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1430707 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-09 06:31:36 +00:00
Thomas Graves e17cecf550 MAPREDUCE-4894. Renewal / cancellation of JobHistory tokens (Siddharth Seth via tgraves 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1429086 13f79535-47bb-0310-9956-ffa450edef68
 2013-01-04 20:15:43 +00:00
Luke Lu 1a49c85438 YARN-223. Update process tree instead of getting new process trees. (Radim Kolar via llu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1424244 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-20 00:20:53 +00:00
Alejandro Abdelnur 8329fae686 MAPREDUCE-4807. Allow MapOutputBuffer to be pluggable. (masokan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1422345 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-15 20:23:08 +00:00
Thomas White 4993314895 MAPREDUCE-4856. TestJobOutputCommitter uses same directory as TestJobCleanup. Contributed by Sandy Ryza. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420634 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-12 12:12:10 +00:00
Alejandro Abdelnur c5080ef336 MAPREDUCE-4861. Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal. (kkambatl via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1420345 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-11 19:26:57 +00:00
Alejandro Abdelnur 054270cc28 MAPREDUCE-4703. Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. (ahmed.radwan via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1419618 13f79535-47bb-0310-9956-ffa450edef68
 2012-12-10 18:07:20 +00:00
Robert Joseph Evans 481f451ca2 MAPREDUCE-4764. repair TestBinaryTokenFile (Ivan A. Veselovsky via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1413739 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-26 17:37:05 +00:00
Thomas White e464607b7d YARN-129. Simplify classpath construction for mini YARN tests. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1411235 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-19 15:12:22 +00:00
Robert Joseph Evans 1f40b8b4e8 MAPREDUCE-4782. NLineInputFormat skips first line of last InputSplit (Mark Fuhs via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1407505 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-09 15:52:33 +00:00
Thomas White 8d483fff2c MAPREDUCE-4777. In TestIFile, testIFileReaderWithCodec relies on testIFileWriterWithCodec. Contributed by Sandy Ryza 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1406645 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-07 14:11:59 +00:00
Robert Joseph Evans 0eadfcfcf5 MAPREDUCE-4771. KeyFieldBasedPartitioner not partitioning properly when configured (jlowe via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1405975 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-05 21:52:06 +00:00
Robert Joseph Evans 1783f209f0 MAPREDUCE-4763 repair test TestUmbilicalProtocolWithJobToken (Ivan A. Veselovsky via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1404988 13f79535-47bb-0310-9956-ffa450edef68
 2012-11-02 15:00:56 +00:00
Alejandro Abdelnur 16e21dfe92 MAPREDUCE-1806. CombineFileInputFormat does not work with paths not on default FS. (Gera Shegalov via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1403614 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-30 05:41:59 +00:00
Suresh Srinivas c195ce4d67 MAPREDUCE-4736. Remove obsolete option [-rootDir] from TestDFSIO. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400313 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 22:05:30 +00:00
Robert Joseph Evans a234472d1c MAPREDUCE-4479. Fix parameter order in assertEquals() in TestCombineInputFileFormat.java (Mariappan Asokan via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1400192 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 17:51:33 +00:00
Suresh Srinivas adcf7d10b9 MAPREDUCE-4735. Make arguments in TestDFSIO case insensitive. Contributed by Brandon Li. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1399952 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-19 02:37:59 +00:00
Suresh Srinivas 94ed14c48c HADOOP-8911. CRLF characters in source and text files. Contributed Raja Aluri. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1397432 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-12 04:35:42 +00:00
Harsh J 5c3a331040 MAPREDUCE-4574. Fix TotalOrderParitioner to work with non-WritableComparable key types. Contributed by Harsh J. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395936 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-09 09:58:21 +00:00
Robert Joseph Evans 49b20c2ed1 MAPREDUCE-4554. Job Credentials are not transmitted if security is turned off (Benoy Antony via bobby) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1395769 13f79535-47bb-0310-9956-ffa450edef68
 2012-10-08 20:50:40 +00:00
Arun Murthy 036530fc12 Reverted MAPREDUCE-4253. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1390652 13f79535-47bb-0310-9956-ffa450edef68
 2012-09-26 18:41:08 +00:00