Commit Graph

2042 Commits

Author SHA1 Message Date
Zhijie Shen 84bc2fe402 MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618269 13f79535-47bb-0310-9956-ffa450edef68
2014-08-15 20:17:45 +00:00
Allen Wittenauer 31e478905a Wrong categorization for a variety of commits in the CHANGES.txt files
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618233 13f79535-47bb-0310-9956-ffa450edef68
2014-08-15 16:47:34 +00:00
Sean Zhong 6d39367f6a MAPREDUCE-6025: distribute nativetask.so to right folder (Manu Zhang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1618120 13f79535-47bb-0310-9956-ffa450edef68
2014-08-15 08:55:49 +00:00
Jason Darrell Lowe 7378af7bca MAPREDUCE-5878. some standard JDK APIs are not part of system classes defaults. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618080 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 23:17:33 +00:00
Allen Wittenauer 2d904b3fa3 MAPREDUCE-5906. Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent" (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618007 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 18:07:06 +00:00
Allen Wittenauer e1f15d2cdf MAPREDUCE-5999. Fix dead link in InputFormat javadoc (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618003 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 17:56:53 +00:00
Allen Wittenauer 7557f7bb2d MAPREDUCE-5998. CompositeInputFormat javadoc is broken (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617995 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 17:18:14 +00:00
Allen Wittenauer 6be98091f7 MAPREDUCE-5950. incorrect description in distcp2 document (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617994 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 17:11:39 +00:00
Allen Wittenauer 92054f2a06 MAPREDUCE-5597. Missing alternatives in javadocs for deprecated constructors in mapreduce.Job (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617986 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 16:00:18 +00:00
Jason Darrell Lowe 9579554988 MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617984 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:55:03 +00:00
Allen Wittenauer e7b540eced MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617982 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:49:10 +00:00
Allen Wittenauer 1e016e6451 MAPREDUCE-4791. Javadoc for KeyValueTextInputFormat should include default separator and how to change it (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617979 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:43:28 +00:00
Allen Wittenauer b3df973adb MAPREDUCE-883. harchive: Document how to unarchive (Akira AJISAKA and Koji Noguchi via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617977 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:39:52 +00:00
Todd Lipcon eace4dd50b MAPREDUCE-6035. native-task: sources/test-sources jar distribution. Contributed by Manu Zhang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1617879 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 04:56:43 +00:00
Todd Lipcon 808bf8bac1 MAPREDUCE-6026. native-task: fix logging. Contributed by Manu Zhang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1617878 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 04:53:36 +00:00
Todd Lipcon 886338ff80 MAPREDUCE-6006. native-task: add native tests to maven and fix bug in pom.xml. Contributed by Binglin Chang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1617877 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 04:47:20 +00:00
Allen Wittenauer e51a5d691b MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617806 13f79535-47bb-0310-9956-ffa450edef68
2014-08-13 19:34:26 +00:00
Allen Wittenauer cc93c1fd75 MAPREDUCE-5944. Remove MRv1 commands from CommandsManual.apt.vm (Akira AJISAKA via aw)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617804 13f79535-47bb-0310-9956-ffa450edef68
2014-08-13 19:18:50 +00:00
Charles Lamb 74f7be0887 merge from trunk r1617527
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1617532 13f79535-47bb-0310-9956-ffa450edef68
2014-08-12 17:02:07 +00:00
Xuan Gong eeb4acd955 YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
2014-08-08 21:38:24 +00:00
Charles Lamb 041b8326a1 MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace extended attributes. (clamb)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616657 13f79535-47bb-0310-9956-ffa450edef68
2014-08-08 01:32:18 +00:00
Andrew Wang 311d2f0773 Merge from trunk to branch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616589 13f79535-47bb-0310-9956-ffa450edef68
2014-08-07 20:26:52 +00:00
Jason Darrell Lowe de2595833c MAPREDUCE-6021. MR AM should have working directory in LD_LIBRARY_PATH. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616585 13f79535-47bb-0310-9956-ffa450edef68
2014-08-07 20:15:56 +00:00
Sean Zhong 4d4fb1723f MAPREDUCE-5978. native-task: remove test case for not supported codec Bzip2Codec and DefaultCodec (Manu Zhang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1616116 13f79535-47bb-0310-9956-ffa450edef68
2014-08-06 07:40:24 +00:00
Sean Zhong 432f641bc2 MAPREDUCE-5976. native-task: should not fail to build if snappy is missing (Manu Zhang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1616115 13f79535-47bb-0310-9956-ffa450edef68
2014-08-06 07:32:49 +00:00
Binglin Chang 7ecaa81d27 MAPREDUCE-5984. native-task: Reuse lz4 sources in hadoop-common. Contributed by Binglin Chang
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1616105 13f79535-47bb-0310-9956-ffa450edef68
2014-08-06 06:01:12 +00:00
Jason Darrell Lowe 857d134fda MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616018 13f79535-47bb-0310-9956-ffa450edef68
2014-08-05 21:52:22 +00:00
Andrew Wang ac73d416f3 Merge from trunk to branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
2014-08-05 02:30:54 +00:00
Binglin Chang 83a396733e MAPREDUCE-6005. native-task: Fix some valgrind errors (Binglin Chang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1615489 13f79535-47bb-0310-9956-ffa450edef68
2014-08-04 06:19:51 +00:00
Xuan Gong e52f67e389 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
2014-07-31 20:06:02 +00:00
Zhijie Shen 1d6e178144 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
2014-07-31 09:27:43 +00:00
Todd Lipcon 78d86a9839 MAPREDUCE-5995. native-task: Revert changes to Text internals. Contributed by Todd Lipcon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613828 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 19:04:14 +00:00
Todd Lipcon 43917e564d Add entry to branch's CHANGES.txt for previous commit
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613827 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 18:54:02 +00:00
Charles Lamb 1d3e9ec935 merge from trunk r1613787
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1613788 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 13:32:51 +00:00
Zhijie Shen 549bcc2c02 MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process is shutting down. Contributed by Wangda Tan.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613743 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 01:37:51 +00:00
Binglin Chang ea9e5b7baa MAPREDUCE-5991. native-task should not run unit tests if native profile is not enabled. Contributed by Binglin Chang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613072 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 11:44:09 +00:00
Todd Lipcon 6635180324 MAPREDUCE-6000. native-task: Simplify ByteBufferDataReader/Writer. Contributed by Todd Lipcon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613036 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 08:24:05 +00:00
Todd Lipcon 77acc70df5 MAPREDUCE-5997. native-task: Use DirectBufferPool from Hadoop Common. Contributed by Todd Lipcon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613034 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 08:20:25 +00:00
Todd Lipcon b2cba48f36 MAPREDUCE-5996. native-task: Rename system tests into standard directory layout. Contributed by Todd Lipcon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613004 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 06:17:33 +00:00
Todd Lipcon d468a92447 MAPREDUCE-5994. Simplify ByteUtils and fix failing test. Contributed by Todd Lipcon.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613003 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 06:14:27 +00:00
Andrew Wang 22ba00de60 Merge from trunk to branch pt 2
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612941 13f79535-47bb-0310-9956-ffa450edef68
2014-07-23 20:45:10 +00:00
Todd Lipcon 341695e731 Merge trunk into branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1612742 13f79535-47bb-0310-9956-ffa450edef68
2014-07-23 01:47:28 +00:00
Todd Lipcon e788e8b6ee MAPREDUCE-5985. native-task: Fix build on macosx. Contributed by Binglin Chang
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1612659 13f79535-47bb-0310-9956-ffa450edef68
2014-07-22 19:55:03 +00:00
Jason Darrell Lowe 60b1e835e0 MAPREDUCE-5963. ShuffleHandler DB schema should be versioned with compatible/incompatible changes. Contributed by Junping Du
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612652 13f79535-47bb-0310-9956-ffa450edef68
2014-07-22 19:22:24 +00:00
Andrew Wang cb20bb6117 Merge from trunk to branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612403 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 21:44:50 +00:00
Jason Darrell Lowe c2174a5536 MAPREDUCE-5756. CombineFileInputFormat.getSplits() including directories in its results. Contributed by Jason Dere
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612400 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 21:24:15 +00:00
Jason Darrell Lowe 9e62bcca4e MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612358 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 17:54:46 +00:00
Jian He 875592220f MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving RM-restart. Contributed by Rohith
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611434 13f79535-47bb-0310-9956-ffa450edef68
2014-07-17 18:46:20 +00:00
Todd Lipcon b2551c06a0 Import initial code for MAPREDUCE-2841 (native output collector)
Contributed by:
  Binglin Chang
  Yang Dong
  Sean Zhong
  Manu Zhang
  Zhongliang Zhu
  Vincent Wang
  Yan Dong
  Cheng Lian
  Xusen Yin
  Fangqin Dai
  Jiang Weihua
  Gansha Wu
  Avik Dey


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1611413 13f79535-47bb-0310-9956-ffa450edef68
2014-07-17 17:44:55 +00:00
Andrew Wang e8a1ff7efe MAPREDUCE-5971. Move the default options for distcp -p to DistCpOptionSwitch. Contributed by Charles Lamb.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611217 13f79535-47bb-0310-9956-ffa450edef68
2014-07-16 23:42:34 +00:00
Karthik Kambatla 43a12f3c01 MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex. (Gera Shegalov via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611196 13f79535-47bb-0310-9956-ffa450edef68
2014-07-16 21:24:51 +00:00
Vinayakumar B dc31d66f8a HADOOP-9921. daemon scripts should remove pid file on stop call after stop or process is found not running ( Contributed by Vinayakumar B)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610964 13f79535-47bb-0310-9956-ffa450edef68
2014-07-16 10:48:15 +00:00
Andrew Wang 56c0bd4d37 Merge from trunk to branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1610853 13f79535-47bb-0310-9956-ffa450edef68
2014-07-15 21:10:24 +00:00
Vinod Kumar Vavilapalli b60ef8b91b MAPREDUCE-5790. Made it easier to enable hprof profile options by default. Contributed by Gera Shegalov.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610578 13f79535-47bb-0310-9956-ffa450edef68
2014-07-15 04:49:01 +00:00
Zhijie Shen 64306aa1b5 MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current attempt is the last retry. Contributed by Wangda Tan.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609649 13f79535-47bb-0310-9956-ffa450edef68
2014-07-11 08:45:40 +00:00
Alejandro Abdelnur 95986dd2fb MAPREDUCE-5890. Support for encrypting Intermediate data and spills in local filesystem. (asuresh via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1609597 13f79535-47bb-0310-9956-ffa450edef68
2014-07-11 00:43:03 +00:00
Vinod Kumar Vavilapalli 4b2ded8202 MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side resource configuration for deciding uber-mode on map-only jobs. Contributed by Siqi Li.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608595 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 20:14:01 +00:00
Chris Nauroth 286ae1edd0 MAPREDUCE-5866. TestFixedLengthInputFormat fails in windows. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608585 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 19:56:44 +00:00
Vinod Kumar Vavilapalli d85f90e5b3 MAPREDUCE-5868. Fixed an issue with TestPipeApplication that was causing the nightly builds to fail. Contributed by Akira Ajisaka.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608579 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 19:45:57 +00:00
Karthik Kambatla 54dc17ce2f Preparing for release 2.6.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607538 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 05:55:05 +00:00
Zhijie Shen c6a09d2110 MAPREDUCE-5900. Changed to the interpret container preemption exit code as a task attempt killing event. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607512 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 01:43:56 +00:00
Jason Darrell Lowe cfd9344540 MAPREDUCE-5939. StartTime showing up as the epoch time in JHS UI after upgrade. Contributed by Chen He
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605892 13f79535-47bb-0310-9956-ffa450edef68
2014-06-26 19:56:09 +00:00
Vinod Kumar Vavilapalli e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
2014-06-24 21:43:22 +00:00
Vinod Kumar Vavilapalli 1355ff6132 MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for binary compatibility with older clients like Hive 0.13. Contributed by Akira Ajisaka.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604230 13f79535-47bb-0310-9956-ffa450edef68
2014-06-20 18:28:21 +00:00
Karthik Kambatla 7b9c074b76 MAPREDUCE-5844. Add a configurable delay to reducer-preemption. (Maysam Yabandeh via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603957 13f79535-47bb-0310-9956-ffa450edef68
2014-06-19 17:22:56 +00:00
Karthik Kambatla bd23a2ff22 MAPREDUCE-5896. InputSplits should indicate which locations have the block cached in memory. (Sandy Ryza via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603670 13f79535-47bb-0310-9956-ffa450edef68
2014-06-18 23:28:50 +00:00
Jian He 0abddac476 MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event at COMMIT_PENDING state. Contributed by Zhijie Shen
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603338 13f79535-47bb-0310-9956-ffa450edef68
2014-06-18 00:51:39 +00:00
Arun Murthy 26936978be Preparing to release hadoop-2.4.1.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602886 13f79535-47bb-0310-9956-ffa450edef68
2014-06-16 15:07:00 +00:00
Uma Maheswara Rao G 2ad014a097 Moved CHANGES.txt entries of MAPREDUCE-5898, MAPREDUCE-5920, HDFS-6464, HDFS-6375 from trunk to 2.5 section on merging HDFS-2006 to branch-2
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602699 13f79535-47bb-0310-9956-ffa450edef68
2014-06-15 14:54:36 +00:00
Chris Nauroth 180ac8dc1d MAPREDUCE-5912. Task.calculateOutputSize does not handle Windows files after MAPREDUCE-5196. Contributed by Remus Rusanu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602282 13f79535-47bb-0310-9956-ffa450edef68
2014-06-12 20:02:45 +00:00
Karthik Kambatla fc7c8f9bf2 HADOOP-10686. Writables are not always configured. (Abraham Elmahrek via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602079 13f79535-47bb-0310-9956-ffa450edef68
2014-06-12 07:23:20 +00:00
Chris Nauroth 473e83ab71 MAPREDUCE-5920. Add Xattr option in DistCp docs. Contributed by Yi Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601983 13f79535-47bb-0310-9956-ffa450edef68
2014-06-11 18:36:06 +00:00
Vinod Kumar Vavilapalli 6480fdfd33 MAPREDUCE-5834. Increased test-timeouts in TestGridMixClasses to avoid occassional failures. Contributed by Mit Desai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601764 13f79535-47bb-0310-9956-ffa450edef68
2014-06-10 20:24:57 +00:00
Chris Nauroth 4888b16a43 MAPREDUCE-5886. Allow wordcount example job to accept multiple input paths. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601704 13f79535-47bb-0310-9956-ffa450edef68
2014-06-10 17:48:52 +00:00
Karthik Kambatla 40ba8c17c1 MAPREDUCE-5777. Support utf-8 text with Byte Order Marker. (Zhihai Xu via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600977 13f79535-47bb-0310-9956-ffa450edef68
2014-06-06 18:37:03 +00:00
Uma Maheswara Rao G f81c7b0252 MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs). Contributed by Yi Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600900 13f79535-47bb-0310-9956-ffa450edef68
2014-06-06 14:45:39 +00:00
Chris Nauroth 806f4d11f2 HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68
2014-05-29 17:26:50 +00:00
Devarajulu K 61c59c0704 MAPREDUCE-5895. Close streams properly to avoid leakage in TaskLog. Contributed by Kousuke Saruta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598209 13f79535-47bb-0310-9956-ffa450edef68
2014-05-29 07:05:26 +00:00
Jason Darrell Lowe 4bb4de93d6 MAPREDUCE-5862. Line records longer than 2x split size aren't handled correctly. Contributed by bc Wong
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598111 13f79535-47bb-0310-9956-ffa450edef68
2014-05-28 19:37:52 +00:00
Jing Zhao 3671a5e16f MAPREDUCE-5899. Support incremental data copy in DistCp. Contributed by Jing Zhao.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596931 13f79535-47bb-0310-9956-ffa450edef68
2014-05-22 18:17:11 +00:00
Jason Darrell Lowe 1f8bb1001a MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job history files generated by 2.0.3 history server. Contributed by Rushabh S Shah
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596295 13f79535-47bb-0310-9956-ffa450edef68
2014-05-20 15:46:54 +00:00
Andrew Wang 4a839f0cc1 Fix CRLFs in TestKillAMPreemptionPolicy.java
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596058 13f79535-47bb-0310-9956-ffa450edef68
2014-05-19 21:18:55 +00:00
Devarajulu K f74e446355 MAPREDUCE-5867. Fix NPE in KillAMPreemptionPolicy related to ProportionalCapacityPreemptionPolicy. Contributed by Sunil G.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595754 13f79535-47bb-0310-9956-ffa450edef68
2014-05-19 06:42:29 +00:00
Chris Nauroth 11be7334c4 MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595283 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 18:25:23 +00:00
Ravi Prakash c9377189a0 MAPREDUCE 5874. Creating MapReduce REST API section. Contributed by Tsuyoshi Ozawa
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595265 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 16:50:58 +00:00
Junping Du 1e5d2c612b MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not need to be volatile. (Contributed by Tsuyoshi OZAWA)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594795 13f79535-47bb-0310-9956-ffa450edef68
2014-05-15 07:07:18 +00:00
Christopher Douglas 5bf7ef839b MAPREDUCE-5821. Avoid unintentional reallocation of byte arrays in segments
during merge. Contributed by Todd Lipcon


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594654 13f79535-47bb-0310-9956-ffa450edef68
2014-05-14 17:51:11 +00:00
Karthik Kambatla f3c3d9e0c6 MAPREDUCE-5652. NM Recovery. ShuffleHandler should handle NM restarts. (Jason Lowe via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594329 13f79535-47bb-0310-9956-ffa450edef68
2014-05-13 19:10:48 +00:00
Jason Darrell Lowe b0e80d1a47 MAPREDUCE-5814. fat jar with *-default.xml may fail when mapreduce.job.classloader=true. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594326 13f79535-47bb-0310-9956-ffa450edef68
2014-05-13 18:55:05 +00:00
Jonathan Turner Eagles 4bc3371824 MAPREDUCE-5888. Failed job leaves hung AM after it unregisters (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594317 13f79535-47bb-0310-9956-ffa450edef68
2014-05-13 18:27:26 +00:00
Karthik Kambatla 5e337149e3 MAPREDUCE-5774. Job overview in History UI should list reducer phases in chronological order. (Gera Shegalov via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593890 13f79535-47bb-0310-9956-ffa450edef68
2014-05-12 06:15:33 +00:00
Jason Darrell Lowe e2c18809d3 MAPREDUCE-5884. History server uses short user name when canceling tokens. Contributed by Mohammad Kamrul Islam
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593422 13f79535-47bb-0310-9956-ffa450edef68
2014-05-08 21:22:26 +00:00
Jonathan Turner Eagles 7fe28fc8f9 MAPREDUCE-5636. Convert MapReduce Tutorial document to APT (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592795 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 16:17:16 +00:00
Jonathan Turner Eagles bd54137afa MAPREDUCE-5637. Convert Hadoop Streaming document to APT (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592789 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 16:01:21 +00:00
Tsz-wo Sze 03db13206f MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable. Contributed by Tsuyoshi OZAWA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592703 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 10:24:10 +00:00
Haohui Mai d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
2014-05-02 22:26:02 +00:00
Jonathan Turner Eagles 693025a3d4 MAPREDUCE-5638. Port Hadoop Archives document to trunk (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1591107 13f79535-47bb-0310-9956-ffa450edef68
2014-04-29 21:23:50 +00:00
Jason Darrell Lowe bb7ce82816 MAPREDUCE-5812. Make job context available to OutputCommitter.isRecoverySupported(). Contributed by Mohammad Kamrul Islam
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590668 13f79535-47bb-0310-9956-ffa450edef68
2014-04-28 15:18:02 +00:00
Eric Yang 39abe66822 MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob.
(Jinghui Wang via Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590263 13f79535-47bb-0310-9956-ffa450edef68
2014-04-26 17:59:14 +00:00
Jason Darrell Lowe 283a3a246a MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state. Contributed by Ming Ma
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590168 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 22:26:24 +00:00
Jonathan Turner Eagles a059eadbe9 MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590058 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 15:23:42 +00:00
Jason Darrell Lowe 5492149c3c MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590046 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 14:53:17 +00:00
Uma Maheswara Rao G 0931bd94be HDFS-6246. Remove 'dfs.support.append' flag from trunk code. Contributed by Uma Maheswara Rao G.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589927 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 05:19:50 +00:00
Haohui Mai c7e87574cb MAPREDUCE-5837. MRAppMaster fails when checking on uber mode. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589903 13f79535-47bb-0310-9956-ffa450edef68
2014-04-24 22:53:46 +00:00
Ravi Prakash a93be7b46c MAPREDUCE-5846. Rumen doesn't understand JobQueueChangedEvent (Nathan Roberts via raviprak)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589595 13f79535-47bb-0310-9956-ffa450edef68
2014-04-24 05:59:28 +00:00
Vinod Kumar Vavilapalli e33ba5d944 MAPREDUCE-5843. Fixed TestMRKeyValueTextInputFormat to not leak files and thus avoid failing on Windows. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589533 13f79535-47bb-0310-9956-ffa450edef68
2014-04-23 22:54:26 +00:00
Jason Darrell Lowe 6957745c2c MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589524 13f79535-47bb-0310-9956-ffa450edef68
2014-04-23 21:53:06 +00:00
Vinod Kumar Vavilapalli 586c46acda MAPREDUCE-5832. Fixed TestJobClient to not fail on JDK7 or on Windows. Contributed by Jian He and Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589315 13f79535-47bb-0310-9956-ffa450edef68
2014-04-23 00:02:09 +00:00
Chris Nauroth 4a91b876db MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589248 13f79535-47bb-0310-9956-ffa450edef68
2014-04-22 18:44:54 +00:00
Chris Nauroth bb4216abf8 MAPREDUCE-5827. Move attribution from 2.5.0 to 2.4.1 in CHANGES.txt.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589238 13f79535-47bb-0310-9956-ffa450edef68
2014-04-22 18:23:27 +00:00
Chris Nauroth 1dfc9c39b6 MAPREDUCE-5827. TestSpeculativeExecutionWithMRApp fails. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589223 13f79535-47bb-0310-9956-ffa450edef68
2014-04-22 18:17:54 +00:00
Chris Nauroth 06c15b6a3e MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589006 13f79535-47bb-0310-9956-ffa450edef68
2014-04-21 23:44:41 +00:00
Chris Nauroth c482cb253c MAPREDUCE-5642. TestMiniMRChildTask fails on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588605 13f79535-47bb-0310-9956-ffa450edef68
2014-04-19 04:07:22 +00:00
Jason Darrell Lowe 8d569c2220 MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly. Contributed by Eric Payne
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588559 13f79535-47bb-0310-9956-ffa450edef68
2014-04-18 20:33:28 +00:00
Chris Nauroth dd7d032457 HADOOP-10499. Remove unused parameter from ProxyUsers.authorize(). Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588098 13f79535-47bb-0310-9956-ffa450edef68
2014-04-16 23:09:08 +00:00
Jonathan Turner Eagles 28d62bf75b MAPREDUCE-3191. docs for map output compression incorrectly reference SequenceFile (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1588009 13f79535-47bb-0310-9956-ffa450edef68
2014-04-16 18:14:53 +00:00
Devarajulu K 14fee881fd MAPREDUCE-5775. Remove unnecessary job.setNumReduceTasks in SleepJob.createJob. Contributed by jhanver chand sharma.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587831 13f79535-47bb-0310-9956-ffa450edef68
2014-04-16 08:32:44 +00:00
Jonathan Turner Eagles 161d97807f MAPREDUCE-5836. Fix typo in RandomTextWriter (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587716 13f79535-47bb-0310-9956-ffa450edef68
2014-04-15 20:33:28 +00:00
Vinod Kumar Vavilapalli 6351ee2a78 MAPREDUCE-5828. Fixed a test issue with TestMapReduceJobControl that was causing it to fail on Windows. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1587089 13f79535-47bb-0310-9956-ffa450edef68
2014-04-13 19:55:32 +00:00
Vinod Kumar Vavilapalli c1127d19c4 MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions on Windows for temporary files and thus also fix the test-issue with
TestHistoryServerFileSystemStateStoreService. Contributed by Varun Vasudev.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586567 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 04:28:57 +00:00
Vinod Kumar Vavilapalli 8d41b363b8 MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster gracefully handle empty-queue names. Contributed by Akira Ajisaka.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586559 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 04:01:39 +00:00
Jason Darrell Lowe 26421dd7dc MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586479 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 22:02:48 +00:00
Vinod Kumar Vavilapalli 05fd3bbe48 MAPREDUCE-5824. Fixed test-failure of TestPipesNonJavaInputFormat in Windows. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586368 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 16:44:04 +00:00
Steve Loughran b8f250a99d HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585932 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 11:11:20 +00:00
Haohui Mai 8ca32df08e HADOOP-10474. Move o.a.h.record to hadoop-streaming. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585886 13f79535-47bb-0310-9956-ffa450edef68
2014-04-09 04:02:46 +00:00
Zhijie Shen f9df4d7377 MAPREDUCE-5818. Added "hsadmin" command into mapred.cmd. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584378 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 20:23:51 +00:00
Aaron Myers 74e3a176c4 Move MAPREDUCE-5014 to the right section now that it's been merged to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584234 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 00:57:00 +00:00
Haohui Mai fb1d7fb596 HDFS-5570. Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584100 13f79535-47bb-0310-9956-ffa450edef68
2014-04-02 17:28:08 +00:00
Sanford Ryza d28be56239 MAPREDUCE-5759. Remove unnecessary conf load in Limits (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583497 13f79535-47bb-0310-9956-ffa450edef68
2014-04-01 00:16:45 +00:00
Arun Murthy fa9eed2ed6 Preparing for hadoop-2.4.0 release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583245 13f79535-47bb-0310-9956-ffa450edef68
2014-03-31 07:36:13 +00:00
Arun Murthy b7b3a7e011 MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the local-filesystem, rather than rely on the classpath. Contributed by Gera Shegalov.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1583050 13f79535-47bb-0310-9956-ffa450edef68
2014-03-29 19:14:53 +00:00
Vinod Kumar Vavilapalli 86ef362a7d MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. Contributed by Akira Ajisaka.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582896 13f79535-47bb-0310-9956-ffa450edef68
2014-03-28 22:16:55 +00:00
Vinod Kumar Vavilapalli 533675bfbb MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582125 13f79535-47bb-0310-9956-ffa450edef68
2014-03-26 23:43:49 +00:00
Vinod Kumar Vavilapalli 6015e95941 MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it recovers from a commit during a previous attempt. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581180 13f79535-47bb-0310-9956-ffa450edef68
2014-03-25 02:00:39 +00:00
Chris Nauroth a90a5c2452 MAPREDUCE-5791. Shuffle phase is slow in Windows - FadviseFileRegion::transferTo does not read disks efficiently. Contributed by Nikola Vujic.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580994 13f79535-47bb-0310-9956-ffa450edef68
2014-03-24 18:57:52 +00:00
Vinod Kumar Vavilapalli 8a06949df8 MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. Contributed by Varun Vasudev.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580100 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 00:32:57 +00:00
Vinod Kumar Vavilapalli a5c08eed16 MAPREDUCE-5787. Added the ability to keep alive shuffle connections in the MapReduce shuffle-handler. Contributed by Rajesh Balamohan.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1580062 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 21:43:55 +00:00
Kihwal Lee 767a98aa92 MAPREDUCE-5804. TestMRJobsWithProfiler#testProfiler timesout. Contributed by Mit Desai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579995 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 17:57:09 +00:00
Vinod Kumar Vavilapalli 396c6c63a2 MAPREDUCE-2349. Modified FileInputFormat to be able to issue file and block location calls in parallel. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1579515 13f79535-47bb-0310-9956-ffa450edef68
2014-03-20 02:46:23 +00:00
Jing Zhao 0b3ee72af8 HDFS-6090. Use MiniDFSCluster.Builder instead of deprecated constructors. Contributed by Akira AJISAKA.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578669 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 23:53:48 +00:00
Vinod Kumar Vavilapalli 96e0ca2d27 YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He.
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578135 13f79535-47bb-0310-9956-ffa450edef68
2014-03-16 18:32:05 +00:00
Jason Darrell Lowe 8a2a741595 MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577692 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 20:33:03 +00:00
Jason Darrell Lowe d9f723ed74 MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577647 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 18:15:50 +00:00
Jason Darrell Lowe 2a6e8610b1 Update CHANGES.txt to show MR-5688 pulled into 2.4.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577586 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 15:55:36 +00:00
Jason Darrell Lowe 0ab1f90d40 MAPREDUCE-5751. MR app master fails to start in some cases if mapreduce.job.classloader is true. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577554 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 14:43:33 +00:00
Christopher Douglas 53790d3300 YARN-1771. Reduce the number of NameNode operations during localization of
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577391 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 00:30:35 +00:00
Jonathan Turner Eagles aab466558a MAPREDUCE-5456. TestFetcher.testCopyFromHostExtraBytes is missing (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577313 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 20:48:06 +00:00
Arpit Agarwal 763b8ebf46 MAPREDUCE-5794. SliveMapper always uses default FileSystem. (Contributed by szetszwo)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577281 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 18:55:29 +00:00
Jonathan Turner Eagles 2c6ce77c5a MAPREDUCE-5713. InputFormat and JobConf JavaDoc Fixes (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577239 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 16:49:27 +00:00
Jason Darrell Lowe 87866d4a56 MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page. Contributed by Rushabh S Shah
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577202 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:35:34 +00:00
Jonathan Turner Eagles a8f302a1e6 MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577188 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:13:07 +00:00
Zhijie Shen 6af044bb75 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577052 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 07:42:29 +00:00
Arun Murthy 3bbd67173e MAPREDUCE-5553. Allow users to easily access completed/pending/successful/failed tasks on MR AM web-ui. Contributed by Paul Han.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576946 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 22:04:48 +00:00
Jason Darrell Lowe cbf1c72d1b MAPREDUCE-5778. JobSummary does not escape newlines in the job name. Contributed by Akira AJISAKA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576940 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 21:57:34 +00:00
Vinod Kumar Vavilapalli 98ecd4ffef MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail when a large value of io.sort.mb is set. Contributed by Karthik Kambatla.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1576170 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 01:29:56 +00:00
Tsz-wo Sze 8d49acf0e9 MAPREDUCE-5780. SliveTest should use the specified path to get the particular FileSystem instead of using the default FileSystem.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1575049 13f79535-47bb-0310-9956-ffa450edef68
2014-03-06 21:26:51 +00:00
Haohui Mai 9192f8446d HDFS-6058. Fix TestHDFSCLI failures after HADOOP-8691 change. Contributed by Akira Ajisaka and Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574792 13f79535-47bb-0310-9956-ffa450edef68
2014-03-06 08:13:40 +00:00
Vinod Kumar Vavilapalli 88245b6a41 YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1574190 13f79535-47bb-0310-9956-ffa450edef68
2014-03-04 20:39:06 +00:00
Jason Darrell Lowe a756de93ef MAPREDUCE-5773. Provide dedicated MRAppMaster syslog length limit. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573775 13f79535-47bb-0310-9956-ffa450edef68
2014-03-03 22:02:18 +00:00
Jason Darrell Lowe 77910d4fe3 MAPREDUCE-5768. TestMRJobs.testContainerRollingLog fails on trunk. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1573035 13f79535-47bb-0310-9956-ffa450edef68
2014-02-28 19:22:01 +00:00
Karthik Kambatla 9bdd272fd7 MAPREDUCE-5665. Add audience annotations to MiniMRYarnCluster and MiniMRCluster. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572795 13f79535-47bb-0310-9956-ffa450edef68
2014-02-28 00:52:30 +00:00
Vinod Kumar Vavilapalli 94b29b3348 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572711 13f79535-47bb-0310-9956-ffa450edef68
2014-02-27 19:23:55 +00:00
Vinod Kumar Vavilapalli cd67d5abcd MAPREDUCE-5766. Moved ping messages from TaskAttempts to be at DEBUG level inside the ApplicationMaster log. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572380 13f79535-47bb-0310-9956-ffa450edef68
2014-02-27 01:29:28 +00:00
Jason Darrell Lowe 658b1bf561 MAPREDUCE-5754. Preserve Job diagnostics in history. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1572269 13f79535-47bb-0310-9956-ffa450edef68
2014-02-26 21:32:34 +00:00
Vinod Kumar Vavilapalli ae29d9ee04 MAPREDUCE-5761. Added a simple log message to denote when encrypted shuffle is on in the shuffle-handler. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1571514 13f79535-47bb-0310-9956-ffa450edef68
2014-02-25 01:24:26 +00:00
Jonathan Turner Eagles 19f870858e MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570631 13f79535-47bb-0310-9956-ffa450edef68
2014-02-21 16:43:11 +00:00
Jonathan Turner Eagles aa4cbcfa79 MAPREDUCE-5671. NaN can be created by client and assign to Progress (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1570620 13f79535-47bb-0310-9956-ffa450edef68
2014-02-21 16:23:12 +00:00
Vinod Kumar Vavilapalli 6b03174aae YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569537 13f79535-47bb-0310-9956-ffa450edef68
2014-02-18 21:50:37 +00:00
Arun Murthy 1417803326 Branching for hadoop-2.5 releases.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569185 13f79535-47bb-0310-9956-ffa450edef68
2014-02-18 04:38:51 +00:00
Jason Darrell Lowe 08e6d312e8 MAPREDUCE-5757. ConcurrentModificationException in JobControl.toList. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1569038 13f79535-47bb-0310-9956-ffa450edef68
2014-02-17 16:14:58 +00:00
Vinod Kumar Vavilapalli 990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
2014-02-14 20:01:02 +00:00
Jason Darrell Lowe c122ef9000 MAPREDUCE-5670. CombineFileRecordReader should report progress when moving to the next file. Contributed by Chen He
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568118 13f79535-47bb-0310-9956-ffa450edef68
2014-02-13 23:16:00 +00:00
Karthik Kambatla 308f5faf18 MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job. (Jason Lowe via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567666 13f79535-47bb-0310-9956-ffa450edef68
2014-02-12 16:04:18 +00:00
Arun Murthy c43c9dd7b5 Preparing to release hadoop-2.3.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1567111 13f79535-47bb-0310-9956-ffa450edef68
2014-02-11 13:25:30 +00:00
Sanford Ryza c79cc3a314 YARN-1497. Command line additions for moving apps between queues (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565754 13f79535-47bb-0310-9956-ffa450edef68
2014-02-07 18:35:44 +00:00
Karthik Kambatla 914a9709b9 MAPREDUCE-5744. Job hangs because RMContainerAllocator.preemptReduce() violates the comparator contract (Gera Shegalov via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565478 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 23:27:57 +00:00
Vinod Kumar Vavilapalli b527a975a4 MAPREDUCE-5743. Fixed CHANGES.txt to give credit to Ted too.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565401 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 19:07:18 +00:00
Karthik Kambatla 4594b74b85 MAPREDUCE-5699. Allow setting tags on MR jobs (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565384 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 18:29:30 +00:00
Zhijie Shen b36cc29073 MAPREDUCE-5743. Fixed the test failure in TestRMContainerAllocator. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1565087 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 06:20:41 +00:00
Vinod Kumar Vavilapalli 10f0901764 Updating branch-2.3's YARN's CHANGES.txt to reflect the recent rebase of unreleased 2.3 to more latest branch-2.
Fixing entry for MAPREDUCE-5732 in trunk and branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562949 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 20:30:12 +00:00
Vinod Kumar Vavilapalli d0b4c7d6b1 Updating branch-2's MAPREDUCE CHANGES.txt to reflect the recent rebase of unreleased 2.3 to more latest branch-2.
Updating trunk's MAPREDUCE CHANGES.txt to fix more differences with branch-2's CHANGES.txt
MAPREDUCE's CHANGES.txt is foobar'ed. There are still a bunch of misplaced CHANGES.txt entries - not fixing them now.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562945 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 20:16:56 +00:00
Vinod Kumar Vavilapalli 95074f0867 Updating trunk's MAPREDUCE CHANGES.txt to reflect the recent rebase of unreleased 2.3 to more latest branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562925 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 19:01:47 +00:00
Sanford Ryza f51c7ebe75 Add missing file for MAPREDUCE-5732
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562642 13f79535-47bb-0310-9956-ffa450edef68
2014-01-29 23:53:13 +00:00
Sanford Ryza db80705719 MAPREDUCE-5732. Report proper queue when job has been automatically placed (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562641 13f79535-47bb-0310-9956-ffa450edef68
2014-01-29 23:51:40 +00:00
Sanford Ryza 2a20fe8370 MAPREDUCE-5464. Add analogs of the SLOTS_MILLIS counters that jive with the YARN resource model (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1562216 13f79535-47bb-0310-9956-ffa450edef68
2014-01-28 20:15:18 +00:00
Suresh Srinivas 33a47743a5 HADOOP-10255. Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561959 13f79535-47bb-0310-9956-ffa450edef68
2014-01-28 07:32:52 +00:00
Vinod Kumar Vavilapalli cbee889711 YARN-321. Merging YARN-321 branch to trunk.
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
2014-01-26 04:51:10 +00:00
Konstantin Boudnik 4c197b5d56 HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring. Contributed by Mikhail Antonov.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560831 13f79535-47bb-0310-9956-ffa450edef68
2014-01-23 22:29:23 +00:00
Karthik Kambatla 122be11c14 MAPREDUCE-5723. MR AM container log can be truncated or empty. (Mohammad Kamrul Islam via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560592 13f79535-47bb-0310-9956-ffa450edef68
2014-01-23 06:06:45 +00:00
Jason Darrell Lowe d1963ad4bd MAPREDUCE-5693. Restore MRv1 behavior for log flush. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560148 13f79535-47bb-0310-9956-ffa450edef68
2014-01-21 19:23:34 +00:00
Sanford Ryza 362e4fc891 MAPREDUCE-5725. Make explicit that TestNetworkedJob relies on the Capacity Scheduler (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559882 13f79535-47bb-0310-9956-ffa450edef68
2014-01-21 01:23:44 +00:00
Karthik Kambatla c5a241f1dd MAPREDUCE-5729. mapred job -list throws NPE (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559811 13f79535-47bb-0310-9956-ffa450edef68
2014-01-20 19:29:41 +00:00
Jason Darrell Lowe 0fd646b967 MAPREDUCE-5717. Task pings are interpreted as task progress. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559256 13f79535-47bb-0310-9956-ffa450edef68
2014-01-17 21:38:37 +00:00
Sanford Ryza f667371746 MAPREDUCE-5650. Job fails when hprof mapreduce.task.profile.map/reduce.params is specified (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559201 13f79535-47bb-0310-9956-ffa450edef68
2014-01-17 17:43:00 +00:00
Sanford Ryza 67cbde3008 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1559050 13f79535-47bb-0310-9956-ffa450edef68
2014-01-17 08:56:22 +00:00
Jason Darrell Lowe ccee5b3e44 Moving MR-5672 from bugs to improvements in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558959 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 23:16:11 +00:00
Jason Darrell Lowe a6ea460a91 MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558948 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 22:56:09 +00:00
Alejandro Abdelnur 3a6f8b8785 MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558852 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 17:10:20 +00:00
Konstantin Boudnik 0b6a236258 Adding 2.0.6-alpha release to the CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1558215 13f79535-47bb-0310-9956-ffa450edef68
2014-01-14 21:49:59 +00:00
Chris Nauroth 8b2c8a657e YARN-1138. yarn.application.classpath is set to point to etc., which does not work on Windows. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556978 13f79535-47bb-0310-9956-ffa450edef68
2014-01-09 23:04:18 +00:00
Alejandro Abdelnur 0c591a624d MAPREDUCE-3310. Addendum fixing javadocs warnings (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1556063 13f79535-47bb-0310-9956-ffa450edef68
2014-01-06 23:21:08 +00:00
Alejandro Abdelnur 76238b9722 MAPREDUCE-3310. Custom grouping comparator cannot be set for Combiners (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555968 13f79535-47bb-0310-9956-ffa450edef68
2014-01-06 18:35:26 +00:00
Karthik Kambatla 1a8781f1f9 MAPREDUCE-5689. MRAppMaster does not preempt reducers when scheduled maps cannot be fulfilled. (lohit via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555161 13f79535-47bb-0310-9956-ffa450edef68
2014-01-03 17:27:36 +00:00
Vinod Kumar Vavilapalli b524501d4f MAPREDUCE-5685. Fixed a bug with JobContext getCacheFiles API inside the WrappedReducer class. Contributed by Yi Song.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1554320 13f79535-47bb-0310-9956-ffa450edef68
2013-12-30 21:14:46 +00:00
Christopher Douglas 47cca0cb6d MAPREDUCE-5196. Add bookkeeping for managing checkpoints of task state.
Contributed by Carlo Curino


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553939 13f79535-47bb-0310-9956-ffa450edef68
2013-12-28 21:58:33 +00:00
Vinod Kumar Vavilapalli cc4c74be09 MAPREDUCE-5694. Fixed MR AppMaster to shutdown the LogManager so as to avoid losing syslog in some conditions. Contributed by Mohammad Kamrul Islam.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553879 13f79535-47bb-0310-9956-ffa450edef68
2013-12-28 17:51:03 +00:00
Karthik Kambatla 1152e23ed0 YARN-1463. Tests should avoid starting http-server where possible or creates spnego keytab/principals (vinodkv via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1553790 13f79535-47bb-0310-9956-ffa450edef68
2013-12-28 06:55:30 +00:00
Sanford Ryza 04dac63695 MAPREDUCE-5550. Task Status message (reporter.setStatus) not shown in UI with Hadoop 2.0 (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552808 13f79535-47bb-0310-9956-ffa450edef68
2013-12-20 21:00:25 +00:00
Sanford Ryza 1b9b956924 MAPREDUCE-5692. Add explicit diagnostics when a task attempt is killed due to speculative execution (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552797 13f79535-47bb-0310-9956-ffa450edef68
2013-12-20 19:53:59 +00:00
Vinod Kumar Vavilapalli 9184c4d179 MAPREDUCE-5687. Correcting the previous commit by ushing the right patch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1552069 13f79535-47bb-0310-9956-ffa450edef68
2013-12-18 19:27:43 +00:00
Vinod Kumar Vavilapalli d52bfdef95 MAPREDUCE-5687. Fixed failure in TestYARNRunner caused by YARN-1446. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551774 13f79535-47bb-0310-9956-ffa450edef68
2013-12-18 00:13:58 +00:00
Christopher Douglas 9ca394d54d MAPREDUCE-5189. Add policies and wiring to respond to preemption requests
from YARN. Contributed by Carlo Curino.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551748 13f79535-47bb-0310-9956-ffa450edef68
2013-12-17 22:54:31 +00:00
Christopher Douglas 9218b4aec1 MAPREDUCE-5197. Add a service for checkpointing task state.
Contributed by Carlo Curino


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551726 13f79535-47bb-0310-9956-ffa450edef68
2013-12-17 21:37:25 +00:00
Jason Darrell Lowe 44429c4bb3 MAPREDUCE-5679. TestJobHistoryParsing has race condition. Contributed by Liyin Liang
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551616 13f79535-47bb-0310-9956-ffa450edef68
2013-12-17 16:45:23 +00:00
Jason Darrell Lowe fc77ed153e MAPREDUCE-5623. TestJobCleanup fails because of RejectedExecutionException and NPE. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551285 13f79535-47bb-0310-9956-ffa450edef68
2013-12-16 17:44:49 +00:00
Chris Nauroth ee0530ee0e MAPREDUCE-5674. Missing start and finish time in mapred.JobStatus. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1550472 13f79535-47bb-0310-9956-ffa450edef68
2013-12-12 18:30:12 +00:00
Jonathan Turner Eagles ee5351bf22 MAPREDUCE-5052. Job History UI and web services confusing job start time and job submit time (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549896 13f79535-47bb-0310-9956-ffa450edef68
2013-12-10 17:17:47 +00:00
Jason Darrell Lowe 18d99c12c3 MAPREDUCE-5656. bzip2 codec can drop records when reading data in splits. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1549705 13f79535-47bb-0310-9956-ffa450edef68
2013-12-09 23:31:30 +00:00
Jason Darrell Lowe 0f2ce45aeb MAPREDUCE-5409. MRAppMaster throws InvalidStateTransitonException: Invalid event: TA_TOO_MANY_FETCH_FAILURE at KILLED for TaskAttemptImpl. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1548197 13f79535-47bb-0310-9956-ffa450edef68
2013-12-05 16:30:12 +00:00
Jonathan Turner Eagles 9ba98e6af1 MAPREDUCE-5632. TestRMContainerAllocator#testUpdatedNodes fails (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547929 13f79535-47bb-0310-9956-ffa450edef68
2013-12-04 22:11:28 +00:00
Jonathan Turner Eagles 5afc1242ea MAPREDUCE-5645. TestFixedLengthInputFormat fails with native libs (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547624 13f79535-47bb-0310-9956-ffa450edef68
2013-12-03 22:44:07 +00:00
Arun Murthy eec0ec2d3f Add ACL check to JHS web-services.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547159 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 19:32:17 +00:00
Jonathan Turner Eagles 74851537de MAPREDUCE-5640. Rename TestLineRecordReader in jobclient module (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547149 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 19:04:51 +00:00
Arun Murthy a4e473c6a9 Renamed releases. 2.2.1 -> 2.3.0 & 2.3.0 -> 2.4.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1547136 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 18:37:39 +00:00
Jing Zhao 2214871d91 HDFS-5545. Allow specifying endpoints for listeners in HttpServer. Contributed by Haohui Mai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1546151 13f79535-47bb-0310-9956-ffa450edef68
2013-11-27 18:20:14 +00:00
Jason Darrell Lowe 0a5b0499f3 MAPREDUCE-5631. TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7. Contributed by Jonathan Eagles
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1544266 13f79535-47bb-0310-9956-ffa450edef68
2013-11-21 17:53:39 +00:00
Colin McCabe efea68dc35 HDFS-5511. improve CacheManipulator interface to allow better unit testing (cmccabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543676 13f79535-47bb-0310-9956-ffa450edef68
2013-11-20 00:48:05 +00:00
Jonathan Turner Eagles fdad179980 MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542733 13f79535-47bb-0310-9956-ffa450edef68
2013-11-17 14:29:33 +00:00
Jonathan Turner Eagles a02e60fefc MAPREDUCE-5373. TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542456 13f79535-47bb-0310-9956-ffa450edef68
2013-11-16 03:29:51 +00:00
Chris Nauroth 88c4d4a714 MAPREDUCE-5616. MR Client-AppMaster RPC max retries on socket timeout is too high. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1542001 13f79535-47bb-0310-9956-ffa450edef68
2013-11-14 18:09:14 +00:00
Sanford Ryza 3d95049f79 MAPREDUCE-5481. Enable uber jobs to have multiple reducers (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541844 13f79535-47bb-0310-9956-ffa450edef68
2013-11-14 07:56:11 +00:00
Steve Loughran 01ccd4d3ab MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541702 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 20:46:54 +00:00
Steve Loughran 4a0f804d5e MAPREDUCE-5431 Missing pom dependency in MR-client
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541682 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 19:52:43 +00:00
Jason Darrell Lowe 744506f34d MAPREDUCE-5610. TestSleepJob fails in jdk7. Contributed by Jonathan Eagles
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541566 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 15:49:29 +00:00
Sanford Ryza e94ed91114 MAPREDUCE-5613. DefaultSpeculator holds and checks hashmap that is alway empty (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1541433 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 09:23:18 +00:00
Sanford Ryza eeda370249 MAPREDUCE-1176. FixedLengthInputFormat and FixedLengthRecordReader (Mariappan Asokan and BitsOfInfo via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540931 13f79535-47bb-0310-9956-ffa450edef68
2013-11-12 03:06:10 +00:00
Jason Darrell Lowe 38a3b925e9 MAPREDUCE-5186. mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail. Contributed by Robert Parker and Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1540813 13f79535-47bb-0310-9956-ffa450edef68
2013-11-11 19:22:38 +00:00
Chris Nauroth bee6822441 MAPREDUCE-5451. MR uses LD_LIBRARY_PATH which doesn't mean anything in Windows. Contributed by Yingda Chen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1539413 13f79535-47bb-0310-9956-ffa450edef68
2013-11-06 18:07:39 +00:00
Colin McCabe 7abf7e8151 HADOOP-9478. Fix race conditions during the initialization of Configuration related to deprecatedKeyMap (cmccabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538248 13f79535-47bb-0310-9956-ffa450edef68
2013-11-02 18:42:27 +00:00
Chris Nauroth 1606a76a54 MAPREDUCE-5604. TestMRAMWithNonNormalizedCapabilities fails on Windows due to exceeding max path length. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1538035 13f79535-47bb-0310-9956-ffa450edef68
2013-11-01 19:08:43 +00:00
Sanford Ryza 86dc50c1eb MAPREDUCE-5601. ShuffleHandler fadvises file regions as DONTNEED even when fetch fails (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1537855 13f79535-47bb-0310-9956-ffa450edef68
2013-11-01 08:51:54 +00:00
Bikas Saha 7dd201c541 Fix inadvertent file changes made via r1536888 (bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536894 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 21:11:32 +00:00
Bikas Saha 03510d00f4 YARN-1068. Add admin support for HA operations (Karthik Kambatla via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536888 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 21:03:54 +00:00
Jason Darrell Lowe c9643795c1 MAPREDUCE-5598. TestUserDefinedCounters.testMapReduceJob is flakey. Contributed by Robert Kanter
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536724 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 14:06:39 +00:00
Jason Darrell Lowe e0c99b80d4 MAPREDUCE-5596. Allow configuring the number of threads used to serve shuffle connections. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536711 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 13:41:50 +00:00
Sanford Ryza 84cec3c805 MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories (Robert Kanter via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1536558 13f79535-47bb-0310-9956-ffa450edef68
2013-10-28 23:48:38 +00:00
Jason Darrell Lowe 298bddf00b MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk. Contributed by Karthik Kambatla
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1534832 13f79535-47bb-0310-9956-ffa450edef68
2013-10-22 22:42:43 +00:00
Jonathan Turner Eagles 2baa42dd01 MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533629 13f79535-47bb-0310-9956-ffa450edef68
2013-10-18 20:57:48 +00:00
Sanford Ryza 7d637a3a99 MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1533624 13f79535-47bb-0310-9956-ffa450edef68
2013-10-18 20:43:53 +00:00
Suresh Srinivas dea1b2e84a HADOOP-10005. No need to check INFO severity level is enabled or not. Contributed by Jackie Chang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532907 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 21:00:07 +00:00
Jonathan Turner Eagles 74a2191aae MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532887 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:21:21 +00:00
Jonathan Turner Eagles 034039b917 MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532875 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 19:57:55 +00:00
Devarajulu K 4407d5f65f MAPREDUCE-5518. Fixed typo "can't read paritions file". Contributed by Albert Chu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532208 13f79535-47bb-0310-9956-ffa450edef68
2013-10-15 06:28:22 +00:00
Chris Nauroth ed45d97ed7 MAPREDUCE-5546. mapred.cmd on Windows set HADOOP_OPTS incorrectly. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1532016 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 18:36:34 +00:00
Chris Nauroth 11716cf390 HADOOP-10040. Revert svn propset for CRLF line endings on Windows files. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531977 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 17:01:33 +00:00
Sanford Ryza 6a1929fe31 MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531822 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 09:11:23 +00:00
Siddharth Seth f2745babf3 MAPREDUCE-5329. Allow MR applications to use additional AuxServices, which are compatible with the default MapReduce shuffle. Contributed by Avner BenHanoch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531741 13f79535-47bb-0310-9956-ffa450edef68
2013-10-13 20:17:53 +00:00
Chris Nauroth bbac0cf05e HADOOP-10040. hadoop.cmd in UNIX format and would not run by default on Windows. Contributed by Chris Nauroth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1531491 13f79535-47bb-0310-9956-ffa450edef68
2013-10-12 03:14:24 +00:00
Vinod Kumar Vavilapalli 9b9ddf29e2 YARN-1283. Fixed RM to give a fully-qualified proxy URL for an application so that clients don't need to do scheme-mangling. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530819 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 22:49:06 +00:00
Jason Darrell Lowe 00395a0064 MAPREDUCE-5102. fix coverage org.apache.hadoop.mapreduce.lib.db and org.apache.hadoop.mapred.lib.db. Contributed by Aleksey Gorshkov, Andrey Klochkov, and Nathan Roberts
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530816 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 22:39:39 +00:00
Jason Darrell Lowe 7b7caa1e56 MAPREDUCE-5569. FloatSplitter is not generating correct splits. Contributed by Nathan Roberts
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530683 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 15:43:36 +00:00
Daryn Sharp b858b812b7 HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules (Ivan A. Veselovsky via daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530667 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 15:09:37 +00:00
Arun Murthy 418773f44e Moving recent bug-fixes to hadoop-2.2.1.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530128 13f79535-47bb-0310-9956-ffa450edef68
2013-10-08 02:03:29 +00:00
Arun Murthy 43c7a834dc Preparing for hadoop-2.2.0 release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529750 13f79535-47bb-0310-9956-ffa450edef68
2013-10-07 05:17:00 +00:00
Vinod Kumar Vavilapalli f0799c5536 MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir cleanup, sending job-end notification correctly when unregister with RM fails. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529682 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 20:53:28 +00:00
Vinod Kumar Vavilapalli 21181b6553 YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529662 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 18:43:36 +00:00
Vinod Kumar Vavilapalli d4324eef14 MAPREDUCE-5533. Fixed MR speculation code to track any TaskAttempts that aren't heart-beating for a while, so that we can aggressively speculate instead of waiting for task-timeout. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529229 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 17:26:56 +00:00
Chris Nauroth 65cd7bf6b1 MAPREDUCE-5442. $HADOOP_MAPRED_HOME/$HADOOP_CONF_DIR setting not working on Windows. Contributed by Yingda Chen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529077 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 05:37:54 +00:00
Hitesh Shah 6ff600d9e3 YARN-1131. logs command should return an appropriate error message if YARN application is still running. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529068 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 02:36:17 +00:00
Bikas Saha fb3f338c65 MAPREDUCE-5489. MR jobs hangs as it does not use the node-blacklisting feature in RM requests (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529005 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:14:35 +00:00
Jonathan Turner Eagles 7b687dda09 YARN-1199. Make NM/RM Versions Available (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529003 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:04:12 +00:00
Sanford Ryza 90a5dc34f7 MAPREDUCE-5554. hdfs-site.xml included in hadoop-mapreduce-client-jobclient tests jar is breaking tests for downstream components (Robert Kanter via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528643 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 21:53:57 +00:00
Arun Murthy 3c7813bbad MAPREDUCE-5459. Update documentation on how to run MRv1 examples on YARN. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528626 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 21:08:27 +00:00
Arun Murthy 07ee205038 MAPREDUCE-5530. Fix compat with hadoop-1 in mapred.lib.CombinFileInputFormat by re-introducing isSplittable(FileSystem, Path) api and ensuring semantic compatibility. Contributed by Robert Kanter.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528533 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 16:08:45 +00:00
Vinod Kumar Vavilapalli 82f4348f27 MAPREDUCE-5536. Fixed MR AM and JHS to respect mapreduce.jobhistory.webapp.https.address. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528251 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:52:25 +00:00
Jason Darrell Lowe db06f1bcb9 MAPREDUCE-4421. Run MapReduce framework via the distributed cache. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528237 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:34:31 +00:00
Sanford Ryza cc8a0cab59 MAPREDUCE-5544. JobClient#getJob loads job conf twice. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1528196 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 20:01:20 +00:00
Arun Murthy 9c24615ae6 MAPREDUCE-5551. Fix compat with hadoop-1 in SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing missing constructors. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527848 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 01:24:27 +00:00
Vinod Kumar Vavilapalli aa12a7914f MAPREDUCE-5538. Fixed MR AppMaster to send job-notification URL only after the job is really done - a bug caused by MAPREDUCE-5505. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527219 13f79535-47bb-0310-9956-ffa450edef68
2013-09-28 19:14:16 +00:00
Arun Murthy a40810d267 MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner by re-introducing (get,set)PartitionFile which takes in JobConf. Contributed by Robert Kanter.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527048 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:42:34 +00:00
Jason Darrell Lowe 711abc3827 MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow times out. Contributed by Robert Kanter
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527037 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:14:07 +00:00
Arun Murthy 58f40d3bf4 MAPREDUCE-5531. Fix compat with hadoop-1 in mapreduce.(TaskID, TaskAttemptID) by re-introducing missing constructors. Contributed by Robert Kanter.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527033 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:08:28 +00:00
Jason Darrell Lowe 2627e352d6 MAPREDUCE-5332. Support token-preserving restart of history server. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1527015 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 18:19:41 +00:00
Arun Murthy 96c11fe602 MAPREDUCE-5514. Fix TestRMContainerAllocator. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526735 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 23:02:51 +00:00
Jason Darrell Lowe cf3d8b2286 MAPREDUCE-5513. ConcurrentModificationException in JobControl. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526688 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 21:10:05 +00:00
Chris Nauroth 7e73b9cde6 MAPREDUCE-5525. Increase timeout of TestDFSIO.testAppend and TestMRJobsWithHistoryService.testJobHistoryData. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526624 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 17:45:14 +00:00
Colin McCabe 8a66e493ba HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping. (Junping Du via Colin Patrick McCabe)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526567 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 15:24:30 +00:00
Vinod Kumar Vavilapalli 476eca47b5 MAPREDUCE-5170. Fixed a wrong log message in CombineFileInputFormat class. Contributed by Sangjin Lee.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526377 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 04:30:08 +00:00
Vinod Kumar Vavilapalli fb48b6cdc9 MAPREDUCE-5503. Fixed a test issue in TestMRJobClient. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526362 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 03:00:28 +00:00
Bikas Saha 524dad1109 MAPREDUCE-5505. Clients should be notified job finished only after job successfully unregistered (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526071 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:46:48 +00:00
Siddharth Seth 10a4289ebf YARN-1229. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526065 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:35:03 +00:00
Vinod Kumar Vavilapalli 78e995a37d YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525947 13f79535-47bb-0310-9956-ffa450edef68
2013-09-24 17:20:18 +00:00
Robert Joseph Evans 743d8ae055 MAPREDUCE-5522. Incorrect oreder expected from JobQueueInfo (Jinghui Wang via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1525670 13f79535-47bb-0310-9956-ffa450edef68
2013-09-23 18:33:12 +00:00
Vinod Kumar Vavilapalli 13420d01f2 YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524864 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 23:35:58 +00:00
Vinod Kumar Vavilapalli af78fd729c MAPREDUCE-5488. Changed MR client to keep trying to reach the application when it sees that on attempt's AM is down. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524856 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 22:35:12 +00:00
Thomas Graves 6b1f5073a7 MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524841 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 20:58:18 +00:00
Sanford Ryza c09f00771e MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again in Limits (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1524408 13f79535-47bb-0310-9956-ffa450edef68
2013-09-18 13:44:33 +00:00
Arun Murthy 492ecd34c4 Preparing for hadoop-2.1.1-beta release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523890 13f79535-47bb-0310-9956-ffa450edef68
2013-09-17 05:16:21 +00:00
Arun Murthy 59587d9fad MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on shuffle completion. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523660 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:18:42 +00:00
Arun Murthy 60a1a70197 YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contibuted by Binglin Chang.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523652 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:04:04 +00:00