Commit Graph

1594 Commits

Author SHA1 Message Date
Jonathan Turner Eagles 72feaf0436 MAPREDUCE-5637. Convert Hadoop Streaming document to APT (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1592792 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 16:05:37 +00:00
Tsz-wo Sze 6b7d074a54 svn merge -c 1592703 from trunk for MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1592704 13f79535-47bb-0310-9956-ffa450edef68
2014-05-06 10:25:10 +00:00
Jonathan Turner Eagles 73e0b8e55a MAPREDUCE-5638. Port Hadoop Archives document to trunk (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1591108 13f79535-47bb-0310-9956-ffa450edef68
2014-04-29 21:27:14 +00:00
Jason Darrell Lowe 3434d7b78b svn merge -c 1590668 FIXES: MAPREDUCE-5812. Make job context available to OutputCommitter.isRecoverySupported(). Contributed by Mohammad Kamrul Islam
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590675 13f79535-47bb-0310-9956-ffa450edef68
2014-04-28 15:27:01 +00:00
Eric Yang fef656527a MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob.
(Jinghui Wang via Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590267 13f79535-47bb-0310-9956-ffa450edef68
2014-04-26 18:02:19 +00:00
Jason Darrell Lowe 2a76cade32 svn merge -c 1590168 FIXES: MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state. Contributed by Ming Ma
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590174 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 22:34:44 +00:00
Jonathan Turner Eagles e0cd544a53 MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590131 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 20:32:09 +00:00
Jason Darrell Lowe aa3d6ed6ea svn merge -c 1590046 FIXES: MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590047 13f79535-47bb-0310-9956-ffa450edef68
2014-04-25 14:56:35 +00:00
Haohui Mai 1566fff15f MAPREDUCE-5837. Merge r1589903 from trunk.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589904 13f79535-47bb-0310-9956-ffa450edef68
2014-04-24 22:55:25 +00:00
Ravi Prakash 10510bf324 MAPREDUCE-5846. Rumen doesn't understand JobQueueChangedEvent (Nathan Roberts via raviprak)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589597 13f79535-47bb-0310-9956-ffa450edef68
2014-04-24 06:05:31 +00:00
Vinod Kumar Vavilapalli b04bb049d2 MAPREDUCE-5843. Fixed TestMRKeyValueTextInputFormat to not leak files and thus avoid failing on Windows. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1589533 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589534 13f79535-47bb-0310-9956-ffa450edef68
2014-04-23 22:54:55 +00:00
Jason Darrell Lowe c02ab95ee6 svn merge -c 1589524 FIXES: MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589526 13f79535-47bb-0310-9956-ffa450edef68
2014-04-23 21:57:55 +00:00
Vinod Kumar Vavilapalli c5450fd0fe MAPREDUCE-5832. Fixed TestJobClient to not fail on JDK7 or on Windows. Contributed by Jian He and Vinod Kumar Vavilapalli.
svn merge --ignore-ancestry -c 1589315 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589316 13f79535-47bb-0310-9956-ffa450edef68
2014-04-23 00:02:46 +00:00
Chris Nauroth 2d8f62f5c1 MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589259 13f79535-47bb-0310-9956-ffa450edef68
2014-04-22 19:32:17 +00:00
Chris Nauroth babe6545d4 MAPREDUCE-5827. Merging change r1589238 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589241 13f79535-47bb-0310-9956-ffa450edef68
2014-04-22 18:25:02 +00:00
Chris Nauroth d87463514a MAPREDUCE-5827. Merging change r1589223 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589236 13f79535-47bb-0310-9956-ffa450edef68
2014-04-22 18:22:26 +00:00
Chris Nauroth f0d3664d83 MAPREDUCE-5852. Merging change r1589006 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589007 13f79535-47bb-0310-9956-ffa450edef68
2014-04-21 23:49:28 +00:00
Chris Nauroth a7a7a16673 MAPREDUCE-5642. Merging change r1588605 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588606 13f79535-47bb-0310-9956-ffa450edef68
2014-04-19 04:21:01 +00:00
Jason Darrell Lowe c8d5d1f82a svn merge -c 1588559 FIXES: MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly. Contributed by Eric Payne
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588561 13f79535-47bb-0310-9956-ffa450edef68
2014-04-18 20:40:00 +00:00
Jonathan Turner Eagles 327efd2672 MAPREDUCE-3191. docs for map output compression incorrectly reference SequenceFile (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588010 13f79535-47bb-0310-9956-ffa450edef68
2014-04-16 18:16:58 +00:00
Devarajulu K 8f0bc218b6 MAPREDUCE-5775. Remove unnecessary job.setNumReduceTasks in SleepJob.createJob. Contributed by jhanver chand sharma.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587832 13f79535-47bb-0310-9956-ffa450edef68
2014-04-16 08:33:36 +00:00
Jonathan Turner Eagles 53ab439204 MAPREDUCE-5836. Fix typo in RandomTextWriter (Akira AJISAKA via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587722 13f79535-47bb-0310-9956-ffa450edef68
2014-04-15 20:46:30 +00:00
Vinod Kumar Vavilapalli 982ab03fbb MAPREDUCE-5828. Fixed a test issue with TestMapReduceJobControl that was causing it to fail on Windows. Contributed by Vinod Kumar Vavilapalli.
svn merge --ignore-ancestry -c 1587089 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587090 13f79535-47bb-0310-9956-ffa450edef68
2014-04-13 19:55:57 +00:00
Vinod Kumar Vavilapalli b92a4cf472 MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions on Windows for temporary files and thus also fix the test-issue with
TestHistoryServerFileSystemStateStoreService. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1586567 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586568 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 04:29:28 +00:00
Vinod Kumar Vavilapalli 7eff6cc305 MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster gracefully handle empty-queue names. Contributed by Akira Ajisaka.
svn merge --ignore-ancestry -c 1586559 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586560 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 04:02:19 +00:00
Jason Darrell Lowe 58aefafe76 svn merge -c 1586479 FIXES: MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586482 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 22:09:16 +00:00
Vinod Kumar Vavilapalli ec907fef0e MAPREDUCE-5824. Fixed test-failure of TestPipesNonJavaInputFormat in Windows. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1586368 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586369 13f79535-47bb-0310-9956-ffa450edef68
2014-04-10 16:44:31 +00:00
Zhijie Shen a4029b2276 MAPREDUCE-5818. Added "hsadmin" command into mapred.cmd. Contributed by Jian He.
svn merge --ignore-ancestry -c 1584378 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1584380 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 20:26:49 +00:00
Aaron Myers 71a7f2b72b MAPREDUCE-5014. Extending DistCp through a custom CopyListing is not possible. (Contributed by Srikanth Sundarrajan)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1584232 13f79535-47bb-0310-9956-ffa450edef68
2014-04-03 00:49:52 +00:00
Sanford Ryza 96106ff46e MAPREDUCE-5759. Remove unnecessary conf load in Limits (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583498 13f79535-47bb-0310-9956-ffa450edef68
2014-04-01 00:17:18 +00:00
Arun Murthy 21a1744039 Preparing for hadoop-2.4.0 release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583246 13f79535-47bb-0310-9956-ffa450edef68
2014-03-31 07:39:04 +00:00
Arun Murthy e052a4579f Merge -c 1583050 from trunk to branch-2 to fix MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the local-filesystem, rather than rely on the classpath. Contributed by Gera Shegalov.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583051 13f79535-47bb-0310-9956-ffa450edef68
2014-03-29 19:16:03 +00:00
Vinod Kumar Vavilapalli 8753d10405 MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. Contributed by Akira Ajisaka.
svn merge --ignore-ancestry -c 1582896 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1582897 13f79535-47bb-0310-9956-ffa450edef68
2014-03-28 22:17:44 +00:00
Vinod Kumar Vavilapalli 039162818b MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka.
svn merge --ignore-ancestry -c 1582125 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1582127 13f79535-47bb-0310-9956-ffa450edef68
2014-03-26 23:44:17 +00:00
Vinod Kumar Vavilapalli e51fbb6a9e MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it recovers from a commit during a previous attempt. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1581180 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581181 13f79535-47bb-0310-9956-ffa450edef68
2014-03-25 02:01:13 +00:00
Chris Nauroth c1e93e1f90 MAPREDUCE-5791. Merging change r1580994 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580996 13f79535-47bb-0310-9956-ffa450edef68
2014-03-24 19:03:09 +00:00
Vinod Kumar Vavilapalli 751bba3d14 MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1580100 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580101 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 00:33:21 +00:00
Vinod Kumar Vavilapalli 444af8fac5 MAPREDUCE-5787. Added the ability to keep alive shuffle connections in the MapReduce shuffle-handler. Contributed by Rajesh Balamohan.
svn merge --ignore-ancestry -c 1580062 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580063 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 21:44:24 +00:00
Kihwal Lee 2c3feeb727 svn merge -c 1579995 merging from trunk to branch-2 to fix:MAPREDUCE-5804. TestMRJobsWithProfiler#testProfiler timesout. Contributed by Mit Desai.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579998 13f79535-47bb-0310-9956-ffa450edef68
2014-03-21 17:58:33 +00:00
Vinod Kumar Vavilapalli 31a3a99f05 MAPREDUCE-2349. Modified FileInputFormat to be able to issue file and block location calls in parallel. Contributed by Siddharth Seth.
svn merge --ignore-ancestry -c 1579515 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579516 13f79535-47bb-0310-9956-ffa450edef68
2014-03-20 02:48:48 +00:00
Vinod Kumar Vavilapalli d470c7b71a YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He.
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
2014-03-16 19:13:16 +00:00
Jason Darrell Lowe c7a6775d94 svn merge -c 1577692 FIXES: MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577696 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 20:37:40 +00:00
Jason Darrell Lowe fd118493c6 svn merge -c 1577647 FIXES: MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577648 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 18:20:04 +00:00
Jason Darrell Lowe dc7cae391d Update CHANGES.txt to show MR-5688 pulled into 2.4.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577587 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 15:55:59 +00:00
Jason Darrell Lowe 838b4d3f84 MAPREDUCE-5751. MR app master fails to start in some cases if mapreduce.job.classloader is true. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577555 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 14:47:37 +00:00
Jonathan Turner Eagles d2990f85b3 MAPREDUCE-5456. TestFetcher.testCopyFromHostExtraBytes is missing (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577318 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 20:58:27 +00:00
Arpit Agarwal fd4d1d1d05 MAPREDUCE-5794. Merging r1577281 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577283 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 18:57:01 +00:00
Jonathan Turner Eagles eaf54d5a10 MAPREDUCE-5713. InputFormat and JobConf JavaDoc Fixes (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577240 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 16:50:13 +00:00
Jason Darrell Lowe 7f5a157995 svn merge -c 1577202 FIXES: MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page. Contributed by Rushabh S Shah
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577207 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:42:51 +00:00
Jonathan Turner Eagles 50c410546b MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577203 13f79535-47bb-0310-9956-ffa450edef68
2014-03-13 15:35:46 +00:00
Arun Murthy 9e695db974 Merge -c 1576946 from trunk to branch-2 to fix MAPREDUCE-5553. Allow users to easily access completed/pending/successful/failed tasks on MR AM web-ui. Contributed by Paul Han.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576947 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 22:05:54 +00:00
Jason Darrell Lowe 804f86f42c svn merge -c 1576940 FIXES: MAPREDUCE-5778. JobSummary does not escape newlines in the job name. Contributed by Akira AJISAKA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576943 13f79535-47bb-0310-9956-ffa450edef68
2014-03-12 21:59:50 +00:00
Vinod Kumar Vavilapalli ad5c0e4613 MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail when a large value of io.sort.mb is set. Contributed by Karthik Kambatla.
svn merge --ignore-ancestry -c 1576170 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576171 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 01:30:20 +00:00
Tsz-wo Sze eb4e72d190 svn merge -c 1575049 from trunk for MAPREDUCE-5780. SliveTest should use the specified path to get the particular FileSystem instead of using the default FileSystem.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575055 13f79535-47bb-0310-9956-ffa450edef68
2014-03-06 21:40:31 +00:00
Jason Darrell Lowe 8da70ca8e0 svn merge -c 1573775 FIXES: MAPREDUCE-5773. Provide dedicated MRAppMaster syslog length limit. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573778 13f79535-47bb-0310-9956-ffa450edef68
2014-03-03 22:12:18 +00:00
Jason Darrell Lowe 4d54763a07 svn merge -c 1573035 FIXES: MAPREDUCE-5768. TestMRJobs.testContainerRollingLog fails on trunk. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573037 13f79535-47bb-0310-9956-ffa450edef68
2014-02-28 19:28:03 +00:00
Karthik Kambatla d00de6418a MAPREDUCE-5665. Add audience annotations to MiniMRYarnCluster and MiniMRCluster. (Anubhav Dhoot via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572796 13f79535-47bb-0310-9956-ffa450edef68
2014-02-28 00:53:23 +00:00
Vinod Kumar Vavilapalli 04474afc81 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He.
svn merge --ignore-ancestry -c 1572711 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572712 13f79535-47bb-0310-9956-ffa450edef68
2014-02-27 19:24:49 +00:00
Jason Darrell Lowe 58d84af922 svn merge -c 1572269 FIXES: Preserve Job diagnostics in history. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572276 13f79535-47bb-0310-9956-ffa450edef68
2014-02-26 21:42:47 +00:00
Vinod Kumar Vavilapalli a6529b363c MAPREDUCE-5761. Added a simple log message to denote when encrypted shuffle is on in the shuffle-handler. Contributed by Jian He.
svn merge --ignore-ancestry -c 1571514 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1571515 13f79535-47bb-0310-9956-ffa450edef68
2014-02-25 01:24:50 +00:00
Jonathan Turner Eagles 555d4b513f MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570635 13f79535-47bb-0310-9956-ffa450edef68
2014-02-21 16:53:15 +00:00
Jonathan Turner Eagles 9ffb6429a4 MAPREDUCE-5671. NaN can be created by client and assign to Progress (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570623 13f79535-47bb-0310-9956-ffa450edef68
2014-02-21 16:32:52 +00:00
Arun Murthy 1ccccb8534 Branching for hadoop-2.4 releases, also set version to 2.5.0-SNAPSHOT in branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569186 13f79535-47bb-0310-9956-ffa450edef68
2014-02-18 04:41:01 +00:00
Jason Darrell Lowe 3452748b21 svn merge -c 1569038 FIXES: MAPREDUCE-5757. ConcurrentModificationException in JobControl.toList. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569043 13f79535-47bb-0310-9956-ffa450edef68
2014-02-17 16:25:55 +00:00
Jason Darrell Lowe 62cc4908c5 svn merge -c 1568118 FIXES: MAPREDUCE-5670. CombineFileRecordReader should report progress when moving to the next file. Contributed by Chen He
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1568126 13f79535-47bb-0310-9956-ffa450edef68
2014-02-13 23:21:38 +00:00
Karthik Kambatla 22a0f31d66 MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job. (Jason Lowe via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567669 13f79535-47bb-0310-9956-ffa450edef68
2014-02-12 16:08:02 +00:00
Arun Murthy 3de6a0219e Preparing to release hadoop-2.3.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567112 13f79535-47bb-0310-9956-ffa450edef68
2014-02-11 13:27:18 +00:00
Karthik Kambatla d082b122f9 MAPREDUCE-5744. Job hangs because RMContainerAllocator.preemptReduce() violates the comparator contract (Gera Shegalov via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565479 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 23:31:04 +00:00
Vinod Kumar Vavilapalli 89fbc0ce18 MAPREDUCE-5743. Fixed CHANGES.txt to give credit to Ted too.
svn merge --ignore-ancestry -c 1565401 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565402 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 19:08:09 +00:00
Karthik Kambatla fa43d8fb45 MAPREDUCE-5699. Allow setting tags on MR jobs (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565385 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 18:30:43 +00:00
Zhijie Shen 5afaafdb7f MAPREDUCE-5743. Fixed the test failure in TestRMContainerAllocator. Contributed by Vinod Kumar Vavilapalli.
svn merge --ignore-ancestry -c 1565087 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565088 13f79535-47bb-0310-9956-ffa450edef68
2014-02-06 06:39:29 +00:00
Vinod Kumar Vavilapalli 73256800db Updating branch-2.3's YARN's CHANGES.txt to reflect the recent rebase of unreleased 2.3 to more latest branch-2.
Fixing entry for MAPREDUCE-5732 in trunk and branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562949 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 20:30:12 +00:00
Vinod Kumar Vavilapalli deb5e05964 Updating branch-2's MAPREDUCE CHANGES.txt to reflect the recent rebase of unreleased 2.3 to more latest branch-2.
Updating trunk's MAPREDUCE CHANGES.txt to fix more differences with branch-2's CHANGES.txt
MAPREDUCE's CHANGES.txt is foobar'ed. There are still a bunch of misplaced CHANGES.txt entries - not fixing them now.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562945 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 20:16:56 +00:00
Sanford Ryza 28b178f55e MAPREDUCE-5732. Report proper queue when job has been automatically placed (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562906 13f79535-47bb-0310-9956-ffa450edef68
2014-01-30 17:57:08 +00:00
Sanford Ryza be75f21b6f MAPREDUCE-5464. Add analogs of the SLOTS_MILLIS counters that jive with the YARN resource model (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562229 13f79535-47bb-0310-9956-ffa450edef68
2014-01-28 21:05:03 +00:00
Karthik Kambatla 139b3a2ac4 MAPREDUCE-5723. MR AM container log can be truncated or empty. (Mohammad Kamrul Islam via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1560593 13f79535-47bb-0310-9956-ffa450edef68
2014-01-23 06:09:21 +00:00
Jason Darrell Lowe 033d60a28f svn merge -c 1560148 FIXES: MAPREDUCE-5693. Restore MRv1 behavior for log flush. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1560149 13f79535-47bb-0310-9956-ffa450edef68
2014-01-21 19:27:57 +00:00
Sanford Ryza 8adbc39208 MAPREDUCE-5725. Make explicit that TestNetworkedJob relies on the Capacity Scheduler (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1559884 13f79535-47bb-0310-9956-ffa450edef68
2014-01-21 01:24:03 +00:00
Karthik Kambatla 57adfd42d4 MAPREDUCE-5729. mapred job -list throws NPE (kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1559812 13f79535-47bb-0310-9956-ffa450edef68
2014-01-20 19:30:12 +00:00
Sanford Ryza 7c7f33a178 MAPREDUCE-5650. Job fails when hprof mapreduce.task.profile.map/reduce.params is specified (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1559202 13f79535-47bb-0310-9956-ffa450edef68
2014-01-17 17:53:23 +00:00
Jason Darrell Lowe 99ce917eaf svn merge -c 1558959 FIXES: Moving MR-5672 from bugs to improvements in CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558961 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 23:18:17 +00:00
Jason Darrell Lowe 0fa22b8120 svn merge -c 1558948 FIXES: MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558957 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 23:13:17 +00:00
Alejandro Abdelnur 0c84c76954 MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558853 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 17:12:31 +00:00
Konstantin Boudnik c9e403f652 Adding 2.0.6-alpha release to the CHANGES.txt
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558216 13f79535-47bb-0310-9956-ffa450edef68
2014-01-14 21:52:06 +00:00
Alejandro Abdelnur 47469c464c MAPREDUCE-3310. Custom grouping comparator cannot be set for Combiners (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1555969 13f79535-47bb-0310-9956-ffa450edef68
2014-01-06 18:37:40 +00:00
Karthik Kambatla 2bbd22bbd5 MAPREDUCE-5689. MRAppMaster does not preempt reducers when scheduled maps cannot be fulfilled. (lohit via kasha)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1555162 13f79535-47bb-0310-9956-ffa450edef68
2014-01-03 17:32:02 +00:00
Vinod Kumar Vavilapalli eb247009b2 MAPREDUCE-5685. Fixed a bug with JobContext getCacheFiles API inside the WrappedReducer class. Contributed by Yi Song.
svn merge --ignore-ancestry -c 1554320 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1554321 13f79535-47bb-0310-9956-ffa450edef68
2013-12-30 21:15:13 +00:00
Vinod Kumar Vavilapalli 234b01e862 MAPREDUCE-5694. Fixed MR AppMaster to shutdown the LogManager so as to avoid losing syslog in some conditions. Contributed by Mohammad Kamrul Islam.
svn merge --ignore-ancestry -c 1553879 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1553880 13f79535-47bb-0310-9956-ffa450edef68
2013-12-28 17:51:37 +00:00
Sanford Ryza 76185bf7a3 MAPREDUCE-5550. Task Status message (reporter.setStatus) not shown in UI with Hadoop 2.0 (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1552809 13f79535-47bb-0310-9956-ffa450edef68
2013-12-20 21:03:55 +00:00
Sanford Ryza d7e028de1a MAPREDUCE-5692. Add explicit diagnostics when a task attempt is killed due to speculative execution (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1552799 13f79535-47bb-0310-9956-ffa450edef68
2013-12-20 19:57:16 +00:00
Vinod Kumar Vavilapalli e89a93b85b MAPREDUCE-5687. Fixed failure in TestYARNRunner caused by YARN-1446. Contributed by Jian He.
svn merge --ignore-ancestry -c 1551774 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551777 13f79535-47bb-0310-9956-ffa450edef68
2013-12-18 00:14:44 +00:00
Jason Darrell Lowe 5975cf8fa9 svn merge -c 1551616 FIXES: MAPREDUCE-5679. TestJobHistoryParsing has race condition. Contributed by Liyin Liang
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551619 13f79535-47bb-0310-9956-ffa450edef68
2013-12-17 16:50:57 +00:00
Jason Darrell Lowe 63cb6749bb svn merge -c 1551285 FIXES: MAPREDUCE-5623. TestJobCleanup fails because of RejectedExecutionException and NPE. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551287 13f79535-47bb-0310-9956-ffa450edef68
2013-12-16 17:51:13 +00:00
Chris Nauroth cb6f28e122 MAPREDUCE-5674. Merging change r1550472 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1550474 13f79535-47bb-0310-9956-ffa450edef68
2013-12-12 18:35:13 +00:00
Jonathan Turner Eagles b43308af0b MAPREDUCE-5052. Job History UI and web services confusing job start time and job submit time (Chen He via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1549903 13f79535-47bb-0310-9956-ffa450edef68
2013-12-10 17:28:45 +00:00
Jason Darrell Lowe 5c1cbcb058 svn merge -c 1549705 FIXES: MAPREDUCE-5656. bzip2 codec can drop records when reading data in splits. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1549710 13f79535-47bb-0310-9956-ffa450edef68
2013-12-09 23:42:36 +00:00
Jason Darrell Lowe 58b397a05b svn merge -c 1548197 FIXES: MAPREDUCE-5409. MRAppMaster throws InvalidStateTransitonException: Invalid event: TA_TOO_MANY_FETCH_FAILURE at KILLED for TaskAttemptImpl. Contributed by Gera Shegalov
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1548198 13f79535-47bb-0310-9956-ffa450edef68
2013-12-05 16:32:44 +00:00
Jonathan Turner Eagles 644750fc7d MAPREDUCE-5632. TestRMContainerAllocator#testUpdatedNodes fails (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1547932 13f79535-47bb-0310-9956-ffa450edef68
2013-12-04 22:18:21 +00:00
Jonathan Turner Eagles 4a9770a48f MAPREDUCE-5645. TestFixedLengthInputFormat fails with native libs (Mit Desai via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1547630 13f79535-47bb-0310-9956-ffa450edef68
2013-12-03 22:54:42 +00:00
Jonathan Turner Eagles 29cdf33779 MAPREDUCE-5640. Rename TestLineRecordReader in jobclient module (Jason Lowe via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1547156 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 19:14:56 +00:00
Arun Murthy f5dc7b0c6b Renamed releases. 2.2.1 -> 2.3.0 & 2.3.0 -> 2.4.0
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1547140 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 18:43:55 +00:00
Jason Darrell Lowe 6d57c95003 svn merge -c 1544266 FIXES: MAPREDUCE-5631. TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7. Contributed by Jonathan Eagles
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1544267 13f79535-47bb-0310-9956-ffa450edef68
2013-11-21 17:55:40 +00:00
Jonathan Turner Eagles 27c0b5fb20 MAPREDUCE-5625. TestFixedLengthInputFormat fails in jdk7 environment (Mariappan Asokan via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1542736 13f79535-47bb-0310-9956-ffa450edef68
2013-11-17 14:39:04 +00:00
Jonathan Turner Eagles 1bbeb539e6 MAPREDUCE-5373. TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1542459 13f79535-47bb-0310-9956-ffa450edef68
2013-11-16 03:34:10 +00:00
Chris Nauroth 6ce1f14fed MAPREDUCE-5616. Merging change r1542001 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1542003 13f79535-47bb-0310-9956-ffa450edef68
2013-11-14 18:12:48 +00:00
Sanford Ryza 71ce967380 MAPREDUCE-5481. Enable uber jobs to have multiple reducers (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541846 13f79535-47bb-0310-9956-ffa450edef68
2013-11-14 08:06:50 +00:00
Steve Loughran bdab9d5cfd MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541701 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 20:45:52 +00:00
Steve Loughran a7bfe3efe3 MAPREDUCE-6524 Move grizzly-test and junit dependencies to test scope
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541699 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 20:43:04 +00:00
Steve Loughran 5dd3da295d MAPREDUCE-5431 Missing pom dependency in MR-client
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541680 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 19:50:17 +00:00
Jason Darrell Lowe 55a9d7dc50 svn merge -c 1541566 FIXES: MAPREDUCE-5610. TestSleepJob fails in jdk7. Contributed by Jonathan Eagles
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541583 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 16:19:21 +00:00
Sanford Ryza 5653892571 MAPREDUCE-5613. DefaultSpeculator holds and checks hashmap that is alway empty (Gera Shegalov via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541435 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 09:26:30 +00:00
Sanford Ryza 7b819a313e MAPREDUCE-1176. FixedLengthInputFormat and FixedLengthRecordReader (Mariappan Asokan and BitsOfInfo via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1540932 13f79535-47bb-0310-9956-ffa450edef68
2013-11-12 03:11:19 +00:00
Jason Darrell Lowe 97dc10bee5 svn merge -c 1540813 FIXES: MAPREDUCE-5186. mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail. Contributed by Robert Parker and Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1540819 13f79535-47bb-0310-9956-ffa450edef68
2013-11-11 19:30:25 +00:00
Chris Nauroth c02e97c531 MAPREDUCE-5451. Merging change r1539413 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1539416 13f79535-47bb-0310-9956-ffa450edef68
2013-11-06 18:17:32 +00:00
Chris Nauroth 5b7f2a5a2d MAPREDUCE-5604. Merging change r1538035 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1538038 13f79535-47bb-0310-9956-ffa450edef68
2013-11-01 19:12:36 +00:00
Sanford Ryza d023d52062 MAPREDUCE-5601. ShuffleHandler fadvises file regions as DONTNEED even when fetch fails (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1537856 13f79535-47bb-0310-9956-ffa450edef68
2013-11-01 08:55:05 +00:00
Jason Darrell Lowe 4b316b63c9 svn merge -c 1536724 FIXES: MAPREDUCE-5598. TestUserDefinedCounters.testMapReduceJob is flakey. Contributed by Robert Kanter
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1536725 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 14:07:33 +00:00
Jason Darrell Lowe 72d881132f svn merge -c 1536711 FIXES: MAPREDUCE-5596. Allow configuring the number of threads used to serve shuffle connections. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1536720 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 13:53:55 +00:00
Sanford Ryza dd3e38280a MAPREDUCE-4680. Job history cleaner should only check timestamps of files in old enough directories (Robert Kanter via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1536561 13f79535-47bb-0310-9956-ffa450edef68
2013-10-28 23:54:52 +00:00
Jason Darrell Lowe d480fb0cbb svn commit -c 1534832 FIXES: MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk. Contributed by Karthik Kambatla
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1534835 13f79535-47bb-0310-9956-ffa450edef68
2013-10-22 22:44:25 +00:00
Jonathan Turner Eagles 3f86a06ec0 MAPREDUCE-5587. TestTextOutputFormat fails on JDK7 (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1533630 13f79535-47bb-0310-9956-ffa450edef68
2013-10-18 21:03:37 +00:00
Sanford Ryza 5c20de6ac3 MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1533626 13f79535-47bb-0310-9956-ffa450edef68
2013-10-18 20:46:05 +00:00
Jonathan Turner Eagles f35f88b1cf MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532890 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:28:13 +00:00
Jonathan Turner Eagles 85b5e96f10 MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532878 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 20:06:53 +00:00
Devarajulu K bb699cfb47 MAPREDUCE-5518. Fixed typo "can't read paritions file". Contributed by Albert Chu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532209 13f79535-47bb-0310-9956-ffa450edef68
2013-10-15 06:30:49 +00:00
Chris Nauroth c5f7e137d3 MAPREDUCE-5546. Merging change r1532016 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532017 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 18:37:41 +00:00
Sanford Ryza 1f77877bd9 MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1531823 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 09:12:11 +00:00
Siddharth Seth 5ff9f8bed2 merge MAPREDUCE-5329. Allow MR applications to use additional AuxServices, which are compatible with the default MapReduce shuffle. Contributed by Avner BenHanoch.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1531742 13f79535-47bb-0310-9956-ffa450edef68
2013-10-13 20:19:06 +00:00
Jason Darrell Lowe 9b5a023fc2 svn merge -c 1530816 FIXES: MAPREDUCE-5102. fix coverage org.apache.hadoop.mapreduce.lib.db and org.apache.hadoop.mapred.lib.db. Contributed by Aleksey Gorshkov, Andrey Klochkov, and Nathan Roberts
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530818 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 22:45:34 +00:00
Jason Darrell Lowe d5f32b0fab svn merge -c 1530683 FIXES: MAPREDUCE-5569. FloatSplitter is not generating correct splits. Contributed by Nathan Roberts
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530684 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 15:44:47 +00:00
Arun Murthy c3dff760d9 Moving recent bug-fixes to hadoop-2.2.1.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530129 13f79535-47bb-0310-9956-ffa450edef68
2013-10-08 02:05:03 +00:00
Arun Murthy 3b514585f1 Preparing for hadoop-2.2.0 release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529752 13f79535-47bb-0310-9956-ffa450edef68
2013-10-07 05:32:18 +00:00
Vinod Kumar Vavilapalli ad9515fa9b MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir cleanup, sending job-end notification correctly when unregister with RM fails. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1529682 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529683 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 20:54:10 +00:00
Vinod Kumar Vavilapalli 856907b8db MAPREDUCE-5533. Fixed MR speculation code to track any TaskAttempts that aren't heart-beating for a while, so that we can aggressively speculate instead of waiting for task-timeout. Contributed by Xuan Gong.
svn merge --ignore-ancestry -c 1529229 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529230 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 17:29:21 +00:00
Chris Nauroth 658905cd44 MAPREDUCE-5442. Merging change r1529077 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529079 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 05:42:58 +00:00
Bikas Saha 53df9bdb14 Merge r1529005 from trunk to branch-2 for MAPREDUCE-5489. MR jobs hangs as it does not use the node-blacklisting feature in RM requests (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529007 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:18:06 +00:00
Sanford Ryza c738c106e9 MAPREDUCE-5554. hdfs-site.xml included in hadoop-mapreduce-client-jobclient tests jar is breaking tests for downstream components (Robert Kanter via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528647 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 22:00:56 +00:00
Arun Murthy ce990cd911 Merge -c 1528626 from trunk to branch-2 to fix MAPREDUCE-5459. Update documentation on how to run MRv1 examples on YARN. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528627 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 21:09:59 +00:00
Arun Murthy f49d9cf7a4 Merge -c 1528533 from trunk to branch-2 to fix MAPREDUCE-5530. Fix compat with hadoop-1 in mapred.lib.CombinFileInputFormat by re-introducing isSplittable(FileSystem, Path) api and ensuring semantic compatibility. Contributed by Robert Kanter.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528535 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 16:09:53 +00:00
Vinod Kumar Vavilapalli 6daca6a93d MAPREDUCE-5536. Fixed MR AM and JHS to respect mapreduce.jobhistory.webapp.https.address. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1528251 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528253 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:54:35 +00:00
Jason Darrell Lowe 7e812e7126 svn merge -c 1528237 FIXES: MAPREDUCE-4421. Run MapReduce framework via the distributed cache. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528242 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:40:01 +00:00
Sanford Ryza 06e8660f2b MAPREDUCE-5544. JobClient#getJob loads job conf twice. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528197 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 20:05:09 +00:00
Arun Murthy 85f5626076 Merge -c 1527848 from trunk to branch-2 to fix MAPREDUCE-5551. Fix compat with hadoop-1 in SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing missing constructors. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527850 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 01:25:31 +00:00
Vinod Kumar Vavilapalli 3c3b52b80b MAPREDUCE-5538. Fixed MR AppMaster to send job-notification URL only after the job is really done - a bug caused by MAPREDUCE-5505. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1527219 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527221 13f79535-47bb-0310-9956-ffa450edef68
2013-09-28 19:14:46 +00:00
Arun Murthy c03040a498 Merge -c 1527048 from trunk to branch-2 to fix MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner by re-introducing (get,set)PartitionFile which takes in JobConf. Contributed by Robert Kanter.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527054 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 20:04:31 +00:00
Jason Darrell Lowe d36dec185e svn commit -c 1527037 FIXES: MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow times out. Contributed by Robert Kanter
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527039 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:17:24 +00:00
Arun Murthy 146d9264a9 Merge -c 1527033 from trunk to branch-2 to fix MAPREDUCE-5531. Fix compat with hadoop-1 in mapreduce.(TaskID, TaskAttemptID) by re-introducing missing constructors. Contributed by Robert Kanter.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527034 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:10:07 +00:00
Jason Darrell Lowe d8e9a472e7 svn merge -c 1527015 FIXES: MAPREDUCE-5332. Support token-preserving restart of history server. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527016 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 18:26:09 +00:00
Arun Murthy 78045b7abc Merge -c 1526735 from trunk to branch-2 to fix MAPREDUCE-5514. Fix TestRMContainerAllocator. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526736 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 23:03:25 +00:00
Jason Darrell Lowe 2b7ae8dd5d svn merge -c 1526688 FIXES: MAPREDUCE-5513. ConcurrentModificationException in JobControl. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526702 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 21:21:34 +00:00
Chris Nauroth 6bdd4791db MAPREDUCE-5525. Merging change r1526624 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526625 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 17:47:45 +00:00
Vinod Kumar Vavilapalli 6e132b1a2b MAPREDUCE-5170. Fixed a wrong log message in CombineFileInputFormat class. Contributed by Sangjin Lee.
svn merge --ignore-ancestry -c 1526377 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526379 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 04:30:36 +00:00
Vinod Kumar Vavilapalli b41137c73a MAPREDUCE-5503. Fixed a test issue in TestMRJobClient. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526364 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 03:02:06 +00:00
Bikas Saha a5ecb4de41 Merge r1526071 from trunk to branch-2 for MAPREDUCE-5505. Clients should be notified job finished only after job successfully unregistered (Zhijie Shen via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526072 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:53:10 +00:00
Siddharth Seth e23d5672a7 merge YARN-1229 from trunk. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526066 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:37:14 +00:00
Vinod Kumar Vavilapalli 2daddb610e YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1525947 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1525948 13f79535-47bb-0310-9956-ffa450edef68
2013-09-24 17:20:48 +00:00
Robert Joseph Evans e2bfdfc585 svn merge -c 1525670 FIXES: MAPREDUCE-5522. Incorrect oreder expected from JobQueueInfo (Jinghui Wang via bobby)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1525673 13f79535-47bb-0310-9956-ffa450edef68
2013-09-23 18:39:47 +00:00
Vinod Kumar Vavilapalli d71bb8385b YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi.
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1524864 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524867 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 23:51:14 +00:00
Vinod Kumar Vavilapalli d8ed9ef4b8 MAPREDUCE-5488. Changed MR client to keep trying to reach the application when it sees that on attempt's AM is down. Contributed by Jian He.
svn merge --ignore-ancestry -c 1524856 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524857 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 22:36:58 +00:00
Thomas Graves 11d0fe9722 MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524843 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 21:00:40 +00:00
Sanford Ryza f0e79b8157 MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again in Limits (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524417 13f79535-47bb-0310-9956-ffa450edef68
2013-09-18 13:58:37 +00:00
Arun Murthy 3836473c73 Preparing for hadoop-2.1.1-beta release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523892 13f79535-47bb-0310-9956-ffa450edef68
2013-09-17 05:23:39 +00:00
Arun Murthy 73c69decf1 Merge -c 1523660 from trunk to branch-2 to fix MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on shuffle completion. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523661 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:19:33 +00:00
Alejandro Abdelnur 7993c89ea8 MAPREDUCE-5379. Include token tracking ids in jobconf. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523607 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 12:20:30 +00:00
Devarajulu K 103226cc91 MAPREDUCE-5164. mapred job and queue commands omit HADOOP_CLIENT_OPTS. Contributed by Nemon Lou.
Merged revision 1522595 from trunk.



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1522597 13f79535-47bb-0310-9956-ffa450edef68
2013-09-12 14:34:39 +00:00
Vinod Kumar Vavilapalli 90f492e532 MAPREDUCE-5497. Changed MRAppMaster to sleep only after doing everything else but just before ClientService to avoid race conditions during RM restart. Contributed by Jian He.
svn merge --ignore-ancestry -c 1521699 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1521700 13f79535-47bb-0310-9956-ffa450edef68
2013-09-11 00:39:21 +00:00
Thomas Graves a574ef2864 MAPREDUCE-5020. Compile failure with JDK8 (Trevor Robinson via tgraves)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1521584 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 18:53:29 +00:00
Devarajulu K d997ca22f9 MAPREDUCE-5414. TestTaskAttempt fails in JDK7 with NPE. Contributed by Nemon Lou.
svn merge --ignore-ancestry -c 1520964 ../../trunk



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1520965 13f79535-47bb-0310-9956-ffa450edef68
2013-09-09 06:46:16 +00:00
Jason Darrell Lowe 5ddee655e4 svn merge -c 1520156 FIXES: MAPREDUCE-5475. MRClientService does not verify ACLs properly. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1520157 13f79535-47bb-0310-9956-ffa450edef68
2013-09-04 22:32:35 +00:00
Sanford Ryza 997e2f4fd3 MAPREDUCE-5484. YarnChild unnecessarily loads job conf twice (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1518858 13f79535-47bb-0310-9956-ffa450edef68
2013-08-29 23:53:29 +00:00
Vinod Kumar Vavilapalli 7b8795012e MAPREDUCE-5441. Changed MR AM to return RUNNING state if exiting when RM commands to reboot, so that client can continue to track the overall job. Contributed by Jian He.
svn merge --ignore-ancestry -c 1518821 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1518824 13f79535-47bb-0310-9956-ffa450edef68
2013-08-29 21:10:54 +00:00
Alejandro Abdelnur 23cbe7af90 MAPREDUCE-5483. revert MAPREDUCE-5357. (rkanter via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1518773 13f79535-47bb-0310-9956-ffa450edef68
2013-08-29 19:26:31 +00:00
Jason Darrell Lowe 35f60e71de svn merge -c 1517097 to revert MAPREDUCE-5475 and YARN-707
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1517098 13f79535-47bb-0310-9956-ffa450edef68
2013-08-24 01:23:39 +00:00
Jason Darrell Lowe 25595863d3 svn merge -c 1517085 FIXES: MAPREDUCE-5475. MRClientService does not verify ACLs properly. Contributed by Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1517086 13f79535-47bb-0310-9956-ffa450edef68
2013-08-23 22:55:45 +00:00
Sanford Ryza 5b8b923cd1 MAPREDUCE-5478. TeraInputFormat unnecessarily defines its own FileSplit subclass (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1517055 13f79535-47bb-0310-9956-ffa450edef68
2013-08-23 21:28:16 +00:00
Vinod Kumar Vavilapalli 91d8768381 MAPREDUCE-5476. Changed MR AM recovery code to cleanup staging-directory only after unregistering from the RM. Contributed by Jian He.
svn merge --ignore-ancestry -c 1516660 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1516662 13f79535-47bb-0310-9956-ffa450edef68
2013-08-22 23:17:57 +00:00
Chris Nauroth 3ad42fba26 MAPREDUCE-5470. Merging change r1516623 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1516625 13f79535-47bb-0310-9956-ffa450edef68
2013-08-22 21:43:57 +00:00
Jason Darrell Lowe 8d1cd8ddb7 svn merge -c 1516594 FIXES: Revert MAPREDUCE-5475
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1516596 13f79535-47bb-0310-9956-ffa450edef68
2013-08-22 20:38:06 +00:00
Arun Murthy b9cee48717 Merge -c 1516361 from trunk to branch-2 to fix MAPREDUCE-5475. Ensure MRClientService verifies ACLs for users. Contributed by Jason Lowe.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1516362 13f79535-47bb-0310-9956-ffa450edef68
2013-08-22 02:00:29 +00:00
Arun Murthy e8268b8a5d Merge -c 1516358 from trunk to branch-2 to fix MAPREDUCE-5468. Fix MR AM recovery for map-only jobs. Contributed by Vinod K. V.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1516359 13f79535-47bb-0310-9956-ffa450edef68
2013-08-22 01:53:18 +00:00
Vinod Kumar Vavilapalli 83c4906a2e MAPREDUCE-5466. Changed MR AM to not promote history files of intermediate AMs in case they are exiting because of errors and thus help history-server pick up the right history file for the last successful AM. Contributed by Jian He.
svn merge --ignore-ancestry -c 1516238 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1516240 13f79535-47bb-0310-9956-ffa450edef68
2013-08-21 18:18:13 +00:00
Jason Darrell Lowe 6f27db80ac svn merge -c 1515863 FIXES: MAPREDUCE-5001. LocalJobRunner has race condition resulting in job failures. Contributed by Sandy Ryza
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1515877 13f79535-47bb-0310-9956-ffa450edef68
2013-08-20 17:00:06 +00:00
Jason Darrell Lowe 0bf25e9d4c svn merge -c 1514768 FIXES: Update CHANGES.txt to reflect merge of MR-1981 to branch-2.1-beta
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1514769 13f79535-47bb-0310-9956-ffa450edef68
2013-08-16 16:14:51 +00:00
Sanford Ryza 0c32694b27 MAPREDUCE-5462. In map-side sort, swap entire meta entries instead of indexes for better cache performance. (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1514610 13f79535-47bb-0310-9956-ffa450edef68
2013-08-16 08:14:58 +00:00
Arun Murthy aa3f77675b Preparing for hadoop-2.1.0-beta rc2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1514471 13f79535-47bb-0310-9956-ffa450edef68
2013-08-15 20:34:25 +00:00
Siddharth Seth 56cfd507f1 Update CHANGES.txt to move YARN-1045 and MAPREDUCE-5352 to the correct version.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1514433 13f79535-47bb-0310-9956-ffa450edef68
2013-08-15 18:46:03 +00:00
Sanford Ryza 0d54ef3b62 MAPREDUCE-5454. TestDFSIO fails intermittently on JDK7 (Karthik Kambatla via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1513376 13f79535-47bb-0310-9956-ffa450edef68
2013-08-13 07:12:49 +00:00
Jason Darrell Lowe 432eef4ede svn merge -c 1511464 FIXES: MAPREDUCE-5425. Junit in TestJobHistoryServer failing in jdk 7. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1511467 13f79535-47bb-0310-9956-ffa450edef68
2013-08-07 20:07:15 +00:00
Sanford Ryza 2a7293a254 MAPREDUCE-434. LocalJobRunner limited to single reducer (Sandy Ryza and Aaron Kimball via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1510867 13f79535-47bb-0310-9956-ffa450edef68
2013-08-06 06:40:42 +00:00
Sanford Ryza 75c40235aa MAPREDUCE-5399. Unnecessary Configuration instantiation in IFileInputStream slows down merge. (Stanislav Barton via Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1510813 13f79535-47bb-0310-9956-ffa450edef68
2013-08-05 23:35:03 +00:00
Sanford Ryza 3490c9aab7 MAPREDUCE-5367. Local jobs all use same local working directory (Sandy Ryza)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1510612 13f79535-47bb-0310-9956-ffa450edef68
2013-08-05 17:12:58 +00:00
Kihwal Lee 7d36680e73 svn merge -c 1510581 Merging from trunk to branch-2 to fix MAPREDUCE-5446. TestJobHistoryEvents and TestJobHistoryParsing have race conditions.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1510583 13f79535-47bb-0310-9956-ffa450edef68
2013-08-05 16:05:04 +00:00
Jason Darrell Lowe 4cd5cb862c svn merge -c 1509900 FIXES: MAPREDUCE-5440. TestCopyCommitter Fails on JDK7. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1509901 13f79535-47bb-0310-9956-ffa450edef68
2013-08-02 22:53:52 +00:00
Jason Darrell Lowe 3251cfca85 svn merge -c 1509499 FIXES: Update fix versions for JIRAs pulled from branch-2 into branch-2.1-beta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1509501 13f79535-47bb-0310-9956-ffa450edef68
2013-08-02 00:43:07 +00:00
Jason Darrell Lowe d10bda6ad9 svn merge -c 1509475 FIXES: Correct fix versions for MAPREDUCE-5428 and YARN-573.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1509476 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 23:55:30 +00:00
Jason Darrell Lowe b4c28a1aa9 svn merge -c 1509401 FIXES: MAPREDUCE-5428. HistoryFileManager doesn't stop threads when service is stopped. Contributed by Karthik Kambatla
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1509404 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 20:03:08 +00:00
Siddharth Seth d7f082ad5f merge MAPREDUCE-5352 from trunk. Optimize node local splits generated by CombineFileInputFormat. (sseth)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1509346 13f79535-47bb-0310-9956-ffa450edef68
2013-08-01 17:43:40 +00:00
Vinod Kumar Vavilapalli d00dcbd2f5 MAPREDUCE-5385. Fixed a bug with JobContext getCacheFiles API. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1508595 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1508596 13f79535-47bb-0310-9956-ffa450edef68
2013-07-30 20:05:19 +00:00
Arun Murthy e65416a459 Updating release date for hadoop-2.1.0-beta.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1508408 13f79535-47bb-0310-9956-ffa450edef68
2013-07-30 12:25:13 +00:00
Jason Darrell Lowe 3bd5174a8e svn merge -c 1508220 FIXES: MAPREDUCE-5411. Refresh size of loaded job cache on history server. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1508224 13f79535-47bb-0310-9956-ffa450edef68
2013-07-29 22:48:00 +00:00
Jason Darrell Lowe 5df508c539 svn merge -c 1507486 FIXES: MAPREDUCE-5419. TestSlive is getting FileNotFound Exception. Contributed by Robert Parker
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1507487 13f79535-47bb-0310-9956-ffa450edef68
2013-07-26 22:03:15 +00:00
Vinod Kumar Vavilapalli 6bc4693384 MAPREDUCE-5421. Fixed TestNonExistentJob failure after YARN-873. Contributed by Junping Du.
svn merge --ignore-ancestry -c 1507401 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1507402 13f79535-47bb-0310-9956-ffa450edef68
2013-07-26 19:16:08 +00:00
Jason Darrell Lowe f6611d89c8 svn merge -c 1507385 FIXES: MAPREDUCE-1981. Improve getSplits performance by using listLocatedStatus. Contributed by Hairong Kuang and Jason Lowe
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1507387 13f79535-47bb-0310-9956-ffa450edef68
2013-07-26 18:22:00 +00:00
Jason Darrell Lowe fca19f6a8a svn merge -c 1507135 FIXES: MAPREDUCE-5386. Ability to refresh history server job retention and job cleaner settings. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1507378 13f79535-47bb-0310-9956-ffa450edef68
2013-07-26 17:32:00 +00:00
Jason Darrell Lowe b11cceb25c svn merge -c 1507104 FIXES: MAPREDUCE-5251. Reducer should not implicate map attempt if it has insufficient space to fetch map output. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1507105 13f79535-47bb-0310-9956-ffa450edef68
2013-07-25 19:45:20 +00:00
Alejandro Abdelnur 98eb429771 MAPREDUCE-5213. Re-assess TokenCache methods marked @Private. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1506759 13f79535-47bb-0310-9956-ffa450edef68
2013-07-24 22:33:59 +00:00
Vinod Kumar Vavilapalli f487f4eb19 YARN-926. Modified ContainerManagerProtcol APIs to take in requests for multiple containers. Contributed by Jian He.
MAPREDUCE-5412. Update MR app to use multiple containers API of ContainerManager after YARN-926. Contributed by Jian He.
svn merge --ignore-ancestry -c 1506391 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1506392 13f79535-47bb-0310-9956-ffa450edef68
2013-07-24 03:42:52 +00:00
Jason Darrell Lowe d6be90bccf svn merge -c 1506226 FIXES: MAPREDUCE-5356. Ability to refresh aggregated log retention period and check interval. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1506230 13f79535-47bb-0310-9956-ffa450edef68
2013-07-23 19:25:19 +00:00
Jason Darrell Lowe 673b762364 svn merge -c 1506154 FIXES: MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1506157 13f79535-47bb-0310-9956-ffa450edef68
2013-07-23 16:58:41 +00:00
Jason Darrell Lowe 726533a741 svn merge -c 1505771 FIXES: MAPREDUCE-5404. HSAdminServer does not use ephemeral ports in minicluster mode. Contributed by Ted Yu
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1505772 13f79535-47bb-0310-9956-ffa450edef68
2013-07-22 19:06:07 +00:00
Arun Murthy 1f65519b8a Fixed CHANGES.txt to reflect that new content in branch-2 goes to hadoop-2.3.0.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1504743 13f79535-47bb-0310-9956-ffa450edef68
2013-07-19 01:30:18 +00:00
Jason Darrell Lowe 9b1bc20045 svn merge -c 1504645 FIXES: MAPREDUCE-5265. History server admin service to refresh user and superuser group mappings. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1504653 13f79535-47bb-0310-9956-ffa450edef68
2013-07-18 20:47:58 +00:00
Jason Darrell Lowe a55ae9e988 svn merge -c 1503957 FIXES: MAPREDUCE-5380. Invalid mapred command should return non-zero exit code. Contributed by Stephen Chu
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1503958 13f79535-47bb-0310-9956-ffa450edef68
2013-07-17 00:31:52 +00:00
Bikas Saha 41c05101e7 Merge r1503934 from trunk to branch-2 for MAPREDUCE-5398. MR changes for YARN-513 (Jian He via bikas)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1503936 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 22:56:54 +00:00
Jason Darrell Lowe 2386e127b8 svn merge -c 1503744 to revert MAPREDUCE-5317. Stale files left behind for failed jobs
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1503747 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 15:06:42 +00:00
Jason Darrell Lowe 293d42edf0 svn merge -c 1503499 FIXES: MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1503505 13f79535-47bb-0310-9956-ffa450edef68
2013-07-15 22:30:39 +00:00
Alejandro Abdelnur f0983d958d updating CHANGES.txt after committing MAPREDUCE-5333,HADOOP-9661,HADOOP-9355,HADOOP-9673,HADOOP-9414,HADOOP-9416,HDFS-4797,YARN-866,YARN-736,YARN-883 to 2.1-beta branch
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1502074 13f79535-47bb-0310-9956-ffa450edef68
2013-07-11 00:21:49 +00:00
Chris Nauroth 9bd8745846 MAPREDUCE-4374. Merging change r1502046 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1502048 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 23:08:34 +00:00
Hitesh Shah 1157061429 merge -c 1501599 from trunk to branch-2 to fix YARN-727, MAPREDUCE-5325. ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1501600 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 23:07:15 +00:00
Chris Nauroth fd159ded9b MAPREDUCE-5187. Merging change r1500929 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1500930 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 20:28:55 +00:00
Chris Nauroth 2a448cc5e7 MAPREDUCE-5366. Merging change r1500842 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1500848 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 17:17:07 +00:00
Chris Nauroth 81bfd0316e MAPREDUCE-5360. Merging change r1500194 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1500195 13f79535-47bb-0310-9956-ffa450edef68
2013-07-06 03:42:43 +00:00
Jason Darrell Lowe afeff43d7a svn merge -c 1499425 FIXES: MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl. Contributed by Devaraj K
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1499426 13f79535-47bb-0310-9956-ffa450edef68
2013-07-03 14:46:54 +00:00
Chris Nauroth 8c10908e1e MAPREDUCE-5357. Merging change r1499210 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1499217 13f79535-47bb-0310-9956-ffa450edef68
2013-07-03 05:37:34 +00:00
Chris Nauroth 0f3bb0567d MAPREDUCE-5359. Merging change r1499153 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1499154 13f79535-47bb-0310-9956-ffa450edef68
2013-07-03 00:01:28 +00:00
Chris Nauroth 6c703d6ef0 MAPREDUCE-5355. Merging change r1499148 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1499149 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 23:32:41 +00:00
Jason Darrell Lowe 199485bcdf svn merge -c 1499125 FIXES: MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the input path dir. Contributed by Devaraj K
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1499127 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 22:00:43 +00:00
Thomas Graves 9520013838 Preparing for 0.23.9 release
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1498478 13f79535-47bb-0310-9956-ffa450edef68
2013-07-01 14:27:48 +00:00
Chris Nauroth 5808ba30dd MAPREDUCE-5349. TestClusterMapReduceTestCase and TestJobName fail on Windows in branch-2. Contributed by Chuan Liu.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1498144 13f79535-47bb-0310-9956-ffa450edef68
2013-06-30 15:53:29 +00:00
Alejandro Abdelnur 5d45387c00 MAPREDUCE-5333. Add test that verifies MRAM works correctly when sending requests with non-normalized capabilities. (ywskycn via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1497881 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 18:55:31 +00:00
Arun Murthy de82f25604 Preparing for hadoop-2.1.0-beta release.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1496790 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 06:51:07 +00:00
Arun Murthy 2977cbfc83 Merge -c 1496741 from trunk to branch-2 to fix MAPREDUCE-5326. Added version to shuffle header. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1496742 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 05:50:23 +00:00
Siddharth Seth aa1eba9930 merge MAPREDUCE-5334 from trunk. Fix failing unit tests - TestContainerLauncher, TestContainerLauncherImpl. Contributed by Vinod Kumar Vavilapalli.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1494821 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 00:36:28 +00:00
Jonathan Turner Eagles d8203c0ae9 MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1494687 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 16:40:20 +00:00
Christopher Douglas 6267cddbd6 MAPREDUCE-5194. Heed interrupts during Fetcher shutdown.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1494417 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 02:12:20 +00:00
Jason Darrell Lowe 20a950a374 MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1494292 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 20:39:56 +00:00
Arun Murthy 51bb83eabf Merge -c 1493570 from trunk to branch-2 to fix MAPREDUCE-5184. Document compatibility for MapReduce applications in hadoop-2 vis-a-vis hadoop-1. Contributed by Zhijie Shen.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1493571 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 19:10:32 +00:00
Arun Murthy a00e4a926d Merge -c 1493445 from trunk to branch-2 to fix MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents. Contributed by Chris Douglas.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1493446 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 03:07:56 +00:00
Alejandro Abdelnur 77ae68f84e MAPREDUCE-5310. MRAM should not normalize allocation request capabilities. (tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1493211 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 19:09:15 +00:00
Arun Murthy 451f725b92 Merge -c 1492962 from trunk to branch-2 to fix MAPREDUCE-5319. Set user.name in job.xml. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1492963 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 06:27:38 +00:00
Jason Darrell Lowe 85543c3683 MAPREDUCE-4019. -list-attempt-ids is not working. Contributed by Ashwin Shankar, Devaraj K, and B Anil Kumar
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1492877 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 21:35:34 +00:00
Vinod Kumar Vavilapalli 027f71eb05 MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed. Contributed by Daryn Sharp.
svn merge --ignore-ancestry -c 1492848 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1492849 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 20:21:03 +00:00
Vinod Kumar Vavilapalli 903f0fea8e YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran.
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.
svn merge --ignore-ancestry -c 1492718 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1492720 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 15:56:44 +00:00
Chris Nauroth 87c96a022e MAPREDUCE-5259. Merging change r1492364 from trunk to branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1492367 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 19:00:05 +00:00
Alejandro Abdelnur 3af49516ca MAPREDUCE-5304. mapreduce.Job killTask/failTask/getTaskCompletionEvents methods have incompatible signature changes. (kkambatl via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1492362 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 18:50:46 +00:00
Daryn Sharp 45ca24c1d8 svn merge -c 1491914 FIXES: MAPREDUCE-5315. DistCp reports success even on failure. (mithun and jlowe via daryn)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1491918 13f79535-47bb-0310-9956-ffa450edef68
2013-06-11 19:30:44 +00:00
Jason Darrell Lowe 12f72e0ba6 svn merge -c 1491611 FIXES: MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching multiple compressed map outputs. Contributed by Nathan Roberts
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1491614 13f79535-47bb-0310-9956-ffa450edef68
2013-06-10 21:45:35 +00:00
Konstantin Boudnik 6e9ad986eb Setting 2.0.5-alpha release date
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1491137 13f79535-47bb-0310-9956-ffa450edef68
2013-06-09 04:51:28 +00:00
Alejandro Abdelnur f55574bfdf MAPREDUCE-5312. TestRMNMInfo is failing. (sandyr via tucu)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1491044 13f79535-47bb-0310-9956-ffa450edef68
2013-06-08 19:43:11 +00:00
Thomas Graves ee792cf084 Updating release date for 0.23.8
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1489902 13f79535-47bb-0310-9956-ffa450edef68
2013-06-05 15:01:24 +00:00