Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,786 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

493 Commits

Author SHA1 Message Date
Arun C. Murthy adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy 175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408)
 2014-11-09 15:03:59 -08:00
Jason Lowe c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov 
(cherry picked from commit 10f9f5101c)
 2014-11-06 15:12:06 +00:00
Zhijie Shen c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 
(cherry picked from commit ed63b11646)
 2014-11-01 00:49:38 -07:00
Vinod Kumar Vavilapalli 063bb0508b MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen. 
(cherry picked from commit 3f1441245d)
 2014-10-29 09:58:50 -07:00
Zhijie Shen 52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 971e91c8c0)
 2014-10-27 21:08:38 -07:00
Zhijie Shen 733380fa70 MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 6b2f11b54b)
 2014-10-27 20:41:02 -07:00
Zhijie Shen 3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be)
 2014-10-23 22:03:30 -07:00
Jian He f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe e17e3bdbc4 MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai 
(cherry picked from commit bbe80cdc7b)
 2014-10-13 15:09:26 +00:00
Karthik Kambatla 7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
 2014-10-11 22:49:46 -07:00
Jason Lowe 979b915ecc MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 
(cherry picked from commit 2e789eb226)
 2014-10-07 14:11:48 +00:00
Zhijie Shen b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Jason Lowe f4534746fb MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs. Contributed by Gera Shegalov 
(cherry picked from commit 7039b98e1c)
 2014-09-22 15:22:58 +00:00
Zhijie Shen 9333ee3068 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He. 
(cherry picked from commit 6fe5c6b746)
 2014-09-18 22:02:53 -07:00
Vinod Kumar Vavilapalli 5cdb24d4b1 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 
(cherry picked from commit 14e2639fd0)
 2014-09-13 18:05:02 -07:00
Jason Lowe 0b3f918b3b MAPREDUCE-6071. JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 45efc966ee)
 2014-09-05 13:48:16 +00:00
Chris Douglas 4f95017602 Fix typos in log messages. Contributed by Ray Chiang 2014-08-28 16:30:13 -07:00
Zhijie Shen 7cad25ec94 MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir creation on Windows. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1619863 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619864 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 17:08:17 +00:00
Jason Darrell Lowe e06838038e svn merge -c 1618691 FIXES: MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2. Contributed by chang li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618692 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:19:09 +00:00
Zhijie Shen 570183a437 MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before concluding the failure by configuration. Contributed by Yunjiong Zhao. 
svn merge --ignore-ancestry -c 1618677 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618679 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 17:59:32 +00:00
Eric Yang 0b20086da1 MAPREDUCE-6033. Updated access check for displaying job information 
(Yu Gao via Eric Yang)



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618649 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 16:14:27 +00:00
Zhijie Shen a7a79a198b MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky. 
svn merge --ignore-ancestry -c 1618269 ../../trunk/
svn merge --ignore-ancestry -c 1618270 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618271 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 20:33:55 +00:00
Xuan Gong 4365c4530b YARN-2212 ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1616891 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:37:35 +00:00
Jason Darrell Lowe 24346da6b6 svn merge -c 1616018 FIXES: MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1616020 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 21:59:01 +00:00
Xuan Gong ecc257e384 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615010 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 21:29:01 +00:00
Zhijie Shen 8cde949397 MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process is shutting down. Contributed by Wangda Tan. 
svn merge --ignore-ancestry -c 1613743 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1613744 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 01:41:44 +00:00
Jason Darrell Lowe 7c44536678 svn merge -c 1612358 FIXES: MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612371 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 18:22:45 +00:00
Jian He 99805f5eac Merge r1611434 from trunk. MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving RM-restart. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1611436 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 19:01:27 +00:00
Karthik Kambatla 4c7db3c6a3 MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex. (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1611197 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 21:30:19 +00:00
Zhijie Shen bba266610c MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current attempt is the last retry. Contributed by Wangda Tan. 
svn merge --ignore-ancestry -c 1609649 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1609650 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 08:49:54 +00:00
Vinod Kumar Vavilapalli fd6f4f72e4 MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side resource configuration for deciding uber-mode on map-only jobs. Contributed by Siqi Li. 
svn merge --ignore-ancestry -c 1608595 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1608596 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:14:33 +00:00
Karthik Kambatla 9133e3142c Preparing for 2.6.0 development. Updated versions. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607541 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 06:11:57 +00:00
Zhijie Shen 38bb24538d MAPREDUCE-5900. Changed to the interpret container preemption exit code as a task attempt killing event. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1607512 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607513 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 01:45:53 +00:00
Vinod Kumar Vavilapalli a2e2c8ad97 YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
svn merge --ignore-ancestry -c 1605205 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605206 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:44:00 +00:00
Karthik Kambatla ff2886e909 MAPREDUCE-5844. Add a configurable delay to reducer-preemption. (Maysam Yabandeh via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1604032 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 21:13:14 +00:00
Jian He d3462755bb Merge r1603338 from trunk. MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event at COMMIT_PENDING state. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603339 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 00:53:23 +00:00
Junping Du 6d3bfa33c9 Merge r1594795 from trunk: MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not need to be volatile. (Contributed by Tsuyoshi OZAWA) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594797 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-15 07:10:42 +00:00
Jonathan Turner Eagles f8dad8a751 MAPREDUCE-5888. Failed job leaves hung AM after it unregisters (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594318 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 18:28:07 +00:00
Jason Darrell Lowe 3434d7b78b svn merge -c 1590668 FIXES: MAPREDUCE-5812. Make job context available to OutputCommitter.isRecoverySupported(). Contributed by Mohammad Kamrul Islam 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590675 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-28 15:27:01 +00:00
Eric Yang fef656527a MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob. 
(Jinghui Wang via Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590267 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-26 18:02:19 +00:00
Jason Darrell Lowe 2a76cade32 svn merge -c 1590168 FIXES: MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590174 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 22:34:44 +00:00
Jason Darrell Lowe aa3d6ed6ea svn merge -c 1590046 FIXES: MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590047 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 14:56:35 +00:00
Haohui Mai 1566fff15f MAPREDUCE-5837. Merge r1589903 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589904 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 22:55:25 +00:00
Jason Darrell Lowe c02ab95ee6 svn merge -c 1589524 FIXES: MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589526 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-23 21:57:55 +00:00
Chris Nauroth 2d8f62f5c1 MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589259 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-22 19:32:17 +00:00
Chris Nauroth f0d3664d83 MAPREDUCE-5852. Merging change r1589006 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589007 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:49:28 +00:00
Jason Darrell Lowe c8d5d1f82a svn merge -c 1588559 FIXES: MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly. Contributed by Eric Payne 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588561 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 20:40:00 +00:00
Vinod Kumar Vavilapalli 7eff6cc305 MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster gracefully handle empty-queue names. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1586559 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586560 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:02:19 +00:00
Jason Darrell Lowe 58aefafe76 svn merge -c 1586479 FIXES: MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586482 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 22:09:16 +00:00