Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,086 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

117 Commits

Author SHA1 Message Date
Arun Murthy fae75c2d7f MAPREDUCE-3721. Fixed a race in shuffle which caused reduces to hang. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1236041 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-26 05:40:29 +00:00
Vinod Kumar Vavilapalli 078ae89a47 MAPREDUCE-3714. Fixed EventFetcher and Fetcher threads to shut-down properly so that reducers don't hang in corner cases. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235545 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 23:18:05 +00:00
Vinod Kumar Vavilapalli dc615c312b MAPREDUCE-3710. Improved FileInputFormat to return better locality for the last split. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235510 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 21:29:39 +00:00
Mahadev Konar 27c56b02b3 MAPREDUCE-3646. Remove redundant URL info from "mapred job" output. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1235034 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-23 22:22:53 +00:00
Arun Murthy d05e6d2671 MAPREDUCE-2450. Fixed a corner case with interrupted communication threads leading to a long timeout in Task. Contributed by Rajesh Balamohan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1232314 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-17 06:10:50 +00:00
Vinod Kumar Vavilapalli 74697f2317 MAPREDUCE-3512. Batching JobHistory flushing to DFS so that we don't flush for every event slowing down AM. Contributed by Siddarth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230353 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-12 01:42:10 +00:00
Mahadev Konar c832b46d0b MAPREDUCE-3651. TestQueueManagerRefresh fails. (Thomas Graves via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1230292 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 22:32:56 +00:00
Mahadev Konar bc4b1f48d3 MAPREDUCE-3380. Token infrastructure for running clients which are not kerberos authenticated. (mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229855 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 01:50:10 +00:00
Vinod Kumar Vavilapalli 7712e70090 MAPREDUCE-3251. Network ACLs can prevent some clients to talk to MR AM. Improved the earlier patch to not to JobHistoryServer repeatedly. Contributed by Anupam Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229787 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 23:07:54 +00:00
Vinod Kumar Vavilapalli 44e0bb831b MAPREDUCE-3382. Enhanced MR AM to use a proxy to ping the job-end notification URL. Contributed by Ravi Prakash. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229736 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 21:08:34 +00:00
Vinod Kumar Vavilapalli 48150cddaf MAPREDUCE-3639. Fixed TokenCache to work with absent FileSystem canonical service-names. (Siddharth Seth via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229444 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 01:38:27 +00:00
Vinod Kumar Vavilapalli 428529b58f MAPREDUCE-3528. Fixed TaskHeartBeatHandler to use a new configuration for the thread loop interval separate from task-timeout configuration property. (Siddharth Seth via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229403 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-09 22:37:00 +00:00
Vinod Kumar Vavilapalli e8645636ce MAPREDUCE-3511. Removed a multitude of cloned/duplicate counters in the AM thereby reducing the AM heap size and preventing full GCs. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1229347 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-09 21:04:58 +00:00
Alejandro Abdelnur 6d45b9e31d HADOOP-7934. Normalize dependencies versions across all modules. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227787 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 19:46:42 +00:00
Siddharth Seth a2bcb867e1 MAPREDUCE-3615. Fix some ant test failures. (Contributed by Thomas Graves) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227741 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 18:47:59 +00:00
Siddharth Seth be285fc27f MAPREDUCE-3595. Add missing TestCounters#testCounterValue test from branch 1 to 0.23 (Contributed by Tom White) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227380 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-04 23:15:41 +00:00
Siddharth Seth f445eb31f1 MAPREDUCE-3529. TokenCache does not cache viewfs credentials correctly. (sseth) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227374 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-04 23:07:27 +00:00
Alejandro Abdelnur c27de4bd0b MAPREDUCE-1744. DistributedCache creates its own FileSytem instance when adding a file/archive to the path. (Dick King via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1227238 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-04 17:51:17 +00:00
Harsh J a9ffbdd0d7 MAPREDUCE-2944. Improve checking of input for JobClient.displayTasks() (XieXianshan via harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1225188 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-28 14:11:02 +00:00
Amar Kamat 264d3b7dd0 MAPREDUCE-3349. Log rack-name in JobHistory for unsuccessful tasks. (Devaraj K and Amar Kamat via amarrk) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221578 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-21 02:58:00 +00:00
Vinod Kumar Vavilapalli e7543b944c MAPREDUCE-3339. Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221523 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 23:27:02 +00:00
Arun Murthy 96247ead03 MAPREDUCE-3376. Fixed Task to ensure it passes reporter to combiners using old MR api. Contributed by Subroto Sanyal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1221501 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-20 22:01:13 +00:00
Arun Murthy a61a18cc09 MAPREDUCE-3563. Fixed LocalJobRunner to work correctly with new mapreduce apis. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1220996 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-19 23:07:17 +00:00
Siddharth Seth c4ff892520 MAPREDUCE-3422. Counter display names are not being picked up. Contributed by Jonathan Eagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1215031 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 03:10:31 +00:00
Mahadev Konar 739f8871f2 MAPREDUCE-3541. Fix broken TestJobQueueClient test. (Ravi Prakash via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1214421 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-14 19:45:59 +00:00
Vinod Kumar Vavilapalli b7ae5a6cb7 MAPREDUCE-3426. Fixed MR AM in uber mode to write map intermediate outputs in the correct directory to work properly in secure mode. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213987 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:35:11 +00:00
Vinod Kumar Vavilapalli 0ea8570be5 MAPREDUCE-2863. Support web services for YARN and MR components. (Thomas Graves via vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213975 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:05:56 +00:00
Thomas White 37b8cc3f19 MAPREDUCE-3542. Support "FileSystemCounter" legacy counter group name for compatibility. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213971 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 22:59:49 +00:00
Mahadev Konar 9b75b05163 MAPREDUCE-3328. mapred queue -list output inconsistent and missing child queues. (Ravi Prakash via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213504 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 23:56:26 +00:00
Mahadev Konar 14e6f1e796 MAPREDUCE-3518. mapred queue -info <queue> -showJobs throws NPE. (Jonathan Eagles via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213464 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 22:08:09 +00:00
Thomas White 6571d39a74 MAPREDUCE-3527. Fix minor API incompatibilities between 1.0 and 0.23. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1213350 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-12 18:29:20 +00:00
Arun Murthy a55599062c MAPREDUCE-3496. Fixed client to print queue acls in consistent order. Contributed by Jonathan Eagles. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1210738 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-06 01:34:03 +00:00
Alejandro Abdelnur f7eda5a8d3 MAPREDUCE-3500. MRJobConfig creates an LD_LIBRARY_PATH using the platform ARCH. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209813 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-03 01:07:39 +00:00
Thomas White 72e9ec7a45 MAPREDUCE-3479. JobClient#getJob cannot find local jobs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209791 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-03 00:16:30 +00:00
Siddharth Seth 102541b12b MAPREDUCE-3443. JobClient and Job should function in the context of the UGI which created them. (Contributed by Mahadev Konar) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1209231 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 20:39:37 +00:00
Mahadev Konar 94bf0dacba MAPREDUCE-3450. NM port info no longer available in JobHistory. (Siddharth Seth via mahadev) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1208327 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-30 08:43:00 +00:00
Thomas White be9c46a447 MAPREDUCE-3433. Finding counters by legacy group name returns empty counters. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1207755 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-29 05:07:47 +00:00
Arun Murthy f17ed541c7 MAPREDUCE-3265. Removed debug logs during job submission to LOG.debug to cut down noise. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1205628 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 22:04:33 +00:00
Vinod Kumar Vavilapalli 9c17410c50 MAPREDUCE-3370. Fixed MiniMRYarnCluster and related tests to not use a hard-coded path for the mr-app jar. Contributed by Ahmed Radwan. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200768 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-11 08:44:52 +00:00
Vinod Kumar Vavilapalli 200bb27557 MAPREDUCE-3392. Fixed Cluster's getDelegationToken's API to return null when there isn't a supported token. Contributed by John George. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200484 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 19:19:25 +00:00
Vinod Kumar Vavilapalli 15150cd997 MAPREDUCE-3280. Removed the unnecessary job user-name configuration in mapred-site.xml. (vinodkv) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1200441 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-10 17:28:16 +00:00
Thomas White 9a6da7979e MAPREDUCE-3344. o.a.h.mapreduce.Reducer since 0.21 blindly casts to ReduceContext.ValueIterator. Contributed by Brock Noland. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1198910 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-07 20:13:00 +00:00
Vinod Kumar Vavilapalli c27601fefe MAPREDUCE-3297. Moved log related components into yarn-common so that HistoryServer and clients can use them without depending on the yarn-server-nodemanager module. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196986 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 08:02:19 +00:00
Arun Murthy 0df4878033 MAPREDUCE-3337. Added missing license headers. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1196977 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-03 07:06:53 +00:00
Arun Murthy cfb6a9883d MAPREDUCE-3237. Move LocalJobRunner to hadoop-mapreduce-client-core. Contributed by Tom White. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195792 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 03:16:44 +00:00
Arun Murthy 5f7e991659 MAPREDUCE-3321. Disabled a few MR tests for 0.23. Contributed by Hitesh Shah. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195743 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-01 00:57:22 +00:00
Arun Murthy 9db078212f MAPREDUCE-3035. Fixed MR JobHistory to ensure rack information is present. Contributed by chakravarthy. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195575 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 17:27:42 +00:00
Vinod Kumar Vavilapalli 47a381e306 MAPREDUCE-3146. Added a MR specific command line to dump logs for a given TaskAttemptID. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1195349 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-31 06:42:06 +00:00
Thomas White cd77a1668f HADOOP-7763. Add top-level navigation to APT docs. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190703 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-29 00:16:16 +00:00
Vinod Kumar Vavilapalli 670fa24b48 MAPREDUCE-2989. Modified JobHistory to link to task and AM logs from the JobHistoryServer. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1190174 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-28 06:45:04 +00:00