Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,073 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1284 Commits

Author SHA1 Message Date
Zhijie Shen 3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be)
 2014-10-23 22:03:30 -07:00
Jian He 1c235a4448 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfb)
 2014-10-22 15:58:26 -07:00
Jason Lowe 8885b75d78 MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in client. Contributed by Rohith 
(cherry picked from commit 209b1699fc)
 2014-10-17 19:53:17 +00:00
Jian He f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe 025b6c13e9 MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov 
(cherry picked from commit f19771a24c)
 2014-10-15 17:51:53 +00:00
Jason Lowe 1179a0cb07 MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting for maps. Contributed by Siqi Li 
(cherry picked from commit b9edad6403)
 2014-10-15 15:54:57 +00:00
Jason Lowe 942fb67a04 MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk. Contributed by Binglin Chang 
(cherry picked from commit 793dbf2465)
 2014-10-13 16:05:29 +00:00
Jason Lowe e17e3bdbc4 MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai 
(cherry picked from commit bbe80cdc7b)
 2014-10-13 15:09:26 +00:00
Karthik Kambatla 7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
 2014-10-11 22:49:46 -07:00
cnauroth f6278cbac5 MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit 777877d079)
 2014-10-09 22:38:25 -07:00
cnauroth 85a2b90740 MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 684170d5ef)
 2014-10-09 22:27:40 -07:00
Jason Lowe 979b915ecc MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 
(cherry picked from commit 2e789eb226)
 2014-10-07 14:11:48 +00:00
subru 30a370e705 MAPREDUCE-6103.Adding reservation APIs to MR resource manager delegate. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit aa92dd45f2)
(cherry picked from commit 3f282762d1)
 2014-10-06 10:21:06 -07:00
Jason Lowe 7b8f2f5b1d MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk. Contributed by Akira AJISAKA 
(cherry picked from commit 23fbfe36c3)
 2014-09-29 18:14:11 +00:00
Jason Lowe 0cfff28f57 HADOOP-11049. javax package system class default is too broad. Contributed by Sangjin Lee 
(cherry picked from commit cf4631d7c9)
 2014-09-29 14:37:28 +00:00
Jian He a6879b606e MAPREDUCE-6087. Fixed wrong config name of MRJobConfig#MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS. Contributed by Akira AJISAKA 
(cherry picked from commit 5f16c98ad6)
 2014-09-26 18:06:19 -07:00
Allen Wittenauer 67d6fc3f01 MAPREDUCE-5945. Update the description of GenericOptionsParser -jt option (Akira AJISAKA via aw) 2014-09-26 14:26:36 -07:00
Allen Wittenauer e5d284664b MAPREDUCE-5796. Use current version of the archive name in DistributedCacheDeploy document (Akira AJISAKA via aw) 2014-09-26 12:59:59 -07:00
Allen Wittenauer fe8222998f MAPREDUCE-6073. Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags (Tsuyoshi OZAWA via aw) 2014-09-26 11:17:11 -07:00
Zhijie Shen 7b3ecc547f MAPREDUCE-5831. Make MR client ignore unknown counters received from AM. Contributed by Junping Du. 
(cherry picked from commit 662fc11ae7)
 2014-09-25 23:46:57 -07:00
Allen Wittenauer 7ffb636a54 MAPREDUCE-6093. minor distcp doc edits (Charles Lamb via aw) 2014-09-25 11:33:01 -07:00
Jason Lowe d5d9fd3255 MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2. Contributed by Mit Desai 
(cherry picked from commit 7af4c3888b)
 2014-09-24 16:10:28 +00:00
Zhijie Shen b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Jason Lowe f4534746fb MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs. Contributed by Gera Shegalov 
(cherry picked from commit 7039b98e1c)
 2014-09-22 15:22:58 +00:00
Allen Wittenauer 0037b8f67b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:08:04 -07:00
Jason Lowe bdb864e5e1 MAPREDUCE-6091. YARNRunner.getJobStatus() fails with ApplicationNotFoundException if the job rolled off the RM view. Contributed by Sangjin Lee 
(cherry picked from commit 951847ba94)
 2014-09-19 20:17:44 +00:00
Zhijie Shen 9333ee3068 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He. 
(cherry picked from commit 6fe5c6b746)
 2014-09-18 22:02:53 -07:00
Karthik Kambatla 372bf54072 MAPREDUCE-6086. mapreduce.job.credentials.binary should allow all URIs. (Zhihai Xu via kasha) 
(cherry picked from commit 52945a33cc)
 2014-09-18 15:47:21 -07:00
Jason Lowe 44c7113133 MAPREDUCE-5891. Improved shuffle error handling across NM restarts. Contributed by Junping Du 
(cherry picked from commit 2c3da25fd7)
 2014-09-18 22:01:42 +00:00
Andrew Wang 5bc71071d1 HADOOP-11106. Document considerations of HAR and Encryption. (clamb via wang) 
(cherry picked from commit fe2f54d4cb)
 2014-09-18 14:51:27 -07:00
Jason Lowe 4b4e44a8aa MAPREDUCE-6090. mapred hsadmin getGroups fails to connect in some cases. Contributed by Robert Kanter 
(cherry picked from commit 1cf3198047)
 2014-09-18 21:02:13 +00:00
Vinod Kumar Vavilapalli 5cdb24d4b1 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 
(cherry picked from commit 14e2639fd0)
 2014-09-13 18:05:02 -07:00
Jason Lowe 6f8abe7efe MAPREDUCE-6070. yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 9e28e089c6)
 2014-09-11 16:20:51 +00:00
Jason Lowe e6f108c99b MAPREDUCE-6075. HistoryServerFileSystemStateStore can create zero-length files. Contributed by Jason Lowe 
(cherry picked from commit 17a025d9dda9949f191b650e3ef96090880f568a)
 2014-09-10 22:12:24 +00:00
Jason Lowe 0b3f918b3b MAPREDUCE-6071. JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 45efc966ee)
 2014-09-05 13:48:16 +00:00
Chris Douglas 69ee11dc93 MAPREDUCE-6063. Correct spill size calculation for spills wrapping the circular buffer. Contributed by zhihai xu. 2014-09-03 17:02:17 -07:00
Andrew Wang 075bb9e380 HADOOP-11021. Configurable replication factor in the hadoop archive command. Contributed by Zhe Zhang. 
(cherry picked from commit ea1c6f31c2)
 2014-08-29 14:47:28 -07:00
Jason Lowe 27086f5942 MAPREDUCE-5931. Validate SleepJob command line parameters. Contributed by Gera Shegalov 
(cherry picked from commit 15366d9227)
 2014-08-29 19:52:10 +00:00
Chris Douglas 4f95017602 Fix typos in log messages. Contributed by Ray Chiang 2014-08-28 16:30:13 -07:00
Alejandro Abdelnur c77bd85b62 HDFS-6134 and HADOOP-10150 subtasks. 2014-08-28 15:03:08 -07:00
Jason Lowe 38df629367 MAPREDUCE-5885. build/test/test.mapred.spill causes release audit warnings. Contributed by Chen He 
(cherry picked from commit 812bd0c0e5)
 2014-08-27 15:22:06 +00:00
Sanford Ryza c38e751bca MAPREDUCE-5130. Fix mapred-default.xml after bad merge. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619930 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 20:58:12 +00:00
Zhijie Shen 7cad25ec94 MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir creation on Windows. Contributed by Zhijie Shen. 
svn merge --ignore-ancestry -c 1619863 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619864 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-22 17:08:17 +00:00
Sanford Ryza 53da85efaf MAPREDUCE-5130. Add missing job config options to mapred-default.xml (Ray Chiang via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619631 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 23:54:38 +00:00
Jason Darrell Lowe 23b4c0b6c1 HADOOP-10893. isolated classloader on the client side. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619605 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 21:41:24 +00:00
Karthik Kambatla c9d7647826 MAPREDUCE-5974. Allow specifying multiple MapOutputCollectors with fallback. (Todd Lipcon via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619496 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 17:39:54 +00:00
Karthik Kambatla c9702526a0 MAPREDUCE-6012. DBInputSplit creates invalid ranges on Oracle. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618696 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:33:59 +00:00
Jason Darrell Lowe e06838038e svn merge -c 1618691 FIXES: MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2. Contributed by chang li 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618692 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:19:09 +00:00
Zhijie Shen 570183a437 MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before concluding the failure by configuration. Contributed by Yunjiong Zhao. 
svn merge --ignore-ancestry -c 1618677 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618679 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 17:59:32 +00:00
Eric Yang 0b20086da1 MAPREDUCE-6033. Updated access check for displaying job information 
(Yu Gao via Eric Yang)



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618649 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 16:14:27 +00:00
Zhijie Shen a7a79a198b MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky. 
svn merge --ignore-ancestry -c 1618269 ../../trunk/
svn merge --ignore-ancestry -c 1618270 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618271 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-15 20:33:55 +00:00
Jason Darrell Lowe c66756ebc1 svn merge -c 1618080 FIXES: MAPREDUCE-5878. some standard JDK APIs are not part of system classes defaults. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618081 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 23:18:45 +00:00
Allen Wittenauer d171c87a7a MAPREDUCE-5906. Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent" (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618008 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 18:07:50 +00:00
Allen Wittenauer a43fd59ba3 MAPREDUCE-5999. Fix dead link in InputFormat javadoc (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618005 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 17:57:36 +00:00
Allen Wittenauer eb11b4b2a7 MAPREDUCE-5998. CompositeInputFormat javadoc is broken (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617997 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 17:18:49 +00:00
Allen Wittenauer 8d1c74766b MAPREDUCE-5950. incorrect description in distcp2 document (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617993 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 17:11:30 +00:00
Jason Darrell Lowe 3755f527a8 svn merge -c 1617984 FIXES: MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617987 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 16:08:53 +00:00
Allen Wittenauer 90478326fc MAPREDUCE-5597. Missing alternatives in javadocs for deprecated constructors in mapreduce.Job (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617985 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:57:42 +00:00
Allen Wittenauer 9a07d2c02c MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617983 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:53:24 +00:00
Allen Wittenauer 89b41b5ce9 MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617981 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:48:31 +00:00
Allen Wittenauer eeb8ff9688 MAPREDUCE-4791. Javadoc for KeyValueTextInputFormat should include default separator and how to change it (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617980 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:44:54 +00:00
Allen Wittenauer 0cac0fafc3 MAPREDUCE-883. harchive: Document how to unarchive (Akira AJISAKA and Koji Noguchi via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617978 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 15:40:31 +00:00
Allen Wittenauer 93e518be3a MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617807 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-13 19:36:08 +00:00
Xuan Gong 4365c4530b YARN-2212 ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1616891 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 21:37:35 +00:00
Jason Darrell Lowe 16c4bd1a96 svn merge -c 1616585 FIXES: MAPREDUCE-6021. MR AM should have working directory in LD_LIBRARY_PATH. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1616591 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 20:29:48 +00:00
Jason Darrell Lowe 24346da6b6 svn merge -c 1616018 FIXES: MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1616020 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-05 21:59:01 +00:00
Xuan Gong ecc257e384 YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik and Arpit Agarwal 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615010 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 21:29:01 +00:00
Zhijie Shen f52092be46 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du. 
svn merge --ignore-ancestry -c 1614838 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1614839 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 09:31:22 +00:00
Zhijie Shen 8cde949397 MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process is shutting down. Contributed by Wangda Tan. 
svn merge --ignore-ancestry -c 1613743 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1613744 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 01:41:44 +00:00
Jason Darrell Lowe a1e6be744c svn merge -c 1612652 FIXES: MAPREDUCE-5963. ShuffleHandler DB schema should be versioned with compatible/incompatible changes. Contributed by Junping Du 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612654 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-22 19:38:11 +00:00
Jason Darrell Lowe 0e2fee873d svn merge -c 1612400 FIXES: MAPREDUCE-5756. CombineFileInputFormat.getSplits() including directories in its results. Contributed by Jason Dere 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612402 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 21:30:33 +00:00
Jason Darrell Lowe 7c44536678 svn merge -c 1612358 FIXES: MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612371 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-21 18:22:45 +00:00
Jian He 99805f5eac Merge r1611434 from trunk. MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving RM-restart. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1611436 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 19:01:27 +00:00
Karthik Kambatla 4c7db3c6a3 MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex. (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1611197 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 21:30:19 +00:00
Vinod Kumar Vavilapalli 2c2c589700 MAPREDUCE-5790. Made it easier to enable hprof profile options by default. Contributed by Gera Shegalov. 
svn merge --ignore-ancestry -c 1610578 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1610579 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-15 04:49:31 +00:00
Zhijie Shen bba266610c MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current attempt is the last retry. Contributed by Wangda Tan. 
svn merge --ignore-ancestry -c 1609649 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1609650 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-11 08:49:54 +00:00
Vinod Kumar Vavilapalli fd6f4f72e4 MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side resource configuration for deciding uber-mode on map-only jobs. Contributed by Siqi Li. 
svn merge --ignore-ancestry -c 1608595 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1608596 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:14:33 +00:00
Chris Nauroth 00ff3d320a MAPREDUCE-5866. Merging change r1608585 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1608588 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 20:00:38 +00:00
Vinod Kumar Vavilapalli 29099c8a5c MAPREDUCE-5868. Fixed an issue with TestPipeApplication that was causing the nightly builds to fail. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1608579 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1608580 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-07 19:46:31 +00:00
Karthik Kambatla 9133e3142c Preparing for 2.6.0 development. Updated versions. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607541 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 06:11:57 +00:00
Zhijie Shen 38bb24538d MAPREDUCE-5900. Changed to the interpret container preemption exit code as a task attempt killing event. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1607512 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607513 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 01:45:53 +00:00
Jason Darrell Lowe a7e8f79eb5 svn merge -c 1605892 FIXES: MAPREDUCE-5939. StartTime showing up as the epoch time in JHS UI after upgrade. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605895 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-26 19:57:31 +00:00
Vinod Kumar Vavilapalli a2e2c8ad97 YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
svn merge --ignore-ancestry -c 1605205 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605206 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:44:00 +00:00
Vinod Kumar Vavilapalli 5f0a84d919 MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for binary compatibility with older clients like Hive 0.13. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1604230 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1604231 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 18:28:51 +00:00
Karthik Kambatla ff2886e909 MAPREDUCE-5844. Add a configurable delay to reducer-preemption. (Maysam Yabandeh via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1604032 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-19 21:13:14 +00:00
Karthik Kambatla 1bbdc43f92 MAPREDUCE-5896. InputSplits should indicate which locations have the block cached in memory. (Sandy Ryza via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603673 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 23:32:36 +00:00
Jian He d3462755bb Merge r1603338 from trunk. MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event at COMMIT_PENDING state. Contributed by Zhijie Shen 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603339 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 00:53:23 +00:00
Karthik Kambatla c43ab7f08e YARN-2155. FairScheduler: Incorrect threshold check for preemption. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1602297 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 21:26:39 +00:00
Uma Maheswara Rao G f089311aba Merge from trunk to branch-2. MAPREDUCE-5920. Add Xattr option in DistCp docs. Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1602119 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-12 11:07:22 +00:00
Karthik Kambatla 3ed429bb62 MAPREDUCE-5777. Support utf-8 text with Byte Order Marker. (Zhihai Xu via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1600979 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 18:39:28 +00:00
Chris Nauroth 7895ecccac HADOOP-10602. Merging change r1598337 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598338 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 17:33:10 +00:00
Devarajulu K 99c0324937 MAPREDUCE-5895. Close streams properly to avoid leakage in TaskLog. Contributed by Kousuke Saruta. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598210 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 07:06:17 +00:00
Jason Darrell Lowe 3045fdc7c6 svn merge -c 1598111 FIXES: MAPREDUCE-5862. Line records longer than 2x split size aren't handled correctly. Contributed by bc Wong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598115 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-28 19:53:08 +00:00
Jason Darrell Lowe fd6d5121e3 svn merge -c 1596295 FIXES: MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job history files generated by 2.0.3 history server. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1596298 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-20 15:52:25 +00:00
Jason Darrell Lowe b6988521f4 Reverted HADOOP-10474 from branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1596048 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-19 20:31:40 +00:00
Ravi Prakash 2125caaa2f MAPREDUCE 5874. Creating MapReduce REST API section. Contributed by Tsuyoshi Ozawa 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1595268 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 17:11:01 +00:00
Junping Du 6d3bfa33c9 Merge r1594795 from trunk: MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not need to be volatile. (Contributed by Tsuyoshi OZAWA) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594797 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-15 07:10:42 +00:00
Christopher Douglas 28301a6b79 MAPREDUCE-5821. Avoid unintentional reallocation of byte arrays in segments 
during merge. Contributed by Todd Lipcon


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594655 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-14 17:51:21 +00:00
Karthik Kambatla 96d4036449 MAPREDUCE-5652. NM Recovery. ShuffleHandler should handle NM restarts. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594332 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 19:14:07 +00:00
Jason Darrell Lowe 890cd33bf7 svn merge -c 1594326 FIXES: MAPREDUCE-5814. fat jar with *-default.xml may fail when mapreduce.job.classloader=true. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594328 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 18:59:56 +00:00
Jonathan Turner Eagles f8dad8a751 MAPREDUCE-5888. Failed job leaves hung AM after it unregisters (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594318 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-13 18:28:07 +00:00
Karthik Kambatla 7fe885d83f MAPREDUCE-5774. Job overview in History UI should list reducer phases in chronological order. (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1593891 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-12 06:16:24 +00:00
Jason Darrell Lowe 3019b2c7f0 svn merge -c 1593422 FIXES: MAPREDUCE-5884. History server uses short user name when canceling tokens. Contributed by Mohammad Kamrul Islam 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1593424 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-08 21:28:43 +00:00
Jonathan Turner Eagles 7ac359f389 MAPREDUCE-5636. Convert MapReduce Tutorial document to APT (Akira AJISAKA via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1592796 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-06 16:18:13 +00:00
Jonathan Turner Eagles 72feaf0436 MAPREDUCE-5637. Convert Hadoop Streaming document to APT (Akira AJISAKA via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1592792 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-06 16:05:37 +00:00
Jonathan Turner Eagles 73e0b8e55a MAPREDUCE-5638. Port Hadoop Archives document to trunk (Akira AJISAKA via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1591108 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-29 21:27:14 +00:00
Jason Darrell Lowe 3434d7b78b svn merge -c 1590668 FIXES: MAPREDUCE-5812. Make job context available to OutputCommitter.isRecoverySupported(). Contributed by Mohammad Kamrul Islam 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590675 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-28 15:27:01 +00:00
Eric Yang fef656527a MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob. 
(Jinghui Wang via Eric Yang)


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590267 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-26 18:02:19 +00:00
Jason Darrell Lowe 2a76cade32 svn merge -c 1590168 FIXES: MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state. Contributed by Ming Ma 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590174 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 22:34:44 +00:00
Jonathan Turner Eagles e0cd544a53 MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590131 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 20:32:09 +00:00
Jason Darrell Lowe aa3d6ed6ea svn merge -c 1590046 FIXES: MAPREDUCE-5749. TestRMContainerAllocator#testReportedAppProgress Failed. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1590047 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 14:56:35 +00:00
Haohui Mai 1566fff15f MAPREDUCE-5837. Merge r1589903 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589904 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 22:55:25 +00:00
Vinod Kumar Vavilapalli b04bb049d2 MAPREDUCE-5843. Fixed TestMRKeyValueTextInputFormat to not leak files and thus avoid failing on Windows. Contributed by Varun Vasudev. 
svn merge --ignore-ancestry -c 1589533 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589534 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-23 22:54:55 +00:00
Jason Darrell Lowe c02ab95ee6 svn merge -c 1589524 FIXES: MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589526 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-23 21:57:55 +00:00
Vinod Kumar Vavilapalli c5450fd0fe MAPREDUCE-5832. Fixed TestJobClient to not fail on JDK7 or on Windows. Contributed by Jian He and Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1589315 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589316 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-23 00:02:46 +00:00
Chris Nauroth 2d8f62f5c1 MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589259 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-22 19:32:17 +00:00
Chris Nauroth d87463514a MAPREDUCE-5827. Merging change r1589223 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589236 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-22 18:22:26 +00:00
Chris Nauroth f0d3664d83 MAPREDUCE-5852. Merging change r1589006 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1589007 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:49:28 +00:00
Chris Nauroth a7a7a16673 MAPREDUCE-5642. Merging change r1588605 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588606 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-19 04:21:01 +00:00
Jason Darrell Lowe c8d5d1f82a svn merge -c 1588559 FIXES: MAPREDUCE-4937. MR AM handles an oversized split metainfo file poorly. Contributed by Eric Payne 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588561 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-18 20:40:00 +00:00
Chris Nauroth 5d90eaa20d HADOOP-10499. Merging change r1588098 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588099 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 23:16:36 +00:00
Jonathan Turner Eagles 327efd2672 MAPREDUCE-3191. docs for map output compression incorrectly reference SequenceFile (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588010 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 18:16:58 +00:00
Devarajulu K 8f0bc218b6 MAPREDUCE-5775. Remove unnecessary job.setNumReduceTasks in SleepJob.createJob. Contributed by jhanver chand sharma. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587832 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-16 08:33:36 +00:00
Jonathan Turner Eagles 53ab439204 MAPREDUCE-5836. Fix typo in RandomTextWriter (Akira AJISAKA via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587722 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-15 20:46:30 +00:00
Vinod Kumar Vavilapalli 982ab03fbb MAPREDUCE-5828. Fixed a test issue with TestMapReduceJobControl that was causing it to fail on Windows. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1587089 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587090 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-13 19:55:57 +00:00
Vinod Kumar Vavilapalli b92a4cf472 MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions on Windows for temporary files and thus also fix the test-issue with 
TestHistoryServerFileSystemStateStoreService. Contributed by Varun Vasudev.
svn merge --ignore-ancestry -c 1586567 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586568 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:29:28 +00:00
Vinod Kumar Vavilapalli 7eff6cc305 MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster gracefully handle empty-queue names. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1586559 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586560 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-11 04:02:19 +00:00
Jason Darrell Lowe 58aefafe76 svn merge -c 1586479 FIXES: MAPREDUCE-5825. Provide diagnostics for reducers killed during ramp down. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586482 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 22:09:16 +00:00
Vinod Kumar Vavilapalli ec907fef0e MAPREDUCE-5824. Fixed test-failure of TestPipesNonJavaInputFormat in Windows. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1586368 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586369 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-10 16:44:31 +00:00
Steve Loughran 770aef5053 HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585933 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 11:11:31 +00:00
Haohui Mai 56268ffc41 HADOOP-10474. Merge r1585886 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585887 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 04:10:23 +00:00
Sanford Ryza 96106ff46e MAPREDUCE-5759. Remove unnecessary conf load in Limits (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583498 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-01 00:17:18 +00:00
Arun Murthy e052a4579f Merge -c 1583050 from trunk to branch-2 to fix MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the local-filesystem, rather than rely on the classpath. Contributed by Gera Shegalov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1583051 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-29 19:16:03 +00:00
Vinod Kumar Vavilapalli 039162818b MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka. 
svn merge --ignore-ancestry -c 1582125 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1582127 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 23:44:17 +00:00
Vinod Kumar Vavilapalli e51fbb6a9e MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it recovers from a commit during a previous attempt. Contributed by Xuan Gong. 
svn merge --ignore-ancestry -c 1581180 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1581181 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-25 02:01:13 +00:00
Chris Nauroth c1e93e1f90 MAPREDUCE-5791. Merging change r1580994 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580996 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-24 19:03:09 +00:00
Vinod Kumar Vavilapalli 751bba3d14 MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. Contributed by Varun Vasudev. 
svn merge --ignore-ancestry -c 1580100 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580101 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-22 00:33:21 +00:00
Vinod Kumar Vavilapalli 444af8fac5 MAPREDUCE-5787. Added the ability to keep alive shuffle connections in the MapReduce shuffle-handler. Contributed by Rajesh Balamohan. 
svn merge --ignore-ancestry -c 1580062 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580063 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 21:44:24 +00:00
Kihwal Lee 2c3feeb727 svn merge -c 1579995 merging from trunk to branch-2 to fix:MAPREDUCE-5804. TestMRJobsWithProfiler#testProfiler timesout. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579998 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-21 17:58:33 +00:00
Vinod Kumar Vavilapalli 31a3a99f05 MAPREDUCE-2349. Modified FileInputFormat to be able to issue file and block location calls in parallel. Contributed by Siddharth Seth. 
svn merge --ignore-ancestry -c 1579515 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1579516 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-20 02:48:48 +00:00
Jing Zhao d611f42c37 Merge r1578669 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578676 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-18 00:10:22 +00:00
Vinod Kumar Vavilapalli d470c7b71a YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. 
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-16 19:13:16 +00:00
Jason Darrell Lowe c7a6775d94 svn merge -c 1577692 FIXES: MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577696 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 20:37:40 +00:00
Jason Darrell Lowe fd118493c6 svn merge -c 1577647 FIXES: MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM. Contributed by Rohith 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577648 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 18:20:04 +00:00
Jason Darrell Lowe 838b4d3f84 MAPREDUCE-5751. MR app master fails to start in some cases if mapreduce.job.classloader is true. Contributed by Sangjin Lee 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577555 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 14:47:37 +00:00
Christopher Douglas 9695bc7af6 YARN-1771. Reduce the number of NameNode operations during localization of 
public resources using a cache. Contributed by Sangjin Lee



git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577392 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-14 00:31:08 +00:00
Jonathan Turner Eagles d2990f85b3 MAPREDUCE-5456. TestFetcher.testCopyFromHostExtraBytes is missing (Jason Lowe via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577318 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 20:58:27 +00:00
Arpit Agarwal fd4d1d1d05 MAPREDUCE-5794. Merging r1577281 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577283 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 18:57:01 +00:00
Jonathan Turner Eagles eaf54d5a10 MAPREDUCE-5713. InputFormat and JobConf JavaDoc Fixes (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577240 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 16:50:13 +00:00
Jason Darrell Lowe 7f5a157995 svn merge -c 1577202 FIXES: MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page. Contributed by Rushabh S Shah 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577207 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 15:42:51 +00:00
Zhijie Shen 7834c7f193 YARN-1389. Made ApplicationClientProtocol and ApplicationHistoryProtocol expose analogous getApplication(s)/Attempt(s)/Container(s) APIs. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1577052 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577053 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 07:45:30 +00:00
Arun Murthy 9e695db974 Merge -c 1576946 from trunk to branch-2 to fix MAPREDUCE-5553. Allow users to easily access completed/pending/successful/failed tasks on MR AM web-ui. Contributed by Paul Han. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576947 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 22:05:54 +00:00
Jason Darrell Lowe 804f86f42c svn merge -c 1576940 FIXES: MAPREDUCE-5778. JobSummary does not escape newlines in the job name. Contributed by Akira AJISAKA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576943 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-12 21:59:50 +00:00
Vinod Kumar Vavilapalli ad5c0e4613 MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail when a large value of io.sort.mb is set. Contributed by Karthik Kambatla. 
svn merge --ignore-ancestry -c 1576170 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576171 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-11 01:30:20 +00:00
Tsz-wo Sze eb4e72d190 svn merge -c 1575049 from trunk for MAPREDUCE-5780. SliveTest should use the specified path to get the particular FileSystem instead of using the default FileSystem. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575055 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-06 21:40:31 +00:00
Haohui Mai 39b70d4e1f Merge r1574792 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574794 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-06 08:16:12 +00:00
Vinod Kumar Vavilapalli 7c17cff55c YARN-986. Changed client side to be able to figure out the right RM Delegation token for the right ResourceManager when HA is enabled. Contributed by Karthik Kambatla. 
svn merge --ignore-ancestry -c 1574190 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1574191 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-04 20:39:33 +00:00
Jason Darrell Lowe 8da70ca8e0 svn merge -c 1573775 FIXES: MAPREDUCE-5773. Provide dedicated MRAppMaster syslog length limit. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573778 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-03 22:12:18 +00:00
Jason Darrell Lowe 4d54763a07 svn merge -c 1573035 FIXES: MAPREDUCE-5768. TestMRJobs.testContainerRollingLog fails on trunk. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1573037 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 19:28:03 +00:00
Karthik Kambatla d00de6418a MAPREDUCE-5665. Add audience annotations to MiniMRYarnCluster and MiniMRCluster. (Anubhav Dhoot via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572796 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-28 00:53:23 +00:00
Vinod Kumar Vavilapalli 04474afc81 MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1572711 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572712 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 19:24:49 +00:00
Vinod Kumar Vavilapalli c5af93cd47 MAPREDUCE-5766. Moved ping messages from TaskAttempts to be at DEBUG level inside the ApplicationMaster log. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572381 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-27 01:30:07 +00:00
Jason Darrell Lowe 58d84af922 svn merge -c 1572269 FIXES: Preserve Job diagnostics in history. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1572276 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-26 21:42:47 +00:00
Vinod Kumar Vavilapalli a6529b363c MAPREDUCE-5761. Added a simple log message to denote when encrypted shuffle is on in the shuffle-handler. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1571514 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1571515 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-25 01:24:50 +00:00
Jonathan Turner Eagles 555d4b513f MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1570635 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-21 16:53:15 +00:00
Vinod Kumar Vavilapalli eeae0cd829 YARN-1590. Fixed ResourceManager, web-app proxy and MR JobHistoryServer to expand _HOST properly in their kerberos principles. Contributed by Mohammad Kamrul Islam. 
svn merge --ignore-ancestry -c 1569537 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569539 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 21:51:06 +00:00
Arun Murthy 1ccccb8534 Branching for hadoop-2.4 releases, also set version to 2.5.0-SNAPSHOT in branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569186 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-18 04:41:01 +00:00
Jason Darrell Lowe 3452748b21 svn merge -c 1569038 FIXES: MAPREDUCE-5757. ConcurrentModificationException in JobControl.toList. Contributed by Jason Lowe 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569043 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-17 16:25:55 +00:00
Vinod Kumar Vavilapalli 7e252cdc9d YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.
svn merge --ignore-ancestry -c 1568501 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1568503 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:32 +00:00
Jason Darrell Lowe 62cc4908c5 svn merge -c 1568118 FIXES: MAPREDUCE-5670. CombineFileRecordReader should report progress when moving to the next file. Contributed by Chen He 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1568126 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-13 23:21:38 +00:00
Karthik Kambatla 22a0f31d66 MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567669 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 16:08:02 +00:00
Sanford Ryza 4032e147e8 YARN-1497. Command line additions for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567485 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 01:22:02 +00:00
Sanford Ryza 81bff204c3 YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567479 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-12 01:00:08 +00:00
Karthik Kambatla d082b122f9 MAPREDUCE-5744. Job hangs because RMContainerAllocator.preemptReduce() violates the comparator contract (Gera Shegalov via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565479 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 23:31:04 +00:00
Karthik Kambatla fa43d8fb45 MAPREDUCE-5699. Allow setting tags on MR jobs (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565385 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 18:30:43 +00:00
Zhijie Shen 5afaafdb7f MAPREDUCE-5743. Fixed the test failure in TestRMContainerAllocator. Contributed by Vinod Kumar Vavilapalli. 
svn merge --ignore-ancestry -c 1565087 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565088 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-06 06:39:29 +00:00
Sanford Ryza 28b178f55e MAPREDUCE-5732. Report proper queue when job has been automatically placed (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562906 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 17:57:08 +00:00
Sanford Ryza be75f21b6f MAPREDUCE-5464. Add analogs of the SLOTS_MILLIS counters that jive with the YARN resource model (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562229 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 21:05:03 +00:00
Vinod Kumar Vavilapalli b7a3d0ae93 YARN-967. Added the client and CLI interfaces for obtaining ApplicationHistory data. Contributed by Mayank Bansal. 
svn merge --ignore-ancestry -c 1556747  ../YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1562202 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 20:00:38 +00:00
Suresh Srinivas afc40e8a66 HADOOP-10255. Merge 1561959 and 1561961 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1561966 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-28 07:47:37 +00:00
Karthik Kambatla 139b3a2ac4 MAPREDUCE-5723. MR AM container log can be truncated or empty. (Mohammad Kamrul Islam via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1560593 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 06:09:21 +00:00
Andrew Wang c4194399d3 Merge HDFS-4949 changes from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1560522 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-22 21:43:00 +00:00
Jason Darrell Lowe 033d60a28f svn merge -c 1560148 FIXES: MAPREDUCE-5693. Restore MRv1 behavior for log flush. Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1560149 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 19:27:57 +00:00
Sanford Ryza 8adbc39208 MAPREDUCE-5725. Make explicit that TestNetworkedJob relies on the Capacity Scheduler (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1559884 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-21 01:24:03 +00:00
Karthik Kambatla 57adfd42d4 MAPREDUCE-5729. mapred job -list throws NPE (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1559812 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-20 19:30:12 +00:00
Sanford Ryza 7c7f33a178 MAPREDUCE-5650. Job fails when hprof mapreduce.task.profile.map/reduce.params is specified (Gera Shegalov via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1559202 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-17 17:53:23 +00:00
Jason Darrell Lowe 0fa22b8120 svn merge -c 1558948 FIXES: MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558957 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 23:13:17 +00:00
Alejandro Abdelnur 0c84c76954 MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558853 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-16 17:12:31 +00:00
Chris Nauroth cf3ac1a59e YARN-1138. Merging change r1556978 from trunk to branch-2. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1556979 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-09 23:10:43 +00:00
Alejandro Abdelnur 9bf09c63f9 MAPREDUCE-3310. Addendum fixing javadocs warnings (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1556064 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 23:21:29 +00:00
Alejandro Abdelnur 47469c464c MAPREDUCE-3310. Custom grouping comparator cannot be set for Combiners (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1555969 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 18:37:40 +00:00
Karthik Kambatla 2bbd22bbd5 MAPREDUCE-5689. MRAppMaster does not preempt reducers when scheduled maps cannot be fulfilled. (lohit via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1555162 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-03 17:32:02 +00:00
Vinod Kumar Vavilapalli eb247009b2 MAPREDUCE-5685. Fixed a bug with JobContext getCacheFiles API inside the WrappedReducer class. Contributed by Yi Song. 
svn merge --ignore-ancestry -c 1554320 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1554321 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-30 21:15:13 +00:00
Vinod Kumar Vavilapalli 234b01e862 MAPREDUCE-5694. Fixed MR AppMaster to shutdown the LogManager so as to avoid losing syslog in some conditions. Contributed by Mohammad Kamrul Islam. 
svn merge --ignore-ancestry -c 1553879 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1553880 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 17:51:37 +00:00
Karthik Kambatla 875a0d55f8 YARN-1463. Tests should avoid starting http-server where possible or creates spnego keytab/principals (vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1553792 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-28 06:58:50 +00:00
Sanford Ryza 76185bf7a3 MAPREDUCE-5550. Task Status message (reporter.setStatus) not shown in UI with Hadoop 2.0 (Gera Shegalov via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1552809 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 21:03:55 +00:00
Sanford Ryza d7e028de1a MAPREDUCE-5692. Add explicit diagnostics when a task attempt is killed due to speculative execution (Gera Shegalov via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1552799 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-20 19:57:16 +00:00
Vinod Kumar Vavilapalli 0cf92b66c5 MAPREDUCE-5687. Correcting the previous commit by using the right patch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1552070 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-18 19:28:14 +00:00
Vinod Kumar Vavilapalli e89a93b85b MAPREDUCE-5687. Fixed failure in TestYARNRunner caused by YARN-1446. Contributed by Jian He. 
svn merge --ignore-ancestry -c 1551774 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551777 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-18 00:14:44 +00:00
Jing Zhao 88d09130e8 HDFS-5545. Merge change r1546151 from trunk. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551703 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 20:59:27 +00:00