Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,987 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1362 Commits

Author SHA1 Message Date
Tsuyoshi Ozawa fa9def5a2a MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy. 
(cherry picked from commit 9a3dda3d34)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduceTutorial.md
 2015-04-28 01:58:12 +09:00
Devaraj K 84ae26573c MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384)
 2015-04-27 15:02:44 +05:30
Gera Shegalov fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b46241)
 2015-04-24 17:38:16 -07:00
Vinod Kumar Vavilapalli 0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297)
 2015-04-23 11:25:30 -07:00
Tsuyoshi Ozawa 542d53a268 MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit e33594dd87)
 2015-04-23 03:22:02 +09:00
Gera Shegalov 6f06e3b7ea MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera) 
(cherry picked from commit 89ded89e86)
 2015-04-21 14:21:57 -07:00
Gera Shegalov 6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc)
 2015-04-21 14:21:56 -07:00
Robert Kanter 756c254293 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f0928)
 2015-04-20 14:14:49 -07:00
Jian He 42cf8e0554 YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang 
(cherry picked from commit bb6dde68f1)
 2015-04-16 19:44:32 -07:00
Akira Ajisaka 51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka 4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35.
 2015-04-11 12:37:44 +09:00
Jian He d3daf9665c YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev 2015-04-10 10:40:31 -07:00
Akira Ajisaka 3de0bf9a35 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
 2015-04-11 01:31:44 +09:00
Akira Ajisaka f603b9ccca MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina. 
(cherry picked from commit 83979e61a4)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-04-10 20:23:46 +09:00
Harsh J b817f0924d MAPREDUCE-6314. TestPipeApplication fails on trunk. Contributed by Varun Vasudev. 
(cherry picked from commit c91b2de68a)
 2015-04-10 14:03:37 +05:30
Robert Kanter 0fdd5c23c3 MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL (rchiang via rkanter) 
(cherry picked from commit c4986b2d00)
 2015-04-09 13:49:03 -07:00
Hitesh Shah 55b794e7fa YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe80)
 2015-04-08 14:14:06 -07:00
Gera Shegalov 3757e731e6 MAPREDUCE-4844. Counters / AbstractCounters have constant references not declared final. (Brahma Reddy Battula via gera) 
(cherry picked from commit a0795c1e81)
 2015-04-03 10:35:18 -07:00
Junping Du cacadea632 MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe. 
(cherry picked from commit eccb7d46ef)
 2015-04-02 12:15:12 -07:00
Vinod Kumar Vavilapalli defae1d399 Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter." 
This reverts commit 5358b83167.

(cherry picked from commit cc0a01c503)
 2015-03-30 14:52:36 -07:00
Vinod Kumar Vavilapalli cc130a033a MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter. 
(cherry picked from commit 5358b83167)
 2015-03-30 10:28:28 -07:00
Vinod Kumar Vavilapalli c5bc48946d YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f51)
 2015-03-30 10:11:12 -07:00
Harsh J 274db918c3 MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula. 
(cherry picked from commit f5432cb47dfacaa57faed32a29d00bbeff8a5385)
 2015-03-28 11:58:44 +05:30
Akira Ajisaka 853cc04668 MAPREDUCE-5762. Port MAPREDUCE-3223 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka) 2015-03-28 03:35:03 +09:00
Tsuyoshi Ozawa f0aa2a7466 MAPREDUCE-6294. Remove an extra parameter described in Javadoc of TockenCache. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 05499b1093)
 2015-03-28 00:08:56 +09:00
Akira Ajisaka 2e509908ee MAPREDUCE-6292. Use org.junit package instead of junit.framework in TestCombineFileInputFormat. (aajisaka) 
(cherry picked from commit c770df49b4)
 2015-03-25 19:01:38 +09:00
Tsuyoshi Ozawa 94eb8e5b3d MAPREDUCE-6285. ClientServiceDelegate should not retry upon AuthenticationException. Contributed by Jonathan Eagles. 
(cherry picked from commit 4170c99147)
 2015-03-25 00:57:08 +09:00
Devaraj K 943d9ee603 MAPREDUCE-6242. Progress report log is incredibly excessive in application 
master. Contributed by Varun Saxena.
 2015-03-23 22:48:00 +05:30
Harsh J 6e6e0e4a37 MAPREDUCE-5448. Addendum fix to remove deprecation warning by junit.Assert import in TestFileOutputCommitter. 
(cherry picked from commit 4cd54d9a29)
 2015-03-22 10:33:29 +05:30
Harsh J d43c0da7d7 MAPREDUCE-5190. Unnecessary condition test in RandomSampler. Contributed by Jingguo Yao. 
(cherry picked from commit 1d5c796d65)
 2015-03-22 10:04:19 +05:30
Harsh J a7f1c1b993 MAPREDUCE-5448. MapFileOutputFormat#getReaders bug with invisible files/folders. Contributed by Maysam Yabandeh. 
(cherry picked from commit b46c2bb51a)
 2015-03-22 09:46:54 +05:30
Harsh J 7c72c7f529 MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu. 
(cherry picked from commit 433542904a)
 2015-03-22 02:51:17 +05:30
Harsh J 418d078bc5 MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang. 
(cherry picked from commit e1e09052e8)
 2015-03-22 02:45:01 +05:30
Harsh J 97da36777f MAPREDUCE-6239. Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core. Contributed by Varun Saxena. 2015-03-22 02:20:11 +05:30
Robert Kanter 308460654a MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) 
(cherry picked from commit d4f7e2507f)
 2015-03-20 13:12:34 -07:00
Jason Lowe cabe676d67 MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for user-specified task output subdirs. Contributed by Gera Shegalov and Siqi Li 
(cherry picked from commit 91baca145a)
 2015-03-19 21:40:51 +00:00
Jason Lowe 24d8c6f355 MAPREDUCE-6277. Job can post multiple history files if attempt loses connection to the RM. Contributed by Chang Li 
(cherry picked from commit 30da99cbaf)
 2015-03-18 19:31:32 +00:00
Tsuyoshi Ozawa bb828565f8 MAPREDUCE-5755. MapTask.MapOutputBuffer#compare/swap should have @Override annotation. (ozawa) 
(cherry picked from commit bb243cea93)
 2015-03-17 14:55:33 +09:00
Harsh J 5e403e8b8d MAPREDUCE-6100. replace mapreduce.job.credentials.binary with MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY for better readability. Contributed by Zhihai Xu. 
(cherry picked from commit f222bde273)
 2015-03-17 11:08:32 +05:30
Harsh J 0e9f78dda2 MAPREDUCE-6105. nconsistent configuration in property mapreduce.reduce.shuffle.merge.percent. Contributed by Ray Chiang. 
(cherry picked from commit 26a23b11598b3757ed4973781890e2cd2fc270df)
 2015-03-17 02:27:33 +05:30
Harsh J 2684b0b440 MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration, for debug purposes. Contributed by Plamen Jeliazkov. 
(cherry picked from commit 571f75b468a4412be2178466cb5d3888b05cd076)
 2015-03-17 01:02:21 +05:30
Tsuyoshi Ozawa 10745fcecd MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu 
(cherry picked from commit 9d38520c8e)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-14 16:45:23 +09:00
Jonathan Eagles 44aedad5dd YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676ab)
 2015-03-13 12:05:21 -05:00
Haohui Mai e75e6c66ea HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki. 2015-03-11 14:23:44 -07:00
Gera Shegalov 6802e8fefc MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output files. (Siqi Li via gera) 
(cherry picked from commit aa92b764a7)
 2015-03-10 11:40:45 -07:00
Tsuyoshi Ozawa 482504befd MAPREDUCE-4742. Fix typo in nnbench#displayUsage. Contributed by Liang Xie. 
(cherry picked from commit 20b8ee1350)
 2015-03-11 00:41:37 +09:00
Junping Du dbcdcb0d3c Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne. 
(cherry picked from commit d39bc903a0)
 2015-03-10 06:23:24 -07:00
Tsuyoshi Ozawa b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Tsuyoshi Ozawa ac874764d5 MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 348208014b)
 2015-03-05 14:13:13 +09:00
Karthik Kambatla f4d6c5e337 MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha) 
(cherry picked from commit c66c3ac6bf)
 2015-03-04 14:47:06 -08:00
Junping Du a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de9402)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Tsuyoshi Ozawa b1e7f9c588 MAPREDUCE-6268. Fix typo in Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit 742f9d90c0)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md
 2015-03-03 16:24:27 +09:00
Akira Ajisaka f7a724ca9e HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki. 
(cherry picked from commit 9112f093cd)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-02-28 17:06:58 -08:00
Konstantin V Shvachko 9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Akira Ajisaka 8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7d)
 2015-02-26 15:06:26 -08:00
Akira Ajisaka 5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a67)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka 5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Wangda Tan cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 
(cherry picked from commit d49ae725d5)
 2015-02-19 11:01:47 -08:00
Plamen Jeliazkov 32d215dc89 MAPREDUCE-6228. Add truncate operation to SLive. Contributed by Plamen Jeliazkov. 2015-02-19 00:14:11 -08:00
Jason Lowe 386e62ace3 MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4981d082d4)
 2015-02-18 19:28:45 +00:00
Tsuyoshi Ozawa 9523b52da5 MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure conditions. Contributed by Devaraj K. 
(cherry picked from commit 7c782047c6)
 2015-02-18 15:46:06 +09:00
Junping Du 1d91daaae9 MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena 
(cherry picked from commit 814afa46ef)
 2015-02-16 09:39:20 -08:00
Tsuyoshi Ozawa 7011125778 MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R. 
(cherry picked from commit 3338f6d907)
 2015-02-15 10:10:11 +09:00
Tsuyoshi Ozawa 7386323735 MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi. 
(cherry picked from commit ba3c80a5ca)
 2015-02-13 16:11:16 +09:00
Tsuyoshi Ozawa 9d57392f5b MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu. 
(cherry picked from commit 9b0ba59b82)
 2015-02-13 02:28:19 +09:00
Tsuyoshi Ozawa 941a923514 MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. 
(cherry picked from commit ac8d52bf50)
 2015-02-12 20:37:14 +09:00
Tsuyoshi Ozawa 9129b3415c MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K. 
(cherry picked from commit b42d09eb62)
 2015-02-12 20:08:47 +09:00
Devaraj K a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead0)
 2015-02-12 13:47:19 +05:30
Tsuyoshi Ozawa b1aad1d941 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b)
 2015-02-10 03:53:15 +09:00
yliu 6933975143 MAPREDUCE-6227. DFSIO for truncate. (shv via yliu) 2015-02-08 02:42:10 +08:00
Robert Kanter 46a2027226 MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk (zxu via rkanter) 
(cherry picked from commit e2ee2ff7d7)
 2015-02-05 14:31:44 -08:00
Jason Lowe eb927a1d6b MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal 
(cherry picked from commit e1990ab427)
 2015-02-05 20:14:13 +00:00
Jason Lowe 2dc4af2b93 MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw) 
(cherry picked from commit fd57ab2002)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-02-05 16:40:09 +00:00
Akira Ajisaka ad4b243586 MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(cherry picked from commit cc6bbfceae)
 2015-02-04 17:46:39 -08:00
Akira Ajisaka 8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Robert Kanter a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b)
 2015-02-02 13:52:29 -08:00
Akira Ajisaka 1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110a)
 2015-01-30 02:50:46 +09:00
Jian He 43b3b43cea MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe 
(cherry picked from commit cff05bff1f)
 2015-01-28 15:52:11 -08:00
Allen Wittenauer 874e38d81f HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw) 2015-01-27 14:32:21 -08:00
Jason Lowe 2cf58ca5a6 MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe 
(cherry picked from commit 56b7ec71a6)
 2015-01-26 16:30:10 +00:00
Allen Wittenauer 683cc18bba HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw) 2015-01-21 08:46:19 -08:00
Akira Ajisaka 2e5eef9da2 MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo) 
(cherry picked from commit d336d13678)
 2015-01-14 17:43:25 +09:00
Xuan 0562f382fa MAPREDUCE-6173. Document the configuration of deploying MR over 
distributed cache with enabling wired encryption at the same time.
Contributed by Junping Du.

(cherry picked from commit 10ac5abb86)
 2015-01-13 10:08:24 -08:00
Brandon Li d847363821 HADOOP-9992. Modify the NN loadGenerator to optionally run as a MapReduce job. Contributed by Akshay Radia 
(cherry picked from commit 09d31bc630)
 2015-01-09 17:25:19 -08:00
Tsuyoshi Ozawa 132fd6ba58 HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa) 
(cherry picked from commit a6ed4894b5)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/java/org/apache/hadoop/mapred/nativetask/kvtest/KVJob.java
 2015-01-08 15:00:02 +09:00
Jason Lowe c22c27068e MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed by Jens Rabe 
(cherry picked from commit a0aeed100b)
 2015-01-06 23:40:42 +00:00
Jason Lowe 2b408d8dc7 HADOOP-11445. Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split. Contributed by Ankit Kamboj 
(cherry picked from commit d02fb53750)
 2015-01-06 21:20:31 +00:00
Harsh J 6d65e74670 MAPREDUCE-6149. Document override log4j.properties in MR job. Contributed by Junping Du. 
(cherry picked from commit 6621c3598e)
 2014-12-30 23:13:19 +05:30
Jian He 9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821)
 2014-12-19 16:56:55 -08:00
Robert Kanter f9341c1e2c MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter) 
(cherry picked from commit 390a7c12f5)
 2014-12-19 15:44:01 -08:00
Karthik Kambatla e6c73a1c1a HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle. (Wei Yan via kasha) 
(cherry picked from commit 6f1e3667cf)
 2014-12-19 15:07:39 -08:00
Robert Kanter 22b38cf29a MAPREDUCE-6045. need close the DataInputStream after open it in TestMapReduce.java (zxu via rkanter) 
(cherry picked from commit d9e4d67d18)
 2014-12-19 12:00:00 -08:00
Gera Shegalov 7c7bccfc31 MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8)
 2014-12-15 19:21:46 -08:00
Jian He d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9e)
 2014-12-15 11:08:44 -08:00
Harsh J a3fa1f0761 MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. 
(cherry picked from commit 298d09c9b5)
 2014-12-15 14:27:49 +05:30
Devaraj K 25f4a0193b MAPREDUCE-6046. Change the class name for logs in RMCommunicator. 
Contributed by Sahil Takiar.

(cherry picked from commit 0bd0229110)
 2014-12-12 11:44:00 +05:30
Gera Shegalov 8437c31b67 HADOOP-11211. mapreduce.job.classloader.system.classes semantics should be order-independent. (Yitong Zhou via gera) 
(cherry picked from commit 0bcea111e5)
 2014-12-11 13:29:40 -08:00
Harsh J 6e06a51e30 MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml. Contributed by James Carman. (harsh) 
(cherry picked from commit cb99f43305)
 2014-12-11 09:46:08 +05:30
Haohui Mai 759179e631 HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories. Contributed by Li Lu. 2014-12-09 10:46:25 -08:00
Harsh J bb1fedfbc3 MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) 
(cherry picked from commit 8963515b88)
 2014-12-08 16:00:12 +05:30
Jason Lowe db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Zhijie Shen d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367d)
 2014-12-01 19:08:31 -08:00
Jason Lowe b6d7b789bd MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith 
(cherry picked from commit 0c588904f8)
 2014-12-01 22:40:38 +00:00
Jason Lowe 2bdef5697b MAPREDUCE-6172. TestDbClasses timeouts are too aggressive. Contributed by Varun Saxena 
(cherry picked from commit 2b30fb1053)
 2014-12-01 21:52:51 +00:00
Jian He 80e9be2d95 MAPREDUCE-5568. Fixed CompletedJob in JHS to show progress percentage correctly in case the number of mappers or reducers is zero. Contributed by MinJi Kim 
(cherry picked from commit 78f7cdbfd6)
 2014-11-25 13:03:40 -08:00
Jonathan Eagles 6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845)
 2014-11-20 23:40:26 -06:00
Karthik Kambatla 3a4211ec2c MAPREDUCE-6169. MergeQueue should release reference to the current item from key and value at the end of the iteration to save memory. (Zhihai Xu via kasha) 
(cherry picked from commit 90194ca1cb)
 2014-11-20 15:37:18 -08:00
Devaraj K 09df35e51d MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown 
fails. Contributed by Rohith.
(cherry picked from commit 7250b0bf91)
 2014-11-18 22:07:26 +05:30
Jason Lowe 0b9a1a3e03 MAPREDUCE-6162. mapred hsadmin fails on a secure cluster. Contributed by Jason Lowe 
(cherry picked from commit 09b3dee122)
 2014-11-17 23:01:16 +00:00
Ravi Prakash 45692effe4 MAPREDUCE-5918. LineRecordReader can return the same decompressor to CodecPool multiple times (Sergey Murylev via raviprak) 2014-11-14 03:48:28 -08:00
Jason Lowe e62a5f9d0a MAPREDUCE-6156. Fetcher - connect() doesn't handle connection refused correctly. Contributed by Junping Du 
(cherry picked from commit 177e8090f5)
 2014-11-13 15:43:28 +00:00
cnauroth 69a7780ee5 HADOOP-7984. Add hadoop --loglevel option to change log level. Contributed by Aikira AJISAKA. 2014-11-12 21:44:05 -08:00
Arun C. Murthy adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy 175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408)
 2014-11-09 15:03:59 -08:00
Arun C. Murthy 1f5bad04ce HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid issues with newer versions of guava in applications. Contributed by Christopher Tubbs. 
(cherry picked from commit 6caa8100d5)
 2014-11-09 05:23:10 -08:00
Kihwal Lee a19123df9a MAPREDUCE-5958. Wrong reduce task progress if map output is compressed. Contributed by Emilio Coppa and Jason Lowe. 
(cherry picked from commit 8f701ae07a)
 2014-11-06 15:56:10 -06:00
Jason Lowe c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov 
(cherry picked from commit 10f9f5101c)
 2014-11-06 15:12:06 +00:00
Jian He 944fb1c3a9 MAPREDUCE-6048. Fixed TestJavaSerialization failure. Contributed by Varun Vasudev 
(cherry picked from commit 73068f677b)
 2014-11-04 20:20:33 -08:00
Haohui Mai fa92b40f09 HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN. Contributed by Robert Kanter. 2014-11-04 17:53:00 -08:00
Haohui Mai aa09c4c342 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 2014-11-04 14:52:51 -08:00
Zhijie Shen c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 
(cherry picked from commit ed63b11646)
 2014-11-01 00:49:38 -07:00
Vinod Kumar Vavilapalli 0ba8d8c19c YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16eb)
 2014-10-30 23:02:06 -07:00
Vinod Kumar Vavilapalli 063bb0508b MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen. 
(cherry picked from commit 3f1441245d)
 2014-10-29 09:58:50 -07:00
Karthik Kambatla 160aebcae2 HADOOP-11243. SSLFactory shouldn't allow SSLv3. (Wei Yan via kasha) 
(cherry picked from commit 3c5f5af118)
 2014-10-28 18:08:00 -07:00
Zhijie Shen 52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 971e91c8c0)
 2014-10-27 21:08:38 -07:00
Zhijie Shen 733380fa70 MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 6b2f11b54b)
 2014-10-27 20:41:02 -07:00
Zhijie Shen 3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be)
 2014-10-23 22:03:30 -07:00
Jian He 1c235a4448 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfb)
 2014-10-22 15:58:26 -07:00
Jason Lowe 8885b75d78 MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in client. Contributed by Rohith 
(cherry picked from commit 209b1699fc)
 2014-10-17 19:53:17 +00:00
Jian He f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe 025b6c13e9 MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov 
(cherry picked from commit f19771a24c)
 2014-10-15 17:51:53 +00:00
Jason Lowe 1179a0cb07 MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting for maps. Contributed by Siqi Li 
(cherry picked from commit b9edad6403)
 2014-10-15 15:54:57 +00:00
Jason Lowe 942fb67a04 MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk. Contributed by Binglin Chang 
(cherry picked from commit 793dbf2465)
 2014-10-13 16:05:29 +00:00
Jason Lowe e17e3bdbc4 MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai 
(cherry picked from commit bbe80cdc7b)
 2014-10-13 15:09:26 +00:00
Karthik Kambatla 7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
 2014-10-11 22:49:46 -07:00
cnauroth f6278cbac5 MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit 777877d079)
 2014-10-09 22:38:25 -07:00
cnauroth 85a2b90740 MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 684170d5ef)
 2014-10-09 22:27:40 -07:00
Jason Lowe 979b915ecc MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 
(cherry picked from commit 2e789eb226)
 2014-10-07 14:11:48 +00:00
subru 30a370e705 MAPREDUCE-6103.Adding reservation APIs to MR resource manager delegate. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit aa92dd45f2)
(cherry picked from commit 3f282762d1)
 2014-10-06 10:21:06 -07:00
Jason Lowe 7b8f2f5b1d MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk. Contributed by Akira AJISAKA 
(cherry picked from commit 23fbfe36c3)
 2014-09-29 18:14:11 +00:00
Jason Lowe 0cfff28f57 HADOOP-11049. javax package system class default is too broad. Contributed by Sangjin Lee 
(cherry picked from commit cf4631d7c9)
 2014-09-29 14:37:28 +00:00
Jian He a6879b606e MAPREDUCE-6087. Fixed wrong config name of MRJobConfig#MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS. Contributed by Akira AJISAKA 
(cherry picked from commit 5f16c98ad6)
 2014-09-26 18:06:19 -07:00
Allen Wittenauer 67d6fc3f01 MAPREDUCE-5945. Update the description of GenericOptionsParser -jt option (Akira AJISAKA via aw) 2014-09-26 14:26:36 -07:00
Allen Wittenauer e5d284664b MAPREDUCE-5796. Use current version of the archive name in DistributedCacheDeploy document (Akira AJISAKA via aw) 2014-09-26 12:59:59 -07:00
Allen Wittenauer fe8222998f MAPREDUCE-6073. Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags (Tsuyoshi OZAWA via aw) 2014-09-26 11:17:11 -07:00
Zhijie Shen 7b3ecc547f MAPREDUCE-5831. Make MR client ignore unknown counters received from AM. Contributed by Junping Du. 
(cherry picked from commit 662fc11ae7)
 2014-09-25 23:46:57 -07:00
Allen Wittenauer 7ffb636a54 MAPREDUCE-6093. minor distcp doc edits (Charles Lamb via aw) 2014-09-25 11:33:01 -07:00
Jason Lowe d5d9fd3255 MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2. Contributed by Mit Desai 
(cherry picked from commit 7af4c3888b)
 2014-09-24 16:10:28 +00:00