Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,919 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

668 Commits

Author SHA1 Message Date
Akira Ajisaka 0c58241b22 MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5)
(cherry picked from commit 1e1ab9bc1b)
 2016-12-10 10:57:33 +09:00
Akira Ajisaka 61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c4)
(cherry picked from commit a3897f77ae)
 2016-11-01 20:52:52 +09:00
Akira Ajisaka 757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be)
(cherry picked from commit ef13a5ef9d)
 2016-10-31 15:19:37 +09:00
Jason Lowe 0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha 8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:04:03 +05:30
Karthik Kambatla c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Jason Lowe f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Jason Lowe 00c8ab91ba MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407)
 2016-08-24 14:42:53 +00:00
Akira Ajisaka 854af9cee1 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059)
(cherry picked from commit 41fe85f868)
 2016-08-23 17:12:32 +09:00
Wangda Tan e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
(cherry picked from commit 65822e55fd)
 2016-08-19 16:49:54 -07:00
Varun Saxena 6f1a9f1b6a MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:08:55 +05:30
Akira Ajisaka 59693ebbd5 MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
(cherry picked from commit e54de94e5a)
 2016-08-04 20:18:37 +09:00
Akira Ajisaka 19062df92f MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
(cherry picked from commit 0fc838540b)
 2016-07-30 11:47:33 +09:00
Robert Kanter ba83b62d36 MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50)
 2016-07-12 16:31:24 -07:00
Akira Ajisaka 5522a46f4c HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
(cherry picked from commit 9d46a49c74)
(cherry picked from commit b3b81b4f0e)
 2016-07-07 15:32:20 -07:00
Akira Ajisaka a0f6109907 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-27 15:19:21 -07:00
Akira Ajisaka ae5dc10ecd MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 
(cherry picked from commit 0f23cd497b)
(cherry picked from commit 2830ee5f0f)
 2016-06-25 00:27:35 +09:00
Wangda Tan 93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee)
 2016-06-16 11:10:46 -07:00
Wangda Tan d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Akira Ajisaka 3bea925cea MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton. 
(cherry picked from commit 0b7b8a3776)
(cherry picked from commit db2d96b378)
 2016-06-10 19:17:26 +09:00
Wangda Tan 19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Wangda Tan 2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f.
 2016-05-31 22:16:53 -07:00
Varun Vasudev 457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Jason Lowe ff8caccc87 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f)
 2016-05-13 14:36:13 +00:00
Steve Loughran 9b8f7a8fb2 MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 18:59:56 +01:00
Eric Payne 3de0ff030c MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-09 18:29:44 +00:00
Wangda Tan 2d9f42d685 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda) 
(cherry picked from commit 6529c87551)
 2016-03-15 17:32:32 -07:00
Harsh J 5f059e03f5 MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh) 2016-03-07 13:13:43 +05:30
Robert Kanter 3d8ae9c577 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e28)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-03-06 19:26:12 -08:00
Masatake Iwasaki 9d551a3adf HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md

(cherry picked from commit 722182fdfb)
 2016-03-04 14:13:30 +09:00
Varun Vasudev cb4f5ebd3e MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5)
(cherry picked from commit f1999fe275)
 2016-02-23 13:07:40 +05:30
Andrew Wang d31660ef4a MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 37577852ba)
(cherry picked from commit 7f6737951a)
 2016-02-19 16:33:44 -08:00
Tsuyoshi Ozawa 7ccb0e0601 MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy. 
(cherry picked from commit 453e7e0d10)
 2016-02-17 14:18:24 +09:00
Akira Ajisaka f943bf0e22 MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun. 
(cherry picked from commit 10393d9a666419a5d8dc0cfb48da96e9c690c11e)
(cherry picked from commit 8cbe65a07dd0deb150e91b13e3a1cec13e3716ee)
 2016-02-10 03:05:23 +09:00
Jason Lowe 1d322ca5cc MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d)
 2016-02-02 19:32:04 +00:00
Akira Ajisaka b85b8d90d1 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 7f215ffdd5)
 2016-01-22 18:38:08 +09:00
Jason Lowe de31bff67d MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 
(cherry picked from commit 9fbd579ab5)
 2016-01-15 16:54:03 +00:00
Akira Ajisaka 6816481c72 MAPREDUCE-6601. Fix typo in Job#setUseNewAPI. Contributed by Kai Sasaki. 
(cherry picked from commit 5cc44d18aa)
(cherry picked from commit 7c4be15991)
 2016-01-15 00:44:52 +09:00
Jason Lowe a22a7f61e4 MAPREDUCE-6473. Job submission can take a long time during Cluster initialization. Contributed by Kuhu Shukla 
(cherry picked from commit f657b54281)
 2016-01-12 23:59:06 +00:00
Junping Du d7e10f8174 MAPREDUCE-6068. Illegal progress value warnings in map tasks. Contributed by Binglin Chang. 
(cherry picked from commit 8c1adeaa26)
(cherry picked from commit 51073c910c)
 2016-01-11 06:14:58 -08:00
Sangjin Lee ae535ec93e MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 
(cherry picked from commit f6f16118d3)
(cherry picked from commit 28fd4c70ca)
 2016-01-06 08:44:37 -08:00
Akira Ajisaka 2139458a98 MAPREDUCE-6589. TestTaskLog outputs a log under directory other than target/test-dir. (aajisaka) 
(cherry picked from commit 2a59d01503)
(cherry picked from commit a8314505d0)
 2015-12-30 01:34:26 +09:00
Akira Ajisaka 7063d8d013 MAPREDUCE-6584. Remove trailing whitespaces from mapred-default.xml. (aajisaka) 
(cherry picked from commit 1a06949126)
 2015-12-30 01:04:40 +09:00
Akira Ajisaka 5f34cbff5c MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki. 
(cherry picked from commit 7995a6ea4d)
(cherry picked from commit 8607cb6074)
 2015-12-21 00:18:40 +09:00
Xuan 93c483889c MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit d4e766de93)
(cherry picked from commit 5891d6ed55)
 2015-12-16 15:31:09 -08:00
Robert Kanter 8c6273d117 MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4)
 2015-11-25 17:03:52 -08:00
Jason Lowe 2f18218508 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G 
(cherry picked from commit f634505d48)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
 2015-11-24 22:15:37 +00:00
Jason Lowe ef3e01a1e4 MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c889)
 2015-11-24 22:01:53 +00:00
Jian He f562c155d2 MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:08:22 -08:00
Jason Lowe ee5c20ddef MAPREDUCE-6533. testDetermineCacheVisibilities of TestClientDistributedCacheManager is broken. Contributed by Chang Li 
(cherry picked from commit df68eac825)
 2015-11-11 17:27:03 +00:00