Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,448 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

685 Commits

Author SHA1 Message Date
Junping Du d9b006ff34 Preparing for 2.8.4 development. 2017-11-13 16:49:15 -08:00
Naganarasimha 01912d18d5 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e)
 2017-11-06 14:09:07 +05:30
Jason Lowe e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Haibo Chen 2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Junping Du 5ad710bc7a MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff60)
(cherry picked from commit f823f9fd78)
 2017-07-07 13:32:21 -07:00
Akira Ajisaka 71f49406f2
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 
(cherry picked from commit ac333155fb)
 2017-07-04 15:26:46 +09:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Jason Lowe 030d521a69 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42)
 2017-05-10 09:59:10 -05:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Zhe Zhang 0acf2bca9c MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687)
(cherry picked from commit 3f5dc51ec9)
 2017-03-29 22:19:50 -07:00
Joe Mészáros 9909174677
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3)
(cherry picked from commit 34a11b9108)
 2017-03-27 13:53:08 +09:00
Akira Ajisaka dd5d0103df
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f)
(cherry picked from commit 120f680318)
 2017-03-07 13:38:51 +09:00
Akira Ajisaka 6833b4ea30 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882)
(cherry picked from commit 69f2ae449b28530d01b73a172f2be6c256dca800)
 2017-02-27 19:47:35 +09:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Junping Du 5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Junping Du 8755a6f5ac Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:13:04 -08:00
Junping Du e8d7801319 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f529.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:05:17 -08:00
Akira Ajisaka 0c58241b22 MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5)
(cherry picked from commit 1e1ab9bc1b)
 2016-12-10 10:57:33 +09:00
Akira Ajisaka 61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c4)
(cherry picked from commit a3897f77ae)
 2016-11-01 20:52:52 +09:00
Akira Ajisaka 757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be)
(cherry picked from commit ef13a5ef9d)
 2016-10-31 15:19:37 +09:00
Jason Lowe 0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha 8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:04:03 +05:30
Karthik Kambatla c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Jason Lowe f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Jason Lowe 00c8ab91ba MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407)
 2016-08-24 14:42:53 +00:00
Akira Ajisaka 854af9cee1 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059)
(cherry picked from commit 41fe85f868)
 2016-08-23 17:12:32 +09:00
Wangda Tan e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
(cherry picked from commit 65822e55fd)
 2016-08-19 16:49:54 -07:00
Varun Saxena 6f1a9f1b6a MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:08:55 +05:30
Akira Ajisaka 59693ebbd5 MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
(cherry picked from commit e54de94e5a)
 2016-08-04 20:18:37 +09:00
Akira Ajisaka 19062df92f MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
(cherry picked from commit 0fc838540b)
 2016-07-30 11:47:33 +09:00
Robert Kanter ba83b62d36 MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50)
 2016-07-12 16:31:24 -07:00
Akira Ajisaka 5522a46f4c HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
(cherry picked from commit 9d46a49c74)
(cherry picked from commit b3b81b4f0e)
 2016-07-07 15:32:20 -07:00
Akira Ajisaka a0f6109907 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-27 15:19:21 -07:00
Akira Ajisaka ae5dc10ecd MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 
(cherry picked from commit 0f23cd497b)
(cherry picked from commit 2830ee5f0f)
 2016-06-25 00:27:35 +09:00
Wangda Tan 93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee)
 2016-06-16 11:10:46 -07:00
Wangda Tan d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Akira Ajisaka 3bea925cea MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton. 
(cherry picked from commit 0b7b8a3776)
(cherry picked from commit db2d96b378)
 2016-06-10 19:17:26 +09:00
Wangda Tan 19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Wangda Tan 2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f.
 2016-05-31 22:16:53 -07:00
Varun Vasudev 457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Jason Lowe ff8caccc87 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f)
 2016-05-13 14:36:13 +00:00
Steve Loughran 9b8f7a8fb2 MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 18:59:56 +01:00
Eric Payne 3de0ff030c MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-09 18:29:44 +00:00
Wangda Tan 2d9f42d685 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda) 
(cherry picked from commit 6529c87551)
 2016-03-15 17:32:32 -07:00
Harsh J 5f059e03f5 MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh) 2016-03-07 13:13:43 +05:30
Robert Kanter 3d8ae9c577 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e28)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-03-06 19:26:12 -08:00
Masatake Iwasaki 9d551a3adf HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md

(cherry picked from commit 722182fdfb)
 2016-03-04 14:13:30 +09:00
Varun Vasudev cb4f5ebd3e MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5)
(cherry picked from commit f1999fe275)
 2016-02-23 13:07:40 +05:30
Andrew Wang d31660ef4a MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 37577852ba)
(cherry picked from commit 7f6737951a)
 2016-02-19 16:33:44 -08:00
Tsuyoshi Ozawa 7ccb0e0601 MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy. 
(cherry picked from commit 453e7e0d10)
 2016-02-17 14:18:24 +09:00