Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,072 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

572 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli cab5d86738 Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev" 
This reverts commit ba02e8f6d1.
 2015-12-16 15:17:01 -08:00
Xuan ba02e8f6d1 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit fc470840a0)
(cherry picked from commit 3b10ff6a6f)
 2015-12-07 14:18:26 -08:00
Jason Lowe e1b6a5413f MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-30 17:45:27 +00:00
Jason Lowe 4ba5667aed MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c889)
 2015-11-24 22:02:21 +00:00
Jian He f95f416fad MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:13:49 -08:00
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Jason Lowe 521196874c MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:10:38 +00:00
Jason Lowe 850d6a3fd3 MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:04:02 +00:00
Jason Lowe 4ea42b84b7 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d)

(cherry picked from commit 87c2d915f1)
 2015-09-10 11:00:00 -07:00
Tsuyoshi Ozawa e8a15495bc MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec)
 2015-09-05 11:24:30 +09:00
Vinod Kumar Vavilapalli a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Karthik Kambatla d404caaa43 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33)
(cherry picked from commit 1a2e6e8091)
 2015-06-05 09:19:52 -07:00
Akira Ajisaka 718dca4859 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cba)
(cherry picked from commit 1cccd1eeb3)
 2015-06-05 01:47:01 +09:00
Tsuyoshi Ozawa 6b03ec5137 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 2015-05-19 12:01:14 -07:00
Vinod Kumar Vavilapalli d9d7bbd99b Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e0)
(cherry picked from commit 87862970f1)
 2015-05-14 17:00:56 -07:00
Vinod Kumar Vavilapalli 0e68e304f7 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e)
 2015-05-12 12:16:16 -07:00
Jason Lowe e7b6a076db MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae28)
 2015-05-04 20:40:43 +00:00
Jason Lowe 30d0f10458 MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne 
(cherry picked from commit bc1bd7e5c4)
 2015-04-28 20:20:32 +00:00
Robert Kanter 0eae3bc0f6 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f0928)
 2015-04-23 17:47:00 -07:00
Vinod Kumar Vavilapalli a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Junping Du 86e8c9958a MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe. 
(cherry picked from commit eccb7d46ef)
 2015-04-02 12:16:22 -07:00
Vinod Kumar Vavilapalli ec123abdc9 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:30:04 -07:00
Vinod Kumar Vavilapalli 1c050bd135 Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:28:51 -07:00
Vinod Kumar Vavilapalli 02646ddba2 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 929b04ce3a.
 2015-03-31 11:26:13 -07:00
Vinod Kumar Vavilapalli 35af6f1802 YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f51)
 2015-03-30 10:11:49 -07:00
Harsh J 929b04ce3a MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu. 2015-03-22 10:18:32 +05:30
Jason Lowe 0be2461df5 MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for user-specified task output subdirs. Contributed by Gera Shegalov and Siqi Li 
(cherry picked from commit 91baca145a)
 2015-03-19 21:41:49 +00:00
Tsuyoshi Ozawa 9f4ec4c540 MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu 
(cherry picked from commit 9d38520c8e)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

(cherry picked from commit 10745fcecd)
 2015-03-14 16:46:53 +09:00
Haohui Mai 245f7b2a77 HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki. 2015-03-11 14:31:16 -07:00
Gera Shegalov fca89922c2 MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output files. (Siqi Li via gera) 
(cherry picked from commit aa92b764a7)
 2015-03-10 11:45:03 -07:00
Junping Du 9d1f67f2f2 Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne. 
(cherry picked from commit d39bc903a0)
(cherry picked from commit dbcdcb0d3c)
 2015-03-10 06:24:34 -07:00
Tsuyoshi Ozawa 28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72db)
 2015-03-09 19:54:45 +09:00
Tsuyoshi Ozawa ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Karthik Kambatla f4d6c5e337 MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha) 
(cherry picked from commit c66c3ac6bf)
 2015-03-04 14:47:06 -08:00
Junping Du a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de9402)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Akira Ajisaka f7a724ca9e HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki. 
(cherry picked from commit 9112f093cd)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-02-28 17:06:58 -08:00
Akira Ajisaka 8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7d)
 2015-02-26 15:06:26 -08:00
Akira Ajisaka 5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a67)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka 5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Jason Lowe 386e62ace3 MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4981d082d4)
 2015-02-18 19:28:45 +00:00
Junping Du 1d91daaae9 MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena 
(cherry picked from commit 814afa46ef)
 2015-02-16 09:39:20 -08:00
Tsuyoshi Ozawa 7011125778 MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R. 
(cherry picked from commit 3338f6d907)
 2015-02-15 10:10:11 +09:00
Tsuyoshi Ozawa 9d57392f5b MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu. 
(cherry picked from commit 9b0ba59b82)
 2015-02-13 02:28:19 +09:00
Tsuyoshi Ozawa 941a923514 MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. 
(cherry picked from commit ac8d52bf50)
 2015-02-12 20:37:14 +09:00
Tsuyoshi Ozawa 9129b3415c MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K. 
(cherry picked from commit b42d09eb62)
 2015-02-12 20:08:47 +09:00
Devaraj K a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead0)
 2015-02-12 13:47:19 +05:30
Tsuyoshi Ozawa b1aad1d941 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b)
 2015-02-10 03:53:15 +09:00
Akira Ajisaka ad4b243586 MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(cherry picked from commit cc6bbfceae)
 2015-02-04 17:46:39 -08:00
Akira Ajisaka 8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Robert Kanter a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b)
 2015-02-02 13:52:29 -08:00