Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,765 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

558 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli d9d7bbd99b Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e0)
(cherry picked from commit 87862970f1)
 2015-05-14 17:00:56 -07:00
Vinod Kumar Vavilapalli 0e68e304f7 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e)
 2015-05-12 12:16:16 -07:00
Jason Lowe e7b6a076db MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae28)
 2015-05-04 20:40:43 +00:00
Jason Lowe 30d0f10458 MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne 
(cherry picked from commit bc1bd7e5c4)
 2015-04-28 20:20:32 +00:00
Robert Kanter 0eae3bc0f6 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f0928)
 2015-04-23 17:47:00 -07:00
Vinod Kumar Vavilapalli a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Junping Du 86e8c9958a MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe. 
(cherry picked from commit eccb7d46ef)
 2015-04-02 12:16:22 -07:00
Vinod Kumar Vavilapalli ec123abdc9 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:30:04 -07:00
Vinod Kumar Vavilapalli 1c050bd135 Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:28:51 -07:00
Vinod Kumar Vavilapalli 02646ddba2 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 929b04ce3a.
 2015-03-31 11:26:13 -07:00
Vinod Kumar Vavilapalli 35af6f1802 YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f51)
 2015-03-30 10:11:49 -07:00
Harsh J 929b04ce3a MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu. 2015-03-22 10:18:32 +05:30
Jason Lowe 0be2461df5 MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for user-specified task output subdirs. Contributed by Gera Shegalov and Siqi Li 
(cherry picked from commit 91baca145a)
 2015-03-19 21:41:49 +00:00
Tsuyoshi Ozawa 9f4ec4c540 MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu 
(cherry picked from commit 9d38520c8e)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

(cherry picked from commit 10745fcecd)
 2015-03-14 16:46:53 +09:00
Haohui Mai 245f7b2a77 HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki. 2015-03-11 14:31:16 -07:00
Gera Shegalov fca89922c2 MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output files. (Siqi Li via gera) 
(cherry picked from commit aa92b764a7)
 2015-03-10 11:45:03 -07:00
Junping Du 9d1f67f2f2 Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne. 
(cherry picked from commit d39bc903a0)
(cherry picked from commit dbcdcb0d3c)
 2015-03-10 06:24:34 -07:00
Tsuyoshi Ozawa 28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72db)
 2015-03-09 19:54:45 +09:00
Tsuyoshi Ozawa ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Karthik Kambatla f4d6c5e337 MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha) 
(cherry picked from commit c66c3ac6bf)
 2015-03-04 14:47:06 -08:00
Junping Du a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de9402)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Akira Ajisaka f7a724ca9e HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki. 
(cherry picked from commit 9112f093cd)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-02-28 17:06:58 -08:00
Akira Ajisaka 8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7d)
 2015-02-26 15:06:26 -08:00
Akira Ajisaka 5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a67)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka 5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Jason Lowe 386e62ace3 MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4981d082d4)
 2015-02-18 19:28:45 +00:00
Junping Du 1d91daaae9 MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena 
(cherry picked from commit 814afa46ef)
 2015-02-16 09:39:20 -08:00
Tsuyoshi Ozawa 7011125778 MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R. 
(cherry picked from commit 3338f6d907)
 2015-02-15 10:10:11 +09:00
Tsuyoshi Ozawa 9d57392f5b MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu. 
(cherry picked from commit 9b0ba59b82)
 2015-02-13 02:28:19 +09:00
Tsuyoshi Ozawa 941a923514 MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. 
(cherry picked from commit ac8d52bf50)
 2015-02-12 20:37:14 +09:00
Tsuyoshi Ozawa 9129b3415c MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K. 
(cherry picked from commit b42d09eb62)
 2015-02-12 20:08:47 +09:00
Devaraj K a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead0)
 2015-02-12 13:47:19 +05:30
Tsuyoshi Ozawa b1aad1d941 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b)
 2015-02-10 03:53:15 +09:00
Akira Ajisaka ad4b243586 MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(cherry picked from commit cc6bbfceae)
 2015-02-04 17:46:39 -08:00
Akira Ajisaka 8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Robert Kanter a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b)
 2015-02-02 13:52:29 -08:00
Jason Lowe 2cf58ca5a6 MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe 
(cherry picked from commit 56b7ec71a6)
 2015-01-26 16:30:10 +00:00
Xuan 0562f382fa MAPREDUCE-6173. Document the configuration of deploying MR over 
distributed cache with enabling wired encryption at the same time.
Contributed by Junping Du.

(cherry picked from commit 10ac5abb86)
 2015-01-13 10:08:24 -08:00
Tsuyoshi Ozawa 132fd6ba58 HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa) 
(cherry picked from commit a6ed4894b5)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/java/org/apache/hadoop/mapred/nativetask/kvtest/KVJob.java
 2015-01-08 15:00:02 +09:00
Jason Lowe c22c27068e MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed by Jens Rabe 
(cherry picked from commit a0aeed100b)
 2015-01-06 23:40:42 +00:00
Jason Lowe 2b408d8dc7 HADOOP-11445. Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split. Contributed by Ankit Kamboj 
(cherry picked from commit d02fb53750)
 2015-01-06 21:20:31 +00:00
Harsh J 6d65e74670 MAPREDUCE-6149. Document override log4j.properties in MR job. Contributed by Junping Du. 
(cherry picked from commit 6621c3598e)
 2014-12-30 23:13:19 +05:30
Robert Kanter f9341c1e2c MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter) 
(cherry picked from commit 390a7c12f5)
 2014-12-19 15:44:01 -08:00
Karthik Kambatla e6c73a1c1a HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle. (Wei Yan via kasha) 
(cherry picked from commit 6f1e3667cf)
 2014-12-19 15:07:39 -08:00
Gera Shegalov 7c7bccfc31 MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8)
 2014-12-15 19:21:46 -08:00
Harsh J a3fa1f0761 MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. 
(cherry picked from commit 298d09c9b5)
 2014-12-15 14:27:49 +05:30
Gera Shegalov 8437c31b67 HADOOP-11211. mapreduce.job.classloader.system.classes semantics should be order-independent. (Yitong Zhou via gera) 
(cherry picked from commit 0bcea111e5)
 2014-12-11 13:29:40 -08:00
Harsh J 6e06a51e30 MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml. Contributed by James Carman. (harsh) 
(cherry picked from commit cb99f43305)
 2014-12-11 09:46:08 +05:30
Harsh J bb1fedfbc3 MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) 
(cherry picked from commit 8963515b88)
 2014-12-08 16:00:12 +05:30
Jason Lowe db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00