Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,235 Commits 326 Branches 371 Tags
Commit Graph

693 Commits

Author SHA1 Message Date
Junping Du
dbcdcb0d3c Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne. 
(cherry picked from commit d39bc903a0069a740744bafe10e506e452ed7018)
 2015-03-10 06:23:24 -07:00
Tsuyoshi Ozawa
b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli
d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Karthik Kambatla
f4d6c5e337 MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha) 
(cherry picked from commit c66c3ac6bf9f63177279feec3f2917e4b882e2bc)
 2015-03-04 14:47:06 -08:00
Junping Du
a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de94028f1e10ca59ce23e963e488fe566909)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Akira Ajisaka
f7a724ca9e HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki. 
(cherry picked from commit 9112f093cde0b8242324aa5267c9beb21c38bf6b)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-02-28 17:06:58 -08:00
Akira Ajisaka
8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7dcb71b59d85b05d0a6da88195ea9902)
 2015-02-26 15:06:26 -08:00
Akira Ajisaka
5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a6747d5d1a8749750698a1ae2f7b68414)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Jason Lowe
386e62ace3 MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4981d082d4f3c82d1c2c900c7488b83bf20301cc)
 2015-02-18 19:28:45 +00:00
Junping Du
1d91daaae9 MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena 
(cherry picked from commit 814afa46efef201cb782072432fc744e1cb9c463)
 2015-02-16 09:39:20 -08:00
Tsuyoshi Ozawa
7011125778 MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R. 
(cherry picked from commit 3338f6d90795dafd445891cbd2f4d46b53012382)
 2015-02-15 10:10:11 +09:00
Tsuyoshi Ozawa
9d57392f5b MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu. 
(cherry picked from commit 9b0ba59b8284fae132535fbca5ce372d7a6c38c0)
 2015-02-13 02:28:19 +09:00
Tsuyoshi Ozawa
941a923514 MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. 
(cherry picked from commit ac8d52bf50bba1a29489ee75fd90717d8a2b0cc9)
 2015-02-12 20:37:14 +09:00
Tsuyoshi Ozawa
9129b3415c MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K. 
(cherry picked from commit b42d09eb62bd1725d70da59f1a6fdac83cea82d1)
 2015-02-12 20:08:47 +09:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
 2015-02-12 13:47:19 +05:30
Tsuyoshi Ozawa
b1aad1d941 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b7cf97d7e0bd84dbe0542b72f304dc9)
 2015-02-10 03:53:15 +09:00
Akira Ajisaka
ad4b243586 MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(cherry picked from commit cc6bbfceae1cddfae6a3892cb7e7104531a689be)
 2015-02-04 17:46:39 -08:00
Akira Ajisaka
8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Robert Kanter
a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b3fea8b418b3526c15022c8a9fedd56)
 2015-02-02 13:52:29 -08:00
Jason Lowe
2cf58ca5a6 MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe 
(cherry picked from commit 56b7ec71a69820ae12b4b9e2eb04b7368f721dbf)
 2015-01-26 16:30:10 +00:00
Xuan
0562f382fa MAPREDUCE-6173. Document the configuration of deploying MR over 
distributed cache with enabling wired encryption at the same time.
Contributed by Junping Du.

(cherry picked from commit 10ac5abb867ead7131f3c76b27e48f7908b1d7f1)
 2015-01-13 10:08:24 -08:00
Tsuyoshi Ozawa
132fd6ba58 HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa) 
(cherry picked from commit a6ed4894b518351bf1b3290e725a475570a21296)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/java/org/apache/hadoop/mapred/nativetask/kvtest/KVJob.java
 2015-01-08 15:00:02 +09:00
Jason Lowe
c22c27068e MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed by Jens Rabe 
(cherry picked from commit a0aeed100b839d5c122974bd0ea12f9168ee36a5)
 2015-01-06 23:40:42 +00:00
Jason Lowe
2b408d8dc7 HADOOP-11445. Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split. Contributed by Ankit Kamboj 
(cherry picked from commit d02fb53750bc592c23ba470ae82eb6f47d9a00ec)
 2015-01-06 21:20:31 +00:00
Harsh J
6d65e74670 MAPREDUCE-6149. Document override log4j.properties in MR job. Contributed by Junping Du. 
(cherry picked from commit 6621c3598e22279cde11eca73cfb5619a8bc8dee)
 2014-12-30 23:13:19 +05:30
Robert Kanter
f9341c1e2c MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter) 
(cherry picked from commit 390a7c12f543b2c94a74f08d6d2a28410472043a)
 2014-12-19 15:44:01 -08:00
Karthik Kambatla
e6c73a1c1a HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle. (Wei Yan via kasha) 
(cherry picked from commit 6f1e3667cff3e95f33ba14a3cb7d1e99518d502a)
 2014-12-19 15:07:39 -08:00
Gera Shegalov
7c7bccfc31 MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8ba82f98f468dc7375fe89c2e0a7912)
 2014-12-15 19:21:46 -08:00
Harsh J
a3fa1f0761 MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. 
(cherry picked from commit 298d09c9b583088f364038adcb1edf1eb1c2c196)
 2014-12-15 14:27:49 +05:30
Gera Shegalov
8437c31b67 HADOOP-11211. mapreduce.job.classloader.system.classes semantics should be order-independent. (Yitong Zhou via gera) 
(cherry picked from commit 0bcea111e5daa9a4315346cf6919a4cfc8d90e0d)
 2014-12-11 13:29:40 -08:00
Harsh J
6e06a51e30 MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml. Contributed by James Carman. (harsh) 
(cherry picked from commit cb99f43305bd1577d4ba9527d237ac6cdb9ae730)
 2014-12-11 09:46:08 +05:30
Harsh J
bb1fedfbc3 MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh) 
(cherry picked from commit 8963515b880b78068791f11abe4f5df332553be1)
 2014-12-08 16:00:12 +05:30
Jason Lowe
db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01ffea1cacf1fa9cbbf73c3f2904d981)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Jason Lowe
2bdef5697b MAPREDUCE-6172. TestDbClasses timeouts are too aggressive. Contributed by Varun Saxena 
(cherry picked from commit 2b30fb1053e70c128b98013fb63cf9a095623be6)
 2014-12-01 21:52:51 +00:00
Karthik Kambatla
3a4211ec2c MAPREDUCE-6169. MergeQueue should release reference to the current item from key and value at the end of the iteration to save memory. (Zhihai Xu via kasha) 
(cherry picked from commit 90194ca1cbd695d48c3705121c2ac9a8554578a2)
 2014-11-20 15:37:18 -08:00
Ravi Prakash
45692effe4 MAPREDUCE-5918. LineRecordReader can return the same decompressor to CodecPool multiple times (Sergey Murylev via raviprak) 2014-11-14 03:48:28 -08:00
Jason Lowe
e62a5f9d0a MAPREDUCE-6156. Fetcher - connect() doesn't handle connection refused correctly. Contributed by Junping Du 
(cherry picked from commit 177e8090f5809beb3ebcb656cd0affbb3f487de8)
 2014-11-13 15:43:28 +00:00
cnauroth
69a7780ee5 HADOOP-7984. Add hadoop --loglevel option to change log level. Contributed by Aikira AJISAKA. 2014-11-12 21:44:05 -08:00
Arun C. Murthy
adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy
1f5bad04ce HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid issues with newer versions of guava in applications. Contributed by Christopher Tubbs. 
(cherry picked from commit 6caa8100d5d2547e34356dc279fd5e65b81a925a)
 2014-11-09 05:23:10 -08:00
Kihwal Lee
a19123df9a MAPREDUCE-5958. Wrong reduce task progress if map output is compressed. Contributed by Emilio Coppa and Jason Lowe. 
(cherry picked from commit 8f701ae07a0b1dc70b8e1eb8d4a5c35c0a1e76da)
 2014-11-06 15:56:10 -06:00
Jason Lowe
c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov 
(cherry picked from commit 10f9f5101c44be7c675a44ded4aad212627ecdee)
 2014-11-06 15:12:06 +00:00
Zhijie Shen
c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 
(cherry picked from commit ed63b116465290fdb0acdf89170025f47b307599)
 2014-11-01 00:49:38 -07:00
Karthik Kambatla
160aebcae2 HADOOP-11243. SSLFactory shouldn't allow SSLv3. (Wei Yan via kasha) 
(cherry picked from commit 3c5f5af1184e85158dec962df0b0bc2be8d0d1e3)
 2014-10-28 18:08:00 -07:00
Zhijie Shen
52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)
 2014-10-27 21:08:38 -07:00
Jian He
1c235a4448 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfbf4cc91ef8e8616c7aafa9de006cde5)
 2014-10-22 15:58:26 -07:00
Jian He
f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe
025b6c13e9 MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov 
(cherry picked from commit f19771a24c2f90982cf6dec35889836a6146c968)
 2014-10-15 17:51:53 +00:00