Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,563 Commits 326 Branches 371 Tags
Commit Graph

1572 Commits

Author SHA1 Message Date
Gera Shegalov
6802e8fefc MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output files. (Siqi Li via gera) 
(cherry picked from commit aa92b764a7ddb888d097121c4d610089a0053d11)
 2015-03-10 11:40:45 -07:00
Tsuyoshi Ozawa
482504befd MAPREDUCE-4742. Fix typo in nnbench#displayUsage. Contributed by Liang Xie. 
(cherry picked from commit 20b8ee1350e62d1b21c951e653302b6e6a8e4f7e)
 2015-03-11 00:41:37 +09:00
Junping Du
dbcdcb0d3c Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne. 
(cherry picked from commit d39bc903a0069a740744bafe10e506e452ed7018)
 2015-03-10 06:23:24 -07:00
Vinod Kumar Vavilapalli
8e6f8e9a04 Adding 2.8 section in CHANGES.txt 2015-03-08 20:26:57 -07:00
Tsuyoshi Ozawa
3c24e50ce2 Update CHANGES.txt for MAPREDUCE-5657. 2015-03-09 12:07:56 +09:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Tsuyoshi Ozawa
ac874764d5 MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 348208014b2023894dc416dcd733a9ccdafc6e25)
 2015-03-05 14:13:13 +09:00
Karthik Kambatla
f4d6c5e337 MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha) 
(cherry picked from commit c66c3ac6bf9f63177279feec3f2917e4b882e2bc)
 2015-03-04 14:47:06 -08:00
Vinod Kumar Vavilapalli
6604613bc5 MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public API in DistCp. Contributed by Jing Zhao. 
(cherry picked from commit 5af693fde26755b6f175bd65f93cf4a80de0d1e0)
 2015-03-03 16:29:36 -08:00
Junping Du
a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de94028f1e10ca59ce23e963e488fe566909)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Tsuyoshi Ozawa
b1e7f9c588 MAPREDUCE-6268. Fix typo in Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit 742f9d90c00f823ad7fea7e79702fcf238fa5721)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md
 2015-03-03 16:24:27 +09:00
Akira Ajisaka
8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7dcb71b59d85b05d0a6da88195ea9902)
 2015-02-26 15:06:26 -08:00
Akira Ajisaka
5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a6747d5d1a8749750698a1ae2f7b68414)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Plamen Jeliazkov
32d215dc89 MAPREDUCE-6228. Add truncate operation to SLive. Contributed by Plamen Jeliazkov. 2015-02-19 00:14:11 -08:00
Jason Lowe
386e62ace3 MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4981d082d4f3c82d1c2c900c7488b83bf20301cc)
 2015-02-18 19:28:45 +00:00
Tsuyoshi Ozawa
9523b52da5 MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure conditions. Contributed by Devaraj K. 
(cherry picked from commit 7c782047c609b29178945bd566a0d162e64dbfdb)
 2015-02-18 15:46:06 +09:00
Junping Du
1d91daaae9 MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena 
(cherry picked from commit 814afa46efef201cb782072432fc744e1cb9c463)
 2015-02-16 09:39:20 -08:00
Tsuyoshi Ozawa
7011125778 MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R. 
(cherry picked from commit 3338f6d90795dafd445891cbd2f4d46b53012382)
 2015-02-15 10:10:11 +09:00
Tsuyoshi Ozawa
7386323735 MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi. 
(cherry picked from commit ba3c80a5ca8fefae9d0e67232f9973e1a0458f58)
 2015-02-13 16:11:16 +09:00
Tsuyoshi Ozawa
9d57392f5b MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu. 
(cherry picked from commit 9b0ba59b8284fae132535fbca5ce372d7a6c38c0)
 2015-02-13 02:28:19 +09:00
Tsuyoshi Ozawa
941a923514 MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. 
(cherry picked from commit ac8d52bf50bba1a29489ee75fd90717d8a2b0cc9)
 2015-02-12 20:37:14 +09:00
Tsuyoshi Ozawa
9129b3415c MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K. 
(cherry picked from commit b42d09eb62bd1725d70da59f1a6fdac83cea82d1)
 2015-02-12 20:08:47 +09:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
 2015-02-12 13:47:19 +05:30
Tsuyoshi Ozawa
b1aad1d941 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b7cf97d7e0bd84dbe0542b72f304dc9)
 2015-02-10 03:53:15 +09:00
yliu
6933975143 MAPREDUCE-6227. DFSIO for truncate. (shv via yliu) 2015-02-08 02:42:10 +08:00
Robert Kanter
46a2027226 MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk (zxu via rkanter) 
(cherry picked from commit e2ee2ff7d7ca429487d7e3883daedffbb269ebd4)
 2015-02-05 14:31:44 -08:00
Jason Lowe
eb927a1d6b MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal 
(cherry picked from commit e1990ab4275b9092cae3573334791f7b2f8df7e5)
 2015-02-05 20:14:13 +00:00
Jason Lowe
2dc4af2b93 MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw) 
(cherry picked from commit fd57ab2002f97dcc83d455a5e0c770c8efde77a4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-02-05 16:40:09 +00:00
Akira Ajisaka
ad4b243586 MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(cherry picked from commit cc6bbfceae1cddfae6a3892cb7e7104531a689be)
 2015-02-04 17:46:39 -08:00
Akira Ajisaka
0a030871dd MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. Contributed by Masatake Iwasaki. 
(cherry picked from commit 34fe11c987730932f99dec6eb458a22624eb075b)
 2015-02-04 09:27:01 -08:00
Akira Ajisaka
8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Robert Kanter
a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b3fea8b418b3526c15022c8a9fedd56)
 2015-02-02 13:52:29 -08:00
Allen Wittenauer
a754f94ed7 MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw) 2015-01-30 11:09:03 -08:00
Allen Wittenauer
3165e778f3 MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw) 2015-01-29 14:18:25 -08:00
Akira Ajisaka
57172c09e2 MAPREDUCE-6231. Grep example job is not working on a fully-distributed cluster. (aajisaka) 
(cherry picked from commit fe2188a0420f501ccfcba92e91d4b564ba86103e)
 2015-01-30 01:11:22 +09:00
Jian He
43b3b43cea MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe 
(cherry picked from commit cff05bff1fe24628677d41a0d537f2c383b44faf)
 2015-01-28 15:52:11 -08:00
Jason Lowe
2cf58ca5a6 MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe 
(cherry picked from commit 56b7ec71a69820ae12b4b9e2eb04b7368f721dbf)
 2015-01-26 16:30:10 +00:00
cnauroth
60dc848f23 MAPREDUCE-3283. mapred classpath CLI does not display the complete classpath. Contributed by Varun Saxena. 2015-01-21 13:52:09 -08:00
Akira Ajisaka
2e5eef9da2 MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo) 
(cherry picked from commit d336d136785ef1e49e0a110a1b9f0d1824829877)
 2015-01-14 17:43:25 +09:00
Xuan
0562f382fa MAPREDUCE-6173. Document the configuration of deploying MR over 
distributed cache with enabling wired encryption at the same time.
Contributed by Junping Du.

(cherry picked from commit 10ac5abb867ead7131f3c76b27e48f7908b1d7f1)
 2015-01-13 10:08:24 -08:00
Tsuyoshi Ozawa
132fd6ba58 HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa) 
(cherry picked from commit a6ed4894b518351bf1b3290e725a475570a21296)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/java/org/apache/hadoop/mapred/nativetask/kvtest/KVJob.java
 2015-01-08 15:00:02 +09:00
Jason Lowe
c22c27068e MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed by Jens Rabe 
(cherry picked from commit a0aeed100b839d5c122974bd0ea12f9168ee36a5)
 2015-01-06 23:40:42 +00:00
Harsh J
6d65e74670 MAPREDUCE-6149. Document override log4j.properties in MR job. Contributed by Junping Du. 
(cherry picked from commit 6621c3598e22279cde11eca73cfb5619a8bc8dee)
 2014-12-30 23:13:19 +05:30
Robert Kanter
f9341c1e2c MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter) 
(cherry picked from commit 390a7c12f543b2c94a74f08d6d2a28410472043a)
 2014-12-19 15:44:01 -08:00
Robert Kanter
22b38cf29a MAPREDUCE-6045. need close the DataInputStream after open it in TestMapReduce.java (zxu via rkanter) 
(cherry picked from commit d9e4d67d18811e16d5b0a76ea8228d333ded195f)
 2014-12-19 12:00:00 -08:00
Gera Shegalov
7c7bccfc31 MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8ba82f98f468dc7375fe89c2e0a7912)
 2014-12-15 19:21:46 -08:00
Harsh J
a3fa1f0761 MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. 
(cherry picked from commit 298d09c9b583088f364038adcb1edf1eb1c2c196)
 2014-12-15 14:27:49 +05:30
Gera Shegalov
6465931c19 MAPREDUCE-4879. TeraOutputFormat may overwrite an existing output directory. (gera) 
(cherry picked from commit 25a04402389dce0222938df4375d4545f8c1f34f)
 2014-12-13 18:12:02 -08:00
Devaraj K
25f4a0193b MAPREDUCE-6046. Change the class name for logs in RMCommunicator. 
Contributed by Sahil Takiar.

(cherry picked from commit 0bd022911013629a8c9e7357fae8cf4399d7a1e3)
 2014-12-12 11:44:00 +05:30