Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,968 Commits 326 Branches 371 Tags
Commit Graph

1264 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Tsuyoshi Ozawa
ac874764d5 MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 348208014b2023894dc416dcd733a9ccdafc6e25)
 2015-03-05 14:13:13 +09:00
Karthik Kambatla
f4d6c5e337 MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha) 
(cherry picked from commit c66c3ac6bf9f63177279feec3f2917e4b882e2bc)
 2015-03-04 14:47:06 -08:00
Junping Du
a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de94028f1e10ca59ce23e963e488fe566909)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Tsuyoshi Ozawa
b1e7f9c588 MAPREDUCE-6268. Fix typo in Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit 742f9d90c00f823ad7fea7e79702fcf238fa5721)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md
 2015-03-03 16:24:27 +09:00
Akira Ajisaka
f7a724ca9e HADOOP-10976. Backport "moving the source code of hadoop-tools docs to the directory under hadoop-tools" to branch-2. Contributed by Masatake Iwasaki. 
(cherry picked from commit 9112f093cde0b8242324aa5267c9beb21c38bf6b)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-02-28 17:06:58 -08:00
Konstantin V Shvachko
9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Akira Ajisaka
8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7dcb71b59d85b05d0a6da88195ea9902)
 2015-02-26 15:06:26 -08:00
Akira Ajisaka
5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a6747d5d1a8749750698a1ae2f7b68414)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Wangda Tan
cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 
(cherry picked from commit d49ae725d5fa3eecf879ac42c42a368dd811f854)
 2015-02-19 11:01:47 -08:00
Plamen Jeliazkov
32d215dc89 MAPREDUCE-6228. Add truncate operation to SLive. Contributed by Plamen Jeliazkov. 2015-02-19 00:14:11 -08:00
Jason Lowe
386e62ace3 MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 4981d082d4f3c82d1c2c900c7488b83bf20301cc)
 2015-02-18 19:28:45 +00:00
Tsuyoshi Ozawa
9523b52da5 MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure conditions. Contributed by Devaraj K. 
(cherry picked from commit 7c782047c609b29178945bd566a0d162e64dbfdb)
 2015-02-18 15:46:06 +09:00
Junping Du
1d91daaae9 MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena 
(cherry picked from commit 814afa46efef201cb782072432fc744e1cb9c463)
 2015-02-16 09:39:20 -08:00
Tsuyoshi Ozawa
7011125778 MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R. 
(cherry picked from commit 3338f6d90795dafd445891cbd2f4d46b53012382)
 2015-02-15 10:10:11 +09:00
Tsuyoshi Ozawa
7386323735 MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi. 
(cherry picked from commit ba3c80a5ca8fefae9d0e67232f9973e1a0458f58)
 2015-02-13 16:11:16 +09:00
Tsuyoshi Ozawa
9d57392f5b MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu. 
(cherry picked from commit 9b0ba59b8284fae132535fbca5ce372d7a6c38c0)
 2015-02-13 02:28:19 +09:00
Tsuyoshi Ozawa
941a923514 MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. 
(cherry picked from commit ac8d52bf50bba1a29489ee75fd90717d8a2b0cc9)
 2015-02-12 20:37:14 +09:00
Tsuyoshi Ozawa
9129b3415c MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K. 
(cherry picked from commit b42d09eb62bd1725d70da59f1a6fdac83cea82d1)
 2015-02-12 20:08:47 +09:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
 2015-02-12 13:47:19 +05:30
Tsuyoshi Ozawa
b1aad1d941 MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. 
(cherry picked from commit 241336ca2b7cf97d7e0bd84dbe0542b72f304dc9)
 2015-02-10 03:53:15 +09:00
yliu
6933975143 MAPREDUCE-6227. DFSIO for truncate. (shv via yliu) 2015-02-08 02:42:10 +08:00
Robert Kanter
46a2027226 MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk (zxu via rkanter) 
(cherry picked from commit e2ee2ff7d7ca429487d7e3883daedffbb269ebd4)
 2015-02-05 14:31:44 -08:00
Jason Lowe
eb927a1d6b MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal 
(cherry picked from commit e1990ab4275b9092cae3573334791f7b2f8df7e5)
 2015-02-05 20:14:13 +00:00
Jason Lowe
2dc4af2b93 MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw) 
(cherry picked from commit fd57ab2002f97dcc83d455a5e0c770c8efde77a4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-02-05 16:40:09 +00:00
Akira Ajisaka
ad4b243586 MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(cherry picked from commit cc6bbfceae1cddfae6a3892cb7e7104531a689be)
 2015-02-04 17:46:39 -08:00
Akira Ajisaka
8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Robert Kanter
a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b3fea8b418b3526c15022c8a9fedd56)
 2015-02-02 13:52:29 -08:00
Akira Ajisaka
1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110af9989829b8a8caa699658a2ba01607)
 2015-01-30 02:50:46 +09:00
Jian He
43b3b43cea MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe 
(cherry picked from commit cff05bff1fe24628677d41a0d537f2c383b44faf)
 2015-01-28 15:52:11 -08:00
Allen Wittenauer
874e38d81f HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw) 2015-01-27 14:32:21 -08:00
Jason Lowe
2cf58ca5a6 MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe 
(cherry picked from commit 56b7ec71a69820ae12b4b9e2eb04b7368f721dbf)
 2015-01-26 16:30:10 +00:00
Allen Wittenauer
683cc18bba HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw) 2015-01-21 08:46:19 -08:00
Akira Ajisaka
2e5eef9da2 MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo) 
(cherry picked from commit d336d136785ef1e49e0a110a1b9f0d1824829877)
 2015-01-14 17:43:25 +09:00
Xuan
0562f382fa MAPREDUCE-6173. Document the configuration of deploying MR over 
distributed cache with enabling wired encryption at the same time.
Contributed by Junping Du.

(cherry picked from commit 10ac5abb867ead7131f3c76b27e48f7908b1d7f1)
 2015-01-13 10:08:24 -08:00
Brandon Li
d847363821 HADOOP-9992. Modify the NN loadGenerator to optionally run as a MapReduce job. Contributed by Akshay Radia 
(cherry picked from commit 09d31bc63035a71620657ab3f787047799bdcd14)
 2015-01-09 17:25:19 -08:00
Tsuyoshi Ozawa
132fd6ba58 HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa) 
(cherry picked from commit a6ed4894b518351bf1b3290e725a475570a21296)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/test/java/org/apache/hadoop/mapred/nativetask/kvtest/KVJob.java
 2015-01-08 15:00:02 +09:00
Jason Lowe
c22c27068e MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed by Jens Rabe 
(cherry picked from commit a0aeed100b839d5c122974bd0ea12f9168ee36a5)
 2015-01-06 23:40:42 +00:00
Jason Lowe
2b408d8dc7 HADOOP-11445. Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split. Contributed by Ankit Kamboj 
(cherry picked from commit d02fb53750bc592c23ba470ae82eb6f47d9a00ec)
 2015-01-06 21:20:31 +00:00
Harsh J
6d65e74670 MAPREDUCE-6149. Document override log4j.properties in MR job. Contributed by Junping Du. 
(cherry picked from commit 6621c3598e22279cde11eca73cfb5619a8bc8dee)
 2014-12-30 23:13:19 +05:30
Jian He
9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821d5bc4267f69d14220757f01cd55715)
 2014-12-19 16:56:55 -08:00
Robert Kanter
f9341c1e2c MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter) 
(cherry picked from commit 390a7c12f543b2c94a74f08d6d2a28410472043a)
 2014-12-19 15:44:01 -08:00
Karthik Kambatla
e6c73a1c1a HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle. (Wei Yan via kasha) 
(cherry picked from commit 6f1e3667cff3e95f33ba14a3cb7d1e99518d502a)
 2014-12-19 15:07:39 -08:00
Robert Kanter
22b38cf29a MAPREDUCE-6045. need close the DataInputStream after open it in TestMapReduce.java (zxu via rkanter) 
(cherry picked from commit d9e4d67d18811e16d5b0a76ea8228d333ded195f)
 2014-12-19 12:00:00 -08:00
Gera Shegalov
7c7bccfc31 MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera) 
(cherry picked from commit af006937e8ba82f98f468dc7375fe89c2e0a7912)
 2014-12-15 19:21:46 -08:00
Jian He
d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367de94689770f57c64bea3b244d7755f6.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9ec8347612b7aeeb5a3a8b7191278790)
 2014-12-15 11:08:44 -08:00
Harsh J
a3fa1f0761 MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena. 
(cherry picked from commit 298d09c9b583088f364038adcb1edf1eb1c2c196)
 2014-12-15 14:27:49 +05:30
Devaraj K
25f4a0193b MAPREDUCE-6046. Change the class name for logs in RMCommunicator. 
Contributed by Sahil Takiar.

(cherry picked from commit 0bd022911013629a8c9e7357fae8cf4399d7a1e3)
 2014-12-12 11:44:00 +05:30