Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,966 Commits 326 Branches 371 Tags
Commit Graph

402 Commits

Author SHA1 Message Date
Jason Lowe
ca6d5b4b92 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3aff3b274c439f72a8bb86094c1ab0e8)
 2015-09-15 19:45:06 +00:00
Hitesh Shah
f6b7dc24f0 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
(cherry picked from commit 55b794e7fa205df655c19bbfe1de99091fa9dc64)
 2015-09-10 10:55:23 -07:00
Vinod Kumar Vavilapalli
a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Xuan
1ad0d43fb4 Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 2032e8d1a0c6abe154a64df69b0124cdae9f78e0.
 2015-06-01 11:29:36 -07:00
Zhijie Shen
2032e8d1a0 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 09:39:49 -07:00
Vinod Kumar Vavilapalli
d9d7bbd99b Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e00acca405e085724c89cda016cf7442)
(cherry picked from commit 87862970f15e980eaf0b25e3eaf507becf349ae5)
 2015-05-14 17:00:56 -07:00
Vinod Kumar Vavilapalli
0e68e304f7 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e9ba48cdb82e5e6e5c10ce5b1407308)
 2015-05-12 12:16:16 -07:00
Robert Kanter
0eae3bc0f6 MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter) 
(cherry picked from commit d50e8f09287deeb51012d08e326a2ed71a6da869)
 2015-04-23 17:47:00 -07:00
Vinod Kumar Vavilapalli
a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli
ec123abdc9 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:30:04 -07:00
Vinod Kumar Vavilapalli
1c050bd135 Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-31 11:28:51 -07:00
Akira Ajisaka
da6228e320 MAPREDUCE-6292. Use org.junit package instead of junit.framework in TestCombineFileInputFormat. (aajisaka) 
(cherry picked from commit c770df49b4c9d5ac5ddaa4ff2ec541e4bd1a8e56)
(cherry picked from commit 2e509908eed4cf2e365ca1fd94aa2ff3d59193c1)
 2015-03-25 19:02:44 +09:00
Tsuyoshi Ozawa
01460c0bc9 MAPREDUCE-6285. ClientServiceDelegate should not retry upon AuthenticationException. Contributed by Jonathan Eagles. 
(cherry picked from commit 4170c99147b0cb6d561ff626cea140e0a061b314)
 2015-03-25 00:57:26 +09:00
Tsuyoshi Ozawa
834e9bc3b2 MAPREDUCE-4742. Fix typo in nnbench#displayUsage. Contributed by Liang Xie. 
(cherry picked from commit 20b8ee1350e62d1b21c951e653302b6e6a8e4f7e)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-03-14 16:58:25 +09:00
Jonathan Eagles
e4d8dddb49 YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676abb2bb500a48b3a0c0809d2a807ab235)
 2015-03-13 12:12:40 -05:00
Junping Du
9d1f67f2f2 Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne. 
(cherry picked from commit d39bc903a0069a740744bafe10e506e452ed7018)
(cherry picked from commit dbcdcb0d3ccc67db12104137d31cfc01cf6825ce)
 2015-03-10 06:24:34 -07:00
Tsuyoshi Ozawa
28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72dbed6fd1f8cae1e12973252462d6ee15)
 2015-03-09 19:54:45 +09:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Wangda Tan
cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 
(cherry picked from commit d49ae725d5fa3eecf879ac42c42a368dd811f854)
 2015-02-19 11:01:47 -08:00
Plamen Jeliazkov
32d215dc89 MAPREDUCE-6228. Add truncate operation to SLive. Contributed by Plamen Jeliazkov. 2015-02-19 00:14:11 -08:00
Tsuyoshi Ozawa
9523b52da5 MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure conditions. Contributed by Devaraj K. 
(cherry picked from commit 7c782047c609b29178945bd566a0d162e64dbfdb)
 2015-02-18 15:46:06 +09:00
yliu
6933975143 MAPREDUCE-6227. DFSIO for truncate. (shv via yliu) 2015-02-08 02:42:10 +08:00
Robert Kanter
46a2027226 MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk (zxu via rkanter) 
(cherry picked from commit e2ee2ff7d7ca429487d7e3883daedffbb269ebd4)
 2015-02-05 14:31:44 -08:00
Akira Ajisaka
8d98e5c960 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 
(cherry picked from commit bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)

Conflicts:
	hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
 2015-02-03 14:35:15 -08:00
Brandon Li
d847363821 HADOOP-9992. Modify the NN loadGenerator to optionally run as a MapReduce job. Contributed by Akshay Radia 
(cherry picked from commit 09d31bc63035a71620657ab3f787047799bdcd14)
 2015-01-09 17:25:19 -08:00
Robert Kanter
f9341c1e2c MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter) 
(cherry picked from commit 390a7c12f543b2c94a74f08d6d2a28410472043a)
 2014-12-19 15:44:01 -08:00
Robert Kanter
22b38cf29a MAPREDUCE-6045. need close the DataInputStream after open it in TestMapReduce.java (zxu via rkanter) 
(cherry picked from commit d9e4d67d18811e16d5b0a76ea8228d333ded195f)
 2014-12-19 12:00:00 -08:00
Jian He
d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367de94689770f57c64bea3b244d7755f6.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9ec8347612b7aeeb5a3a8b7191278790)
 2014-12-15 11:08:44 -08:00
Harsh J
6e06a51e30 MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml. Contributed by James Carman. (harsh) 
(cherry picked from commit cb99f43305bd1577d4ba9527d237ac6cdb9ae730)
 2014-12-11 09:46:08 +05:30
Jason Lowe
db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01ffea1cacf1fa9cbbf73c3f2904d981)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Zhijie Shen
d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367de94689770f57c64bea3b244d7755f6)
 2014-12-01 19:08:31 -08:00
Arun C. Murthy
adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy
175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408c6613d2c9aa89203cfa3110d830538)
 2014-11-09 15:03:59 -08:00
Jason Lowe
c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov 
(cherry picked from commit 10f9f5101c44be7c675a44ded4aad212627ecdee)
 2014-11-06 15:12:06 +00:00
Jian He
944fb1c3a9 MAPREDUCE-6048. Fixed TestJavaSerialization failure. Contributed by Varun Vasudev 
(cherry picked from commit 73068f677bc45029743ba2e0b3c0256a8069c13e)
 2014-11-04 20:20:33 -08:00
Haohui Mai
fa92b40f09 HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN. Contributed by Robert Kanter. 2014-11-04 17:53:00 -08:00
Zhijie Shen
c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 
(cherry picked from commit ed63b116465290fdb0acdf89170025f47b307599)
 2014-11-01 00:49:38 -07:00
Vinod Kumar Vavilapalli
0ba8d8c19c YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16ebd06bb3aabeb523cd17259008e243ef)
 2014-10-30 23:02:06 -07:00
Vinod Kumar Vavilapalli
063bb0508b MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen. 
(cherry picked from commit 3f1441245db047b5b7289fe06c4c3306fe8189af)
 2014-10-29 09:58:50 -07:00
Zhijie Shen
52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)
 2014-10-27 21:08:38 -07:00
Zhijie Shen
733380fa70 MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 6b2f11b54bc679b0715fe66bd129e340e8c61c5c)
 2014-10-27 20:41:02 -07:00
Jason Lowe
8885b75d78 MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in client. Contributed by Rohith 
(cherry picked from commit 209b1699fcd150676d4cc47e8e817796086c1986)
 2014-10-17 19:53:17 +00:00
Jian He
f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe
025b6c13e9 MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov 
(cherry picked from commit f19771a24c2f90982cf6dec35889836a6146c968)
 2014-10-15 17:51:53 +00:00
Jason Lowe
942fb67a04 MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk. Contributed by Binglin Chang 
(cherry picked from commit 793dbf2465c01426a93bb26d08e377ffd3915340)
 2014-10-13 16:05:29 +00:00
Karthik Kambatla
7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
 2014-10-11 22:49:46 -07:00
cnauroth
f6278cbac5 MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit 777877d079ecb025c293deac741800800e413179)
 2014-10-09 22:38:25 -07:00
subru
30a370e705 MAPREDUCE-6103.Adding reservation APIs to MR resource manager delegate. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit aa92dd45f2d8c89a8a17ad2e4449aa3ff08bc53a)
(cherry picked from commit 3f282762d1afc916de9207d3adeda852ca344853)
 2014-10-06 10:21:06 -07:00
Jason Lowe
7b8f2f5b1d MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk. Contributed by Akira AJISAKA 
(cherry picked from commit 23fbfe36c31674907a26c8ecc6fd47888dfcc1ea)
 2014-09-29 18:14:11 +00:00
Jason Lowe
0cfff28f57 HADOOP-11049. javax package system class default is too broad. Contributed by Sangjin Lee 
(cherry picked from commit cf4631d7c9448205f4952341c071e1650e7922dc)
 2014-09-29 14:37:28 +00:00