Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,237 Commits 326 Branches 371 Tags
Commit Graph

1305 Commits

Author SHA1 Message Date
Jonathan Eagles
6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845f89317eb9efad332e6657c56736a4d)
 2014-11-20 23:40:26 -06:00
Karthik Kambatla
3a4211ec2c MAPREDUCE-6169. MergeQueue should release reference to the current item from key and value at the end of the iteration to save memory. (Zhihai Xu via kasha) 
(cherry picked from commit 90194ca1cbd695d48c3705121c2ac9a8554578a2)
 2014-11-20 15:37:18 -08:00
Devaraj K
09df35e51d MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown 
fails. Contributed by Rohith.
(cherry picked from commit 7250b0bf914a55d0fa4802834de7f1909f1b0d6b)
 2014-11-18 22:07:26 +05:30
Jason Lowe
0b9a1a3e03 MAPREDUCE-6162. mapred hsadmin fails on a secure cluster. Contributed by Jason Lowe 
(cherry picked from commit 09b3dee122ad20fae22ae8de74886dbf11c39343)
 2014-11-17 23:01:16 +00:00
Ravi Prakash
45692effe4 MAPREDUCE-5918. LineRecordReader can return the same decompressor to CodecPool multiple times (Sergey Murylev via raviprak) 2014-11-14 03:48:28 -08:00
Jason Lowe
e62a5f9d0a MAPREDUCE-6156. Fetcher - connect() doesn't handle connection refused correctly. Contributed by Junping Du 
(cherry picked from commit 177e8090f5809beb3ebcb656cd0affbb3f487de8)
 2014-11-13 15:43:28 +00:00
cnauroth
69a7780ee5 HADOOP-7984. Add hadoop --loglevel option to change log level. Contributed by Aikira AJISAKA. 2014-11-12 21:44:05 -08:00
Arun C. Murthy
adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy
175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408c6613d2c9aa89203cfa3110d830538)
 2014-11-09 15:03:59 -08:00
Arun C. Murthy
1f5bad04ce HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid issues with newer versions of guava in applications. Contributed by Christopher Tubbs. 
(cherry picked from commit 6caa8100d5d2547e34356dc279fd5e65b81a925a)
 2014-11-09 05:23:10 -08:00
Kihwal Lee
a19123df9a MAPREDUCE-5958. Wrong reduce task progress if map output is compressed. Contributed by Emilio Coppa and Jason Lowe. 
(cherry picked from commit 8f701ae07a0b1dc70b8e1eb8d4a5c35c0a1e76da)
 2014-11-06 15:56:10 -06:00
Jason Lowe
c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov 
(cherry picked from commit 10f9f5101c44be7c675a44ded4aad212627ecdee)
 2014-11-06 15:12:06 +00:00
Jian He
944fb1c3a9 MAPREDUCE-6048. Fixed TestJavaSerialization failure. Contributed by Varun Vasudev 
(cherry picked from commit 73068f677bc45029743ba2e0b3c0256a8069c13e)
 2014-11-04 20:20:33 -08:00
Haohui Mai
fa92b40f09 HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN. Contributed by Robert Kanter. 2014-11-04 17:53:00 -08:00
Haohui Mai
aa09c4c342 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 2014-11-04 14:52:51 -08:00
Zhijie Shen
c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 
(cherry picked from commit ed63b116465290fdb0acdf89170025f47b307599)
 2014-11-01 00:49:38 -07:00
Vinod Kumar Vavilapalli
0ba8d8c19c YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16ebd06bb3aabeb523cd17259008e243ef)
 2014-10-30 23:02:06 -07:00
Vinod Kumar Vavilapalli
063bb0508b MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen. 
(cherry picked from commit 3f1441245db047b5b7289fe06c4c3306fe8189af)
 2014-10-29 09:58:50 -07:00
Karthik Kambatla
160aebcae2 HADOOP-11243. SSLFactory shouldn't allow SSLv3. (Wei Yan via kasha) 
(cherry picked from commit 3c5f5af1184e85158dec962df0b0bc2be8d0d1e3)
 2014-10-28 18:08:00 -07:00
Zhijie Shen
52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)
 2014-10-27 21:08:38 -07:00
Zhijie Shen
733380fa70 MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 6b2f11b54bc679b0715fe66bd129e340e8c61c5c)
 2014-10-27 20:41:02 -07:00
Zhijie Shen
3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be1a3b2e4624f31f84656f4a32dadce9)
 2014-10-23 22:03:30 -07:00
Jian He
1c235a4448 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfbf4cc91ef8e8616c7aafa9de006cde5)
 2014-10-22 15:58:26 -07:00
Jason Lowe
8885b75d78 MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in client. Contributed by Rohith 
(cherry picked from commit 209b1699fcd150676d4cc47e8e817796086c1986)
 2014-10-17 19:53:17 +00:00
Jian He
f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe
025b6c13e9 MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov 
(cherry picked from commit f19771a24c2f90982cf6dec35889836a6146c968)
 2014-10-15 17:51:53 +00:00
Jason Lowe
1179a0cb07 MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting for maps. Contributed by Siqi Li 
(cherry picked from commit b9edad64034a9c8a121ec2b37792c190ba561e26)
 2014-10-15 15:54:57 +00:00
Jason Lowe
942fb67a04 MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk. Contributed by Binglin Chang 
(cherry picked from commit 793dbf2465c01426a93bb26d08e377ffd3915340)
 2014-10-13 16:05:29 +00:00
Jason Lowe
e17e3bdbc4 MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai 
(cherry picked from commit bbe80cdc7b37cae50e46eaa00e75240d4a2aac68)
 2014-10-13 15:09:26 +00:00
Karthik Kambatla
7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
 2014-10-11 22:49:46 -07:00
cnauroth
f6278cbac5 MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. Contributed by Chris Nauroth. 
(cherry picked from commit 777877d079ecb025c293deac741800800e413179)
 2014-10-09 22:38:25 -07:00
cnauroth
85a2b90740 MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. Contributed by Chris Nauroth. 
(cherry picked from commit 684170d5efdc11bcd9967168f44a428d1a72af3c)
 2014-10-09 22:27:40 -07:00
Jason Lowe
979b915ecc MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 
(cherry picked from commit 2e789eb2261fa607489d8ae630fb77e8147d0266)
 2014-10-07 14:11:48 +00:00
subru
30a370e705 MAPREDUCE-6103.Adding reservation APIs to MR resource manager delegate. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit aa92dd45f2d8c89a8a17ad2e4449aa3ff08bc53a)
(cherry picked from commit 3f282762d1afc916de9207d3adeda852ca344853)
 2014-10-06 10:21:06 -07:00
Jason Lowe
7b8f2f5b1d MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk. Contributed by Akira AJISAKA 
(cherry picked from commit 23fbfe36c31674907a26c8ecc6fd47888dfcc1ea)
 2014-09-29 18:14:11 +00:00
Jason Lowe
0cfff28f57 HADOOP-11049. javax package system class default is too broad. Contributed by Sangjin Lee 
(cherry picked from commit cf4631d7c9448205f4952341c071e1650e7922dc)
 2014-09-29 14:37:28 +00:00
Jian He
a6879b606e MAPREDUCE-6087. Fixed wrong config name of MRJobConfig#MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS. Contributed by Akira AJISAKA 
(cherry picked from commit 5f16c98ad66b562760d5d947151a38f14875d21a)
 2014-09-26 18:06:19 -07:00
Allen Wittenauer
67d6fc3f01 MAPREDUCE-5945. Update the description of GenericOptionsParser -jt option (Akira AJISAKA via aw) 2014-09-26 14:26:36 -07:00
Allen Wittenauer
e5d284664b MAPREDUCE-5796. Use current version of the archive name in DistributedCacheDeploy document (Akira AJISAKA via aw) 2014-09-26 12:59:59 -07:00
Allen Wittenauer
fe8222998f MAPREDUCE-6073. Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags (Tsuyoshi OZAWA via aw) 2014-09-26 11:17:11 -07:00
Zhijie Shen
7b3ecc547f MAPREDUCE-5831. Make MR client ignore unknown counters received from AM. Contributed by Junping Du. 
(cherry picked from commit 662fc11ae7ccea9d05dfe54b2a794bb08cfd70a4)
 2014-09-25 23:46:57 -07:00
Allen Wittenauer
7ffb636a54 MAPREDUCE-6093. minor distcp doc edits (Charles Lamb via aw) 2014-09-25 11:33:01 -07:00
Jason Lowe
d5d9fd3255 MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2. Contributed by Mit Desai 
(cherry picked from commit 7af4c3888bf7be0822dc880170e5e3d09e9280f1)
 2014-09-24 16:10:28 +00:00
Zhijie Shen
b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Jason Lowe
f4534746fb MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs. Contributed by Gera Shegalov 
(cherry picked from commit 7039b98e1c459e9e0d8caa28cdaa2868e2bde2eb)
 2014-09-22 15:22:58 +00:00
Allen Wittenauer
0037b8f67b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:08:04 -07:00
Jason Lowe
bdb864e5e1 MAPREDUCE-6091. YARNRunner.getJobStatus() fails with ApplicationNotFoundException if the job rolled off the RM view. Contributed by Sangjin Lee 
(cherry picked from commit 951847ba94442b8d928435decfbea9ea20882912)
 2014-09-19 20:17:44 +00:00
Zhijie Shen
9333ee3068 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He. 
(cherry picked from commit 6fe5c6b746a40019b9a43676c33efec0f971c4b9)
 2014-09-18 22:02:53 -07:00
Karthik Kambatla
372bf54072 MAPREDUCE-6086. mapreduce.job.credentials.binary should allow all URIs. (Zhihai Xu via kasha) 
(cherry picked from commit 52945a33cc3fcb3b961cce0e7e3ca01291f2a223)
 2014-09-18 15:47:21 -07:00
Jason Lowe
44c7113133 MAPREDUCE-5891. Improved shuffle error handling across NM restarts. Contributed by Junping Du 
(cherry picked from commit 2c3da25fd718b3a9c1ed67f05b577975ae613f4e)
 2014-09-18 22:01:42 +00:00