Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,013 Commits 326 Branches 371 Tags
Commit Graph

527 Commits

Author SHA1 Message Date
Jason Lowe
81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:16 +00:00
Devaraj K
c3d083a9ff MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9d0827b238fadd260c1dd2889531fd4)
 2015-04-30 12:36:00 +05:30
Vinod Kumar Vavilapalli
32dc13d907 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8)
 2015-04-27 14:59:57 -07:00
Gera Shegalov
fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b462412bbadad4a1a13ed4c597927b0cf188)
 2015-04-24 17:38:16 -07:00
Gera Shegalov
6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc647074f0bf1cc73c3029f1352f51d5)
 2015-04-21 14:21:56 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka
4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35094f3c2ac216992d861729251b6a3d.
 2015-04-11 12:37:44 +09:00
Akira Ajisaka
3de0bf9a35 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
 2015-04-11 01:31:44 +09:00
Hitesh Shah
55b794e7fa YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
 2015-04-08 14:14:06 -07:00
Jason Lowe
24d8c6f355 MAPREDUCE-6277. Job can post multiple history files if attempt loses connection to the RM. Contributed by Chang Li 
(cherry picked from commit 30da99cbaf36aeef38a858251ce8ffa5eb657b38)
 2015-03-18 19:31:32 +00:00
Tsuyoshi Ozawa
10745fcecd MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu 
(cherry picked from commit 9d38520c8e42530a817a7f69c9aa73a9ad40639c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-14 16:45:23 +09:00
Jonathan Eagles
44aedad5dd YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676abb2bb500a48b3a0c0809d2a807ab235)
 2015-03-13 12:05:21 -05:00
Tsuyoshi Ozawa
b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli
d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Tsuyoshi Ozawa
ac874764d5 MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 348208014b2023894dc416dcd733a9ccdafc6e25)
 2015-03-05 14:13:13 +09:00
Junping Du
a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de94028f1e10ca59ce23e963e488fe566909)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Tsuyoshi Ozawa
7386323735 MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi. 
(cherry picked from commit ba3c80a5ca8fefae9d0e67232f9973e1a0458f58)
 2015-02-13 16:11:16 +09:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
 2015-02-12 13:47:19 +05:30
Jason Lowe
eb927a1d6b MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal 
(cherry picked from commit e1990ab4275b9092cae3573334791f7b2f8df7e5)
 2015-02-05 20:14:13 +00:00
Robert Kanter
a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b3fea8b418b3526c15022c8a9fedd56)
 2015-02-02 13:52:29 -08:00
Akira Ajisaka
1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110af9989829b8a8caa699658a2ba01607)
 2015-01-30 02:50:46 +09:00
Jian He
43b3b43cea MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe 
(cherry picked from commit cff05bff1fe24628677d41a0d537f2c383b44faf)
 2015-01-28 15:52:11 -08:00
Akira Ajisaka
2e5eef9da2 MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo) 
(cherry picked from commit d336d136785ef1e49e0a110a1b9f0d1824829877)
 2015-01-14 17:43:25 +09:00
Jian He
d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367de94689770f57c64bea3b244d7755f6.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9ec8347612b7aeeb5a3a8b7191278790)
 2014-12-15 11:08:44 -08:00
Devaraj K
25f4a0193b MAPREDUCE-6046. Change the class name for logs in RMCommunicator. 
Contributed by Sahil Takiar.

(cherry picked from commit 0bd022911013629a8c9e7357fae8cf4399d7a1e3)
 2014-12-12 11:44:00 +05:30
Haohui Mai
759179e631 HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories. Contributed by Li Lu. 2014-12-09 10:46:25 -08:00
Jason Lowe
db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01ffea1cacf1fa9cbbf73c3f2904d981)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Zhijie Shen
d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367de94689770f57c64bea3b244d7755f6)
 2014-12-01 19:08:31 -08:00
Jason Lowe
b6d7b789bd MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith 
(cherry picked from commit 0c588904f8b68cad219d2bd8e33081d5cae656e5)
 2014-12-01 22:40:38 +00:00
Jonathan Eagles
6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845f89317eb9efad332e6657c56736a4d)
 2014-11-20 23:40:26 -06:00
Devaraj K
09df35e51d MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown 
fails. Contributed by Rohith.
(cherry picked from commit 7250b0bf914a55d0fa4802834de7f1909f1b0d6b)
 2014-11-18 22:07:26 +05:30
Jason Lowe
0b9a1a3e03 MAPREDUCE-6162. mapred hsadmin fails on a secure cluster. Contributed by Jason Lowe 
(cherry picked from commit 09b3dee122ad20fae22ae8de74886dbf11c39343)
 2014-11-17 23:01:16 +00:00
Arun C. Murthy
adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy
175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408c6613d2c9aa89203cfa3110d830538)
 2014-11-09 15:03:59 -08:00
Jason Lowe
c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov 
(cherry picked from commit 10f9f5101c44be7c675a44ded4aad212627ecdee)
 2014-11-06 15:12:06 +00:00
Zhijie Shen
c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du. 
(cherry picked from commit ed63b116465290fdb0acdf89170025f47b307599)
 2014-11-01 00:49:38 -07:00
Vinod Kumar Vavilapalli
063bb0508b MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen. 
(cherry picked from commit 3f1441245db047b5b7289fe06c4c3306fe8189af)
 2014-10-29 09:58:50 -07:00
Zhijie Shen
52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)
 2014-10-27 21:08:38 -07:00
Zhijie Shen
733380fa70 MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter. 
(cherry picked from commit 6b2f11b54bc679b0715fe66bd129e340e8c61c5c)
 2014-10-27 20:41:02 -07:00
Zhijie Shen
3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be1a3b2e4624f31f84656f4a32dadce9)
 2014-10-23 22:03:30 -07:00
Jian He
f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe
e17e3bdbc4 MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai 
(cherry picked from commit bbe80cdc7b37cae50e46eaa00e75240d4a2aac68)
 2014-10-13 15:09:26 +00:00
Karthik Kambatla
7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha) 
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
 2014-10-11 22:49:46 -07:00
Jason Lowe
979b915ecc MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai 
(cherry picked from commit 2e789eb2261fa607489d8ae630fb77e8147d0266)
 2014-10-07 14:11:48 +00:00
Zhijie Shen
b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Jason Lowe
f4534746fb MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs. Contributed by Gera Shegalov 
(cherry picked from commit 7039b98e1c459e9e0d8caa28cdaa2868e2bde2eb)
 2014-09-22 15:22:58 +00:00
Zhijie Shen
9333ee3068 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He. 
(cherry picked from commit 6fe5c6b746a40019b9a43676c33efec0f971c4b9)
 2014-09-18 22:02:53 -07:00
Vinod Kumar Vavilapalli
5cdb24d4b1 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong. 
(cherry picked from commit 14e2639fd0d53f7e0b58f2f4744af44983d4e867)
 2014-09-13 18:05:02 -07:00