522 Commits

Author SHA1 Message Date
Akira Ajisaka
48d6e994bf MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li.
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
(cherry picked from commit 51c458a5c4562c54ff147824816f7b44c4b9b8f6)
2015-04-16 10:16:48 -05:00
Akira Ajisaka
a41c1c80b0 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li."
This reverts commit 743c74a5730aa0bbd2acd6880f69a0133d4f6c7c.
2015-04-11 12:38:05 +09:00
Akira Ajisaka
743c74a573 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li.
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
(cherry picked from commit f267fe567cab1db6508412509743e80f59ea69da)
2015-04-11 01:30:41 +09:00
Vinod Kumar Vavilapalli
a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli
ec123abdc9 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)"
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
2015-03-31 11:30:04 -07:00
Jason Lowe
502c040c97 MAPREDUCE-6277. Job can post multiple history files if attempt loses connection to the RM. Contributed by Chang Li
(cherry picked from commit 30da99cbaf36aeef38a858251ce8ffa5eb657b38)
2015-03-18 19:32:04 +00:00
Tsuyoshi Ozawa
9f4ec4c540 MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu
(cherry picked from commit 9d38520c8e42530a817a7f69c9aa73a9ad40639c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

(cherry picked from commit 10745fcecd13bf4119e8766e80188697f78aeaa3)
2015-03-14 16:46:53 +09:00
Jonathan Eagles
e4d8dddb49 YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles)
(cherry picked from commit 8180e676abb2bb500a48b3a0c0809d2a807ab235)
2015-03-13 12:12:40 -05:00
Tsuyoshi Ozawa
28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa)
(cherry picked from commit b46f9e72dbed6fd1f8cae1e12973252462d6ee15)
2015-03-09 19:54:45 +09:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Tsuyoshi Ozawa
ac874764d5 MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula.
(cherry picked from commit 348208014b2023894dc416dcd733a9ccdafc6e25)
2015-03-05 14:13:13 +09:00
Junping Du
a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe.
(cherry picked from commit 4228de94028f1e10ca59ce23e963e488fe566909)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
2015-03-03 02:23:00 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula.
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
2015-02-24 11:33:05 -08:00
Tsuyoshi Ozawa
7386323735 MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi.
(cherry picked from commit ba3c80a5ca8fefae9d0e67232f9973e1a0458f58)
2015-02-13 16:11:16 +09:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
2015-02-12 13:47:19 +05:30
Jason Lowe
eb927a1d6b MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal
(cherry picked from commit e1990ab4275b9092cae3573334791f7b2f8df7e5)
2015-02-05 20:14:13 +00:00
Robert Kanter
a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter)
(cherry picked from commit 8acc5e9b4b3fea8b418b3526c15022c8a9fedd56)
2015-02-02 13:52:29 -08:00
Akira Ajisaka
1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka)
(cherry picked from commit 342efa110af9989829b8a8caa699658a2ba01607)
2015-01-30 02:50:46 +09:00
Jian He
43b3b43cea MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe
(cherry picked from commit cff05bff1fe24628677d41a0d537f2c383b44faf)
2015-01-28 15:52:11 -08:00
Akira Ajisaka
2e5eef9da2 MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo)
(cherry picked from commit d336d136785ef1e49e0a110a1b9f0d1824829877)
2015-01-14 17:43:25 +09:00
Jian He
d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai."
This reverts commit 51af8d367de94689770f57c64bea3b244d7755f6.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9ec8347612b7aeeb5a3a8b7191278790)
2014-12-15 11:08:44 -08:00
Devaraj K
25f4a0193b MAPREDUCE-6046. Change the class name for logs in RMCommunicator.
Contributed by Sahil Takiar.

(cherry picked from commit 0bd022911013629a8c9e7357fae8cf4399d7a1e3)
2014-12-12 11:44:00 +05:30
Haohui Mai
759179e631 HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories. Contributed by Li Lu. 2014-12-09 10:46:25 -08:00
Jason Lowe
db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov
(cherry picked from commit 03ab24aa01ffea1cacf1fa9cbbf73c3f2904d981)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
2014-12-03 17:12:47 +00:00
Zhijie Shen
d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai.
(cherry picked from commit 51af8d367de94689770f57c64bea3b244d7755f6)
2014-12-01 19:08:31 -08:00
Jason Lowe
b6d7b789bd MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith
(cherry picked from commit 0c588904f8b68cad219d2bd8e33081d5cae656e5)
2014-12-01 22:40:38 +00:00
Jonathan Eagles
6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles)
(cherry picked from commit c298a9a845f89317eb9efad332e6657c56736a4d)
2014-11-20 23:40:26 -06:00
Devaraj K
09df35e51d MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown
fails. Contributed by Rohith.
(cherry picked from commit 7250b0bf914a55d0fa4802834de7f1909f1b0d6b)
2014-11-18 22:07:26 +05:30
Jason Lowe
0b9a1a3e03 MAPREDUCE-6162. mapred hsadmin fails on a secure cluster. Contributed by Jason Lowe
(cherry picked from commit 09b3dee122ad20fae22ae8de74886dbf11c39343)
2014-11-17 23:01:16 +00:00
Arun C. Murthy
adfb830a2b Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT. 2014-11-09 19:19:02 -08:00
Arun C. Murthy
175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles.
(cherry picked from commit 43cd07b408c6613d2c9aa89203cfa3110d830538)
2014-11-09 15:03:59 -08:00
Jason Lowe
c0bbb72912 MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov
(cherry picked from commit 10f9f5101c44be7c675a44ded4aad212627ecdee)
2014-11-06 15:12:06 +00:00
Zhijie Shen
c4aaa4db6c MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du.
(cherry picked from commit ed63b116465290fdb0acdf89170025f47b307599)
2014-11-01 00:49:38 -07:00
Vinod Kumar Vavilapalli
063bb0508b MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen.
(cherry picked from commit 3f1441245db047b5b7289fe06c4c3306fe8189af)
2014-10-29 09:58:50 -07:00
Zhijie Shen
52695d38fc MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter.
(cherry picked from commit 971e91c8c03a23e4613ed3f071b4f982ee5a1b63)
2014-10-27 21:08:38 -07:00
Zhijie Shen
733380fa70 MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter.
(cherry picked from commit 6b2f11b54bc679b0715fe66bd129e340e8c61c5c)
2014-10-27 20:41:02 -07:00
Zhijie Shen
3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He.
(cherry picked from commit 0f3b6900be1a3b2e4624f31f84656f4a32dadce9)
2014-10-23 22:03:30 -07:00
Jian He
f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Jason Lowe
e17e3bdbc4 MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai
(cherry picked from commit bbe80cdc7b37cae50e46eaa00e75240d4a2aac68)
2014-10-13 15:09:26 +00:00
Karthik Kambatla
7bfd9e068d MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
2014-10-11 22:49:46 -07:00
Jason Lowe
979b915ecc MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai
(cherry picked from commit 2e789eb2261fa607489d8ae630fb77e8147d0266)
2014-10-07 14:11:48 +00:00
Zhijie Shen
b8d645539d MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev. 2014-09-22 10:41:55 -07:00
Jason Lowe
f4534746fb MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs. Contributed by Gera Shegalov
(cherry picked from commit 7039b98e1c459e9e0d8caa28cdaa2868e2bde2eb)
2014-09-22 15:22:58 +00:00
Zhijie Shen
9333ee3068 YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He.
(cherry picked from commit 6fe5c6b746a40019b9a43676c33efec0f971c4b9)
2014-09-18 22:02:53 -07:00
Vinod Kumar Vavilapalli
5cdb24d4b1 YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong.
(cherry picked from commit 14e2639fd0d53f7e0b58f2f4744af44983d4e867)
2014-09-13 18:05:02 -07:00
Jason Lowe
0b3f918b3b MAPREDUCE-6071. JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs. Contributed by Tsuyoshi OZAWA
(cherry picked from commit 45efc966ee1a1bda0271464dcee2564862d53b1f)
2014-09-05 13:48:16 +00:00
Chris Douglas
4f95017602 Fix typos in log messages. Contributed by Ray Chiang 2014-08-28 16:30:13 -07:00
Zhijie Shen
7cad25ec94 MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir creation on Windows. Contributed by Zhijie Shen.
svn merge --ignore-ancestry -c 1619863 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619864 13f79535-47bb-0310-9956-ffa450edef68
2014-08-22 17:08:17 +00:00
Jason Darrell Lowe
e06838038e svn merge -c 1618691 FIXES: MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2. Contributed by chang li
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618692 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 18:19:09 +00:00
Zhijie Shen
570183a437 MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before concluding the failure by configuration. Contributed by Yunjiong Zhao.
svn merge --ignore-ancestry -c 1618677 ../../trunk/


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1618679 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 17:59:32 +00:00