Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,355 Commits 326 Branches 371 Tags
Commit Graph

545 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Wangda Tan
7cd0874dfd MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda) 
(cherry picked from commit 3164e7d83875aa6b7435d1dfe61ac280aa277f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-05-27 14:43:20 -07:00
cnauroth
6ed8989a6f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
 2015-05-27 14:21:15 -07:00
Tsuyoshi Ozawa
e694c3339e MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi. 
(cherry picked from commit 022f49d59e404678d0f322bb1b67208f3c17b7ef)
 2015-05-26 23:59:20 +09:00
Vinod Kumar Vavilapalli
87862970f1 Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e00acca405e085724c89cda016cf7442)
 2015-05-14 16:56:56 -07:00
Devaraj K
454236ec19 MAPREDUCE-5708. Duplicate String.format in 
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.

(cherry picked from commit 05ff54c66c49301c4ec2549704d9d459e784572c)
 2015-05-14 22:11:15 +05:30
Vinayakumar B
cda1962996 MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad) 
(cherry picked from commit 360dff5903085c3c7f02ccf9f17d71a842275e48)
 2015-05-12 12:50:44 +05:30
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Karthik Kambatla
52469b10dd MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha) 
(cherry picked from commit 1773aac780585871072960a5863af461e112a030)
 2015-05-09 14:43:57 -07:00
Karthik Kambatla
e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79a7eba6818313960624380bacfe0bb2)
 2015-05-09 14:35:13 -07:00
Junping Du
39c188449e MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong. 
(cherry picked from commit df36ad0a08261b03c250b6f745b27e5f83e4286e)
 2015-05-09 06:12:01 -07:00
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
5cc036d473 MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit ecfa05227489724f12775b94a9f9184b66e865af)
 2015-05-08 18:17:35 +09:00
Tsuyoshi Ozawa
1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b3e497fa1aaaeb85ba055f87d9849f7)
 2015-05-08 16:00:41 +09:00
Jason Lowe
976a3c1f9d MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne 
(cherry picked from commit f30065c8b6099372f57015b505434120fe83c2b0)
 2015-05-07 22:06:54 +00:00
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Robert Kanter
bd207b6cc0 MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:47:52 -07:00
Xuan
02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04f712b7594c4e6e7ce50fd314f7c342)
 2015-05-05 10:45:18 -07:00
Jason Lowe
81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:16 +00:00
Devaraj K
c3d083a9ff MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9d0827b238fadd260c1dd2889531fd4)
 2015-04-30 12:36:00 +05:30
Vinod Kumar Vavilapalli
32dc13d907 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8)
 2015-04-27 14:59:57 -07:00
Gera Shegalov
fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b462412bbadad4a1a13ed4c597927b0cf188)
 2015-04-24 17:38:16 -07:00
Gera Shegalov
6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc647074f0bf1cc73c3029f1352f51d5)
 2015-04-21 14:21:56 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka
4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35094f3c2ac216992d861729251b6a3d.
 2015-04-11 12:37:44 +09:00
Akira Ajisaka
3de0bf9a35 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
 2015-04-11 01:31:44 +09:00
Hitesh Shah
55b794e7fa YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
 2015-04-08 14:14:06 -07:00
Jason Lowe
24d8c6f355 MAPREDUCE-6277. Job can post multiple history files if attempt loses connection to the RM. Contributed by Chang Li 
(cherry picked from commit 30da99cbaf36aeef38a858251ce8ffa5eb657b38)
 2015-03-18 19:31:32 +00:00
Tsuyoshi Ozawa
10745fcecd MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu 
(cherry picked from commit 9d38520c8e42530a817a7f69c9aa73a9ad40639c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-14 16:45:23 +09:00
Jonathan Eagles
44aedad5dd YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676abb2bb500a48b3a0c0809d2a807ab235)
 2015-03-13 12:05:21 -05:00
Tsuyoshi Ozawa
b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli
d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa
ccde4aed18 MAPREDUCE-5657. Backport '[JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments.' Contributed by Akira AJISAKA. 2015-03-09 12:00:04 +09:00
Tsuyoshi Ozawa
ac874764d5 MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 348208014b2023894dc416dcd733a9ccdafc6e25)
 2015-03-05 14:13:13 +09:00
Junping Du
a9613bcc94 Ability to limit running map and reduce tasks. Contributed by Jason Lowe. 
(cherry picked from commit 4228de94028f1e10ca59ce23e963e488fe566909)

Conflicts:
    hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-03-03 02:23:00 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Tsuyoshi Ozawa
7386323735 MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi. 
(cherry picked from commit ba3c80a5ca8fefae9d0e67232f9973e1a0458f58)
 2015-02-13 16:11:16 +09:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
 2015-02-12 13:47:19 +05:30
Jason Lowe
eb927a1d6b MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal 
(cherry picked from commit e1990ab4275b9092cae3573334791f7b2f8df7e5)
 2015-02-05 20:14:13 +00:00
Robert Kanter
a3a7efe748 MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter) 
(cherry picked from commit 8acc5e9b4b3fea8b418b3526c15022c8a9fedd56)
 2015-02-02 13:52:29 -08:00
Akira Ajisaka
1081b4c4a7 HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka) 
(cherry picked from commit 342efa110af9989829b8a8caa699658a2ba01607)
 2015-01-30 02:50:46 +09:00
Jian He
43b3b43cea MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe 
(cherry picked from commit cff05bff1fe24628677d41a0d537f2c383b44faf)
 2015-01-28 15:52:11 -08:00
Akira Ajisaka
2e5eef9da2 MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo) 
(cherry picked from commit d336d136785ef1e49e0a110a1b9f0d1824829877)
 2015-01-14 17:43:25 +09:00
Jian He
d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367de94689770f57c64bea3b244d7755f6.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9ec8347612b7aeeb5a3a8b7191278790)
 2014-12-15 11:08:44 -08:00
Devaraj K
25f4a0193b MAPREDUCE-6046. Change the class name for logs in RMCommunicator. 
Contributed by Sahil Takiar.

(cherry picked from commit 0bd022911013629a8c9e7357fae8cf4399d7a1e3)
 2014-12-12 11:44:00 +05:30
Haohui Mai
759179e631 HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories. Contributed by Li Lu. 2014-12-09 10:46:25 -08:00
Jason Lowe
db723a8499 MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov 
(cherry picked from commit 03ab24aa01ffea1cacf1fa9cbbf73c3f2904d981)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2014-12-03 17:12:47 +00:00
Zhijie Shen
d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367de94689770f57c64bea3b244d7755f6)
 2014-12-01 19:08:31 -08:00
Jason Lowe
b6d7b789bd MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith 
(cherry picked from commit 0c588904f8b68cad219d2bd8e33081d5cae656e5)
 2014-12-01 22:40:38 +00:00
Jonathan Eagles
6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845f89317eb9efad332e6657c56736a4d)
 2014-11-20 23:40:26 -06:00