Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,226 Commits 326 Branches 371 Tags
Commit Graph

670 Commits

Author SHA1 Message Date
Jason Lowe
3c4c52a095 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
 2015-09-28 22:13:03 +00:00
Jian He
d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f0671f8ef30dbe7432079c18362b434)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He
c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10ee937a04e833a66e0a4642e776e510)
 2015-09-24 10:23:25 -07:00
Jian He
89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849fcb273d32e840d8562eaaee904af054)
 2015-09-24 10:23:13 -07:00
Tsuyoshi Ozawa
52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab0fae3839c9cc5d552aa0304444f956)
 2015-09-22 12:54:44 +09:00
Zhihai Xu
88a3a07636 MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 
(cherry picked from commit e3ace31e2b7e3a8dc6cf3dc044d5ef2a9aa12e29)
 2015-09-19 00:12:15 -07:00
Jason Lowe
f27f1ef44c MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 
(cherry picked from commit ee4ee6af6a5a6299d27462adb6944206039bbbae)
 2015-09-17 21:38:44 +00:00
Jason Lowe
4861fadb17 MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li 
(cherry picked from commit 3f82f582e51c514cc0f052c828c2f58c6e8927ad)
 2015-09-17 18:18:28 +00:00
Vinod Kumar Vavilapalli
9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Karthik Kambatla
20734320a1 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a1979e8f70f8355c096874921d368342ef)
 2015-08-15 00:53:13 -07:00
Karthik Kambatla
b826168173 MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab8d17e839a1ef0d7076efc78d28724a)
 2015-08-14 12:40:38 -07:00
Jason Lowe
508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010b72f58ddede5303f2b88404263d4bf)
 2015-07-31 18:19:52 +00:00
Zhihai Xu
9d40eead81 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 
(cherry picked from commit 93d50b782494af7eef980c4d596a59ff4e11646e)
 2015-07-30 23:12:30 -07:00
Chris Douglas
2a0e551cd8 MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:56:21 -07:00
Akira Ajisaka
0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d90e26616accee73b1f7743aab5df692)
 2015-07-13 17:53:27 +09:00
Jason Lowe
8a4970e845 MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li 
(cherry picked from commit eac1d1894354e90d314087af8e7fb168ddef9a3d)
 2015-07-01 18:02:17 +00:00
Jason Lowe
f3195eda27 MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li 
(cherry picked from commit 826715622e6937887a4e20b3ce327d7e2fd89009)
 2015-07-01 17:50:47 +00:00
Jason Lowe
df4e1e4965 MAPREDUCE-6376. Add avro binary support for jhist files. Contributed by Ray Chiang 
(cherry picked from commit 2ac87df578accb6e612f70ded76271cb5082ee10)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-07-01 16:07:54 +00:00
Jason Lowe
fdfd5be2b3 MAPREDUCE-6384. Add the last reporting reducer info for too many fetch failure diagnostics. Contributed by Chang Li 
(cherry picked from commit b6ba56457c6b01dae795c11d587c3fe3855ee707)
 2015-06-30 21:24:38 +00:00
Gera Shegalov
d8cc8f54dd MAPREDUCE-6408. Queue name and user name should be printed on the job page. (Siqi Li via gera) 
(cherry picked from commit 11ac84820746e43f1e7072e08f62ddf81713aeb2)
 2015-06-22 14:42:31 -07:00
Gera Shegalov
34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 
(cherry picked from commit 63d40d528654707b3f56619a4eb20e21cdeadc7e)
 2015-06-20 11:52:11 -07:00
Devaraj K
5c3070dc16 MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by 
Siqi Li and Gera Shegalov.

(cherry picked from commit 311a4179cc53a2754d1185b1cb10e6adb1b95d3d)
 2015-06-20 10:37:42 +05:30
Gera Shegalov
01991876ed MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery. (Brahma Reddy Battula via gera) 
(cherry picked from commit 10107243be66bae2212a2cd8575f9f5ade13fe9e)
 2015-06-19 02:29:31 -07:00
Devaraj K
5426e3e929 MAPREDUCE-6350. JobHistory doesn't support fully-functional search. 
Contributed by Siqi Li.
 2015-06-10 12:43:31 +05:30
Tsuyoshi Ozawa
c6cdecc6b3 MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang. 
(cherry picked from commit 6afe20a7a4bbfa30fce3e3c9873ad43201987998)
 2015-06-08 15:06:23 -07:00
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Wangda Tan
7cd0874dfd MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda) 
(cherry picked from commit 3164e7d83875aa6b7435d1dfe61ac280aa277f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-05-27 14:43:20 -07:00
cnauroth
6ed8989a6f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
 2015-05-27 14:21:15 -07:00
Tsuyoshi Ozawa
e694c3339e MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi. 
(cherry picked from commit 022f49d59e404678d0f322bb1b67208f3c17b7ef)
 2015-05-26 23:59:20 +09:00
Vinod Kumar Vavilapalli
87862970f1 Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e00acca405e085724c89cda016cf7442)
 2015-05-14 16:56:56 -07:00
Devaraj K
454236ec19 MAPREDUCE-5708. Duplicate String.format in 
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.

(cherry picked from commit 05ff54c66c49301c4ec2549704d9d459e784572c)
 2015-05-14 22:11:15 +05:30
Vinayakumar B
cda1962996 MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad) 
(cherry picked from commit 360dff5903085c3c7f02ccf9f17d71a842275e48)
 2015-05-12 12:50:44 +05:30
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Karthik Kambatla
52469b10dd MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha) 
(cherry picked from commit 1773aac780585871072960a5863af461e112a030)
 2015-05-09 14:43:57 -07:00
Karthik Kambatla
e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79a7eba6818313960624380bacfe0bb2)
 2015-05-09 14:35:13 -07:00
Junping Du
39c188449e MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong. 
(cherry picked from commit df36ad0a08261b03c250b6f745b27e5f83e4286e)
 2015-05-09 06:12:01 -07:00
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
5cc036d473 MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit ecfa05227489724f12775b94a9f9184b66e865af)
 2015-05-08 18:17:35 +09:00
Tsuyoshi Ozawa
1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b3e497fa1aaaeb85ba055f87d9849f7)
 2015-05-08 16:00:41 +09:00
Jason Lowe
976a3c1f9d MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne 
(cherry picked from commit f30065c8b6099372f57015b505434120fe83c2b0)
 2015-05-07 22:06:54 +00:00
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Robert Kanter
bd207b6cc0 MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:47:52 -07:00
Xuan
02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04f712b7594c4e6e7ce50fd314f7c342)
 2015-05-05 10:45:18 -07:00
Jason Lowe
81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:16 +00:00
Devaraj K
c3d083a9ff MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9d0827b238fadd260c1dd2889531fd4)
 2015-04-30 12:36:00 +05:30
Vinod Kumar Vavilapalli
32dc13d907 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d1d5f50c605bdb0e3b13f44c86660c8)
 2015-04-27 14:59:57 -07:00
Gera Shegalov
fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b462412bbadad4a1a13ed4c597927b0cf188)
 2015-04-24 17:38:16 -07:00
Gera Shegalov
6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc647074f0bf1cc73c3029f1352f51d5)
 2015-04-21 14:21:56 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka
4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35094f3c2ac216992d861729251b6a3d.
 2015-04-11 12:37:44 +09:00