Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,367 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

603 Commits

Author SHA1 Message Date
Jason Lowe d1aa3e0f89 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81)
 2016-05-06 22:27:24 +00:00
Vinod Kumar Vavilapalli e9c7a17bc8 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720)
 2016-05-05 19:05:44 -07:00
Robert Kanter f9d6092108 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb3)
 2016-05-04 22:42:34 -07:00
Robert Kanter cca8e9b90c MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter) 
(cherry picked from commit ecce3b7d53)
 2016-05-04 22:19:42 -07:00
Xuan 94e4d349b0 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
 2016-04-16 19:40:11 -07:00
Wangda Tan ab23acdfda MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 11:16:37 -07:00
Robert Kanter 3584cbc355 MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter) 
(cherry picked from commit 3be1ab485f)
 2016-04-06 17:16:10 -07:00
Robert Kanter f9f57265cf MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77)
 2016-03-03 16:38:25 -08:00
Varun Vasudev 212c519ad3 MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana. 
(cherry picked from commit c50d1e54000c76880a041ce5959a2eb23c86bd35)
 2016-02-18 14:19:01 +05:30
Akira Ajisaka 15ef3a85d9 Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta." 
This reverts commit e4c01b8b1c.
 2016-02-10 23:45:15 +09:00
Akira Ajisaka 9a55d3a7cb MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun. 
(cherry picked from commit 10393d9a666419a5d8dc0cfb48da96e9c690c11e)
 2016-02-10 03:05:16 +09:00
Akira Ajisaka e4c01b8b1c HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit fe124da5ff)
 2016-02-06 19:32:50 +09:00
Jian He ad42229683 MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline response. Contributed by Li Lu 2016-01-25 23:01:29 -08:00
Akira Ajisaka a9386cae6e MAPREDUCE-6614. Remove unnecessary code in TestMapreduceConfigFields. Contributed by Kai Sasaki. 2016-01-25 11:39:42 +09:00
Akira Ajisaka 7f215ffdd5 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 2016-01-22 18:37:17 +09:00
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Wangda Tan 15e174fe79 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:54:09 +08:00
Akira Ajisaka 6d235f0b91 MAPREDUCE-6593. TestJobHistoryEventHandler.testTimelineEventHandling fails on trunk because of NPE. Contributed by Naganarasimha G R. 
(cherry picked from commit 7dafee11d8)
 2016-01-04 16:31:28 +09:00
Wangda Tan ca575e6add MAPREDUCE-6574. MR AM should print host of failed tasks. (Mohammad Shahid Khan via wangda) 
(cherry picked from commit 32b3dc11eb)
 2015-12-28 11:04:30 -08:00
Jian He 23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)
 2015-12-03 12:49:22 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Junping Du 628c782870 Tests in mapreduce-client-app are writing outside of target. Contributed by Akira AJISAKA. 
(cherry picked from commit 15d577bfbb)
 2015-11-25 09:16:22 -08:00
Akira Ajisaka c6cf7910a4 MAPREDUCE-6553. Replace '\u2b05' with '<-' in rendering job configuration. Contributed by Gabor Liptak. 
(cherry picked from commit 5ec44fc35c)
 2015-11-25 16:33:18 +09:00
Jason Lowe 2f18218508 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G 
(cherry picked from commit f634505d48)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
 2015-11-24 22:15:37 +00:00
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a1)
 2015-11-23 17:20:24 -08:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Jian He f562c155d2 MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:08:22 -08:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Tsuyoshi Ozawa 1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:35 +09:00
Jason Lowe 145058003a MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G 
(cherry picked from commit cf953b6258)
 2015-10-29 18:05:56 +00:00
Karthik Kambatla 2d8f9e3fd9 MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 
(cherry picked from commit 4aa9b3e75c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-10-09 07:54:47 -07:00
rohithsharmaks 7dd9ebdebc MAPREDUCE-6485. Create a new task attempt with failed map task priority if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks) 
(cherry picked from commit 439f43ad3d)
 2015-10-02 20:36:20 +05:30
Akira Ajisaka e42f576232 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3)
 2015-10-01 10:26:00 +02:00
Jason Lowe 3c4c52a095 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81)
 2015-09-28 22:13:03 +00:00
Jian He d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10)
 2015-09-24 10:23:25 -07:00
Jian He 89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849f)
 2015-09-24 10:23:13 -07:00
Tsuyoshi Ozawa 52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab)
 2015-09-22 12:54:44 +09:00
Zhihai Xu 88a3a07636 MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 
(cherry picked from commit e3ace31e2b)
 2015-09-19 00:12:15 -07:00
Jason Lowe f27f1ef44c MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 
(cherry picked from commit ee4ee6af6a)
 2015-09-17 21:38:44 +00:00
Jason Lowe 4861fadb17 MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li 
(cherry picked from commit 3f82f582e5)
 2015-09-17 18:18:28 +00:00
Vinod Kumar Vavilapalli 9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Karthik Kambatla 20734320a1 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a197)
 2015-08-15 00:53:13 -07:00
Karthik Kambatla b826168173 MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab)
 2015-08-14 12:40:38 -07:00
Jason Lowe 508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010)
 2015-07-31 18:19:52 +00:00
Zhihai Xu 9d40eead81 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 
(cherry picked from commit 93d50b7824)
 2015-07-30 23:12:30 -07:00
Chris Douglas 2a0e551cd8 MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:56:21 -07:00
Akira Ajisaka 0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d9)
 2015-07-13 17:53:27 +09:00
Jason Lowe 8a4970e845 MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li 
(cherry picked from commit eac1d18943)
 2015-07-01 18:02:17 +00:00
Jason Lowe f3195eda27 MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li 
(cherry picked from commit 826715622e)
 2015-07-01 17:50:47 +00:00