Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,562 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2776 Commits

Author SHA1 Message Date
Wanqiang Ji e4011e6886
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165)
 2019-05-22 14:25:34 +09:00
Adam Antal ddf8859655 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 856cbf62d3)
 2019-04-01 10:38:58 -07:00
Steve Loughran 60c9042286
HADOOP-16058. S3A tests to include Terasort. 
Contributed by Steve Loughran.

This includes
 - HADOOP-15890. Some S3A committer tests don't match ITest* pattern; don't run in maven
 - MAPREDUCE-7090. BigMapOutput example doesn't work with paths off cluster fs
 - MAPREDUCE-7091. Terasort on S3A to switch to new committers
 - MAPREDUCE-7092. MR examples to work better against cloud stores
 2019-03-29 15:25:45 +00:00
David Mollitor 9a449ac075
HADOOP-16196. Path Parameterize Comparable. 
Author:    David Mollitor <david.mollitor@cloudera.com>

(cherry picked from commit 246ab77f28)
 2019-03-22 10:27:17 +00:00
Weiwei Yang 3d86dd8931 MAPREDUCE-7192. JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable. Contributed by Jiandan Yang. 
(cherry picked from commit 159a715eef)
 2019-03-13 17:41:31 +08:00
Weiwei Yang 9a2b24642d MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang. 
(cherry picked from commit f0605146b3)
 2019-03-11 16:03:38 +08:00
Wangda Tan 649da5af04 Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa)
 2019-02-05 14:05:51 -08:00
Akira Ajisaka 97fa326b79
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin. 
(cherry picked from commit c991e2ca5a)
 2019-02-04 14:04:07 +09:00
Weiwei Yang 0d78725fa8 MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui. 
(cherry picked from commit 9e7196749e)
 2019-01-02 21:48:57 +08:00
Akira Ajisaka 495285e50e
MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin. 
(cherry picked from commit e0d75088f2)
 2018-12-13 16:27:52 +09:00
Jason Lowe 298a1f29e8 MAPREDUCE-7159. FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used. Contributed by Peter Bacsko 
(cherry picked from commit 9886c27c71)
 2018-12-06 14:57:45 -06:00
Jason Lowe 7a78bdf7bb MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla 
(cherry picked from commit 4d8de7ab69)
 2018-11-28 15:54:59 -06:00
Akira Ajisaka 2e6b9b174e
MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin. 
(cherry picked from commit 1aad99a718)
 2018-11-27 20:02:11 +09:00
Wangda Tan f214af74b2 MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda) 
Change-Id: I99ace87980da03bb35a8012cea7218d602a8817a
(cherry picked from commit 762a56cc64)
 2018-11-13 11:35:13 -08:00
Jason Lowe ff5a2cb5c9 MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko 
(cherry picked from commit ba1f9d66d9)
 2018-11-06 17:57:09 -06:00
Jason Lowe 9265934201 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c726)
 2018-11-06 08:47:30 -06:00
bibinchundatt aed71316e5 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T. 
(cherry picked from commit 74a5e683fe)
 2018-10-25 00:18:39 +05:30
Sunil G bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Xiao Chen 424c99681b MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb064)
 2018-10-16 10:24:15 -07:00
Wangda Tan 230d90c5b9 MAPREDUCE-7125. JobResourceUploader creates LocalFileSystem when it's not necessary. (Peter Cseh via wangda) 
Change-Id: I1aa720ed03739f6f4abeec46f6068e2ab332987a
 2018-09-24 13:52:53 -07:00
Jason Lowe 8382b860d4 MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 2018-09-18 17:06:32 -05:00
Jason Lowe 34b2237e42 MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 2018-09-18 16:56:31 -05:00
Eric E Payne 2cf8927683 MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 2018-09-18 20:26:28 +00:00
Jason Lowe 488806baca MAPREDUCE-7140. Refactoring TaskAttemptInfo to separate Map and Reduce tasks. Contributed by Oleksandr Shevchenko 2018-09-14 15:10:27 -05:00
Jason Lowe 2886024ac3 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 2018-09-13 14:41:38 -05:00
bibinchundatt 8cf6a9a2bd YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha 5dc7d6e0f3 YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha 0a01b1350d YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Jason Lowe 598380ad5a MAPREDUCE-7136. TestMRAppMetrics should shutdown DefaultMetricsSystem after completion. Contributed by Oleksandr Shevchenko 2018-09-11 13:10:50 -05:00
Jason Lowe 7ce997af97 MAPREDUCE-7135. TestTaskAttemptContainerRequest should reset UserGroupInformation. Contributed by Oleksandr Shevchenko 2018-09-11 11:16:33 -05:00
Jason Lowe eb0b5a844f MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu 2018-09-06 14:13:29 -05:00
Steve Loughran 5a0babf765
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs. 
Contributed by Steve Loughran.
 2018-08-30 14:49:53 +01:00
Haibo Chen 84973d1049 MAPREDUCE-6861. Add metrics tags for ShuffleClientMetrics. (Contributed by Zoltan Siegl) 2018-08-27 16:53:06 +02:00
Márton Elek 77b015000a HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor. 2018-08-17 11:10:29 +02:00
Wangda Tan b3b4d4ccb5 MAPREDUCE-7118. Distributed cache conflicts breaks backwards compatability. (Jason Lowe via wangda) 
Change-Id: I89ab4852b4ad305fec19812e8931c59d96581376
 2018-07-19 14:26:05 -07:00
Akira Ajisaka 88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Todd Lipcon 7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Miklos Szegedi ae055622ed MAPREDUCE-7114. Make FrameworkUploader symlink ignore improvement. Contributed by Gergo Repas. 2018-06-22 13:05:41 -07:00
Miklos Szegedi d6ee4290df MAPREDUCE-7113. Typos in test names in TestTaskAttempt: "testAppDiognostic". Contributed by Szilard Nemeth. 2018-06-20 12:04:44 -07:00
Vidura Mudalige 1e94e5977f
MAPREDUCE-7063. Fix log level inconsistency in CombineFileInputFormat.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-06-18 14:25:11 -07:00
Arun Suresh 5670e89b2e MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via asuresh) 2018-06-12 15:36:52 -07:00
Gera Shegalov 04b74eddc6 MAPREDUCE-7108. TestFileOutputCommitter fails on Windows. (Zuoming Zhang via gera) 2018-06-12 11:21:51 -07:00
Inigo Goiri 0afc036deb MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 2018-06-05 12:24:57 -07:00
Xiao Chen 6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Inigo Goiri dad1bb868f MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola. 2018-06-04 12:55:54 -07:00
Inigo Goiri a804b7c9d2 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 2018-06-02 20:25:42 -07:00
Rohith Sharma K S d5e69d8994 MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user(addendum). Contributed by Sunil Govindan. 2018-06-02 09:27:03 +05:30
Takanobu Asanuma d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-31 18:31:10 +09:00
Rohith Sharma K S 88cbe57c06 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 2018-05-28 14:05:49 +05:30
Miklos Szegedi a2cdffb95a MAPREDUCE-7094. LocalDistributedCacheManager leaves classloaders open, which leaks FDs. Contributed by Adam Szita. 2018-05-17 11:16:04 -07:00