Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,744 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

868 Commits

Author SHA1 Message Date
lzx404243 8f70d48d14
MAPREDUCE-7311. Clear filesystem statistics after tests in TestTaskProgressReporter (#2500) 
Co-authored-by: Zhengxi Li <zli89@illinois.edu>
(cherry picked from commit 6187f76f11)
 2021-09-01 17:20:48 +09:00
Brahma Reddy Battula 3cf2479c04 Preparing for 3.2.4 development. 2021-08-08 18:35:16 +05:30
Eric Badger 7ad844ee9e MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka. 
(cherry picked from commit da93cd962c)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
 2021-04-23 18:53:09 +00:00
Jim Brennan 96fe9d2282 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d131)
 2021-03-22 19:07:53 +00:00
Jim Brennan 38f86cc8c7 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-16 19:17:18 +00:00
Peter Bacsko 82db198d24 MAPREDUCE-7304. Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class. Contributed by Zoltan Erdmann 2020-11-20 13:15:47 +01:00
He Xiaoqiao 3274fd139d
Preparing for 3.2.3 development 2020-10-16 14:52:41 +08:00
zz 0873555f04
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache (#2223) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-19 23:35:24 -07:00
ywheel 3a5f48acab
MAPREDUCE-7051. Fix typo in MultipleOutputFormat (#338) 
(cherry picked from commit cf4eb75608)
 2020-07-30 13:30:46 +09:00
Ahmed Hussein 43a865dc07 HADOOP-17101. Replace Guava Function with Java8+ Function 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 98fcffe93f)
 2020-07-15 10:18:47 -05:00
Eric Badger d07e00008c Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)." 
This reverts commit 6bf255b4ea.
 2020-06-08 20:19:14 +00:00
Eric E Payne 6bf255b4ea MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 
(cherry picked from commit e2322e1117)
 2020-04-27 19:56:48 +00:00
Eric E Payne 9c53616190 MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit 11d17417ce)
 2020-04-13 19:07:24 +00:00
Sergey Pogorelov d7cc897bdc
MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793) 
(cherry picked from commit b343e1533b)
 2020-01-06 12:40:29 +09:00
Ahmed Hussein 56988e88f7 MAPREDUCE-7208. Tuning TaskRuntimeEstimator. (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit ed302f1fed)
 2019-11-05 15:21:51 -06:00
Rohith Sharma K S 7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
Szilard Nemeth e611fb878b MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko. 2019-08-01 16:05:28 +02:00
Sean Mackrory e0b3cbd221 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:53:40 -06:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
David Mollitor 9a449ac075
HADOOP-16196. Path Parameterize Comparable. 
Author:    David Mollitor <david.mollitor@cloudera.com>

(cherry picked from commit 246ab77f28)
 2019-03-22 10:27:17 +00:00
Akira Ajisaka 495285e50e
MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin. 
(cherry picked from commit e0d75088f2)
 2018-12-13 16:27:52 +09:00
Jason Lowe 7a78bdf7bb MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla 
(cherry picked from commit 4d8de7ab69)
 2018-11-28 15:54:59 -06:00
Akira Ajisaka 2e6b9b174e
MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin. 
(cherry picked from commit 1aad99a718)
 2018-11-27 20:02:11 +09:00
Wangda Tan f214af74b2 MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda) 
Change-Id: I99ace87980da03bb35a8012cea7218d602a8817a
(cherry picked from commit 762a56cc64)
 2018-11-13 11:35:13 -08:00
Sunil G bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Xiao Chen 424c99681b MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb064)
 2018-10-16 10:24:15 -07:00
Wangda Tan 230d90c5b9 MAPREDUCE-7125. JobResourceUploader creates LocalFileSystem when it's not necessary. (Peter Cseh via wangda) 
Change-Id: I1aa720ed03739f6f4abeec46f6068e2ab332987a
 2018-09-24 13:52:53 -07:00
Jason Lowe 2886024ac3 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 2018-09-13 14:41:38 -05:00
Steve Loughran 5a0babf765
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs. 
Contributed by Steve Loughran.
 2018-08-30 14:49:53 +01:00
Haibo Chen 84973d1049 MAPREDUCE-6861. Add metrics tags for ShuffleClientMetrics. (Contributed by Zoltan Siegl) 2018-08-27 16:53:06 +02:00
Todd Lipcon 7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Vidura Mudalige 1e94e5977f
MAPREDUCE-7063. Fix log level inconsistency in CombineFileInputFormat.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-06-18 14:25:11 -07:00
Arun Suresh 5670e89b2e MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via asuresh) 2018-06-12 15:36:52 -07:00
Gera Shegalov 04b74eddc6 MAPREDUCE-7108. TestFileOutputCommitter fails on Windows. (Zuoming Zhang via gera) 2018-06-12 11:21:51 -07:00
Inigo Goiri 0afc036deb MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 2018-06-05 12:24:57 -07:00
Xiao Chen 6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Takanobu Asanuma d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-31 18:31:10 +09:00
Eric Yang 8cdb032aff YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen
 2018-05-04 12:36:31 -04:00
Bibin A Chundatt 1a95a4524a
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-02 16:14:28 +09:00
Jason Lowe 68c6ec719d MAPREDUCE-7086. Add config to allow FileInputFormat to ignore directories when recursive=false. Contributed by Sergey Shelukhin 2018-05-01 16:19:53 -05:00
Ewan Higgs 84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Robert Kanter 1b9ecc264a MAPREDUCE-7072. mapred job -history prints duplicate counter in human output (wilfreds via rkanter) 2018-04-24 11:30:38 -07:00
Wangda Tan 995cba65fe MAPREDUCE-7077. Pipe mapreduce job fails with Permission denied for jobTokenPassword. (Akira Ajisaka via wangda) 
Change-Id: Ie8f01425d58409fa3661f768205b7616128c8aa4
(cherry picked from commit 035e0f97ea)
 2018-04-13 13:52:10 -07:00
Jason Lowe 4571351ccc MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 2018-04-12 11:12:46 -05:00
Rohith Sharma K S 6cc59a09e7 MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri. 2018-04-11 09:45:39 +05:30
Steve Loughran dde1579096 HADOOP-15146. Remove DataOutputByteBuffer. 
Contributed by BELUGA BEHR.
 2018-04-02 15:52:40 +01:00
Takanobu Asanuma 86816da5b4
MAPREDUCE-7066. TestQueue fails on Java9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-19 09:39:49 +09:00
Jason Lowe 32fa3a63e0 MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao 2018-03-09 10:41:16 -06:00
Weiwei Yang 6e6945cd78 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-01 10:18:53 +08:00