Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,552 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1982 Commits

Author SHA1 Message Date
Jim Brennan 96fe9d2282 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d131)
 2021-03-22 19:07:53 +00:00
Jim Brennan 38f86cc8c7 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-16 19:17:18 +00:00
Akira Ajisaka 0b278502e2
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 23b343aed1)

 Conflicts:
	dev-support/docker/Dockerfile
	hadoop-project/pom.xml

(cherry picked from commit de2904f123)

 Conflicts:
	dev-support/docker/Dockerfile_aarch64
	hadoop-cloud-storage-project/hadoop-cos/pom.xml
	hadoop-common-project/hadoop-registry/pom.xml
	hadoop-project/pom.xml
 2021-03-11 15:37:56 +09:00
Eric Badger 86b387622a [MAPREDUCE-7234] ClientHSSecurityInfo class is in wrong META-INF file. 
Contributed by Eric Badger.

(cherry picked from commit a17dc4e821)
 2021-03-02 21:49:40 +00:00
Ahmed Hussein e4c638eb21 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit e04bcb3a06)
 2021-02-26 20:38:13 +00:00
Eric Badger dbd4ec5e2d MAPREDUCE-7319. Log list of mappers at trace level in ShuffleHandler audit log. 
Contributed by Jim Brennan.

(cherry picked from commit 462561654b)
 2021-02-09 19:11:45 +00:00
lzx404243 53460b664d
MAPREDUCE-7310. Clear the fileMap in JHEventHandlerForSigtermTest (#2499) 
Co-authored-by: Zhengxi Li <zli89@illinois.edu>
Reviewed-by: Wanqiang Ji <jiwq@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ca7dd5fad3)
 2021-01-12 15:55:44 +09:00
Akira Ajisaka 066d6f5d0d
MAPREDUCE-7284. TestCombineFileInputFormat#testMissingBlocks fails (#2136) 
(cherry picked from commit 3e7b65c3e5)
 2020-12-04 10:08:22 +09:00
Peter Bacsko d727ab4906 MAPREDUCE-7309. Improve performance of reading resource request for mapper/reducers from config. Contributed by Peter Bacsko & Wangda Tan. 2020-11-25 11:44:48 +01:00
Peter Bacsko 82db198d24 MAPREDUCE-7304. Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class. Contributed by Zoltan Erdmann 2020-11-20 13:15:47 +01:00
He Xiaoqiao 3274fd139d
Preparing for 3.2.3 development 2020-10-16 14:52:41 +08:00
Swaroopa Kadam 55f01bda8e
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:09:50 -07:00
zz 0873555f04
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache (#2223) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-19 23:35:24 -07:00
ywheel 3a5f48acab
MAPREDUCE-7051. Fix typo in MultipleOutputFormat (#338) 
(cherry picked from commit cf4eb75608)
 2020-07-30 13:30:46 +09:00
Ayush Saxena 27a97e4f28 HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-22 18:39:49 +05:30
Ahmed Hussein 43a865dc07 HADOOP-17101. Replace Guava Function with Java8+ Function 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 98fcffe93f)
 2020-07-15 10:18:47 -05:00
Eric Badger d07e00008c Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)." 
This reverts commit 6bf255b4ea.
 2020-06-08 20:19:14 +00:00
Wilfred Spiegelenburg e2713d5788
MAPREDUCE-7278. Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false 
Contributed by Tarun Parimi.

(cherry picked from commit 10db97df1c)
 2020-05-28 13:39:57 +10:00
Eric E Payne 6bf255b4ea MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 
(cherry picked from commit e2322e1117)
 2020-04-27 19:56:48 +00:00
Eric E Payne 9c53616190 MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit 11d17417ce)
 2020-04-13 19:07:24 +00:00
Szilard Nemeth 8daa8556d6 MAPREDUCE-7266. historyContext doesn't need to be a class attribute inside JobHistoryServer. Contributed by Siddharth Ahuja 
(cherry picked from commit e2c87b7917)
 2020-04-09 12:42:21 +02:00
Szilard Nemeth 6aec712c6c YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-11 09:18:44 +01:00
Eric E Payne f62c472944 MAPREDUCE-7079: JobHistory#ServiceStop implementation is incorrect. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit b897f6834b)
 2020-01-29 17:10:13 +00:00
Szilard Nemeth 351c64f327 MAPREDUCE-7260. Cross origin request support for Job history server web UI. Contributed by Adam Antal 2020-01-29 14:47:10 +01:00
Ahmed Hussein 96caaf7185 MAPREDUCE-7259. testSpeculateSuccessfulWithUpdateEvents fails Intermittently 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 08251538fe)
 2020-01-28 10:58:42 -06:00
Ahmed Hussein 6538e57c41 MAPREDUCE-7262. MRApp helpers block for long intervals (500ms) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 3f01c48106)
 2020-01-27 16:10:42 -06:00
Szilard Nemeth da416c826f YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:18:35 +01:00
kevinzhao1661 45b23fd74a
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684) 
(cherry picked from commit b8518e6dab)
 2020-01-16 12:58:07 +09:00
Szilard Nemeth 0c2e312fef YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-15 10:51:02 +01:00
Masatake Iwasaki 483d54f1a7
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804) 
(cherry picked from commit a2fdd7c2b5)
 2020-01-15 12:09:59 +09:00
Eric E Payne 3ba0fd1e50 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e)
 2020-01-09 17:22:10 +00:00
Ahmed Hussein ee7c56c6de MAPREDUCE-7252. Handling 0 progress in SimpleExponential task runtime estimator 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit cdd6efd3ab)
 2020-01-08 11:09:58 -06:00
Sergey Pogorelov d7cc897bdc
MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793) 
(cherry picked from commit b343e1533b)
 2020-01-06 12:40:29 +09:00
prabhujoseph e8304426b1 MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 14:32:28 +05:30
prabhujoseph 66e58b5525 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 16:09:24 +05:30
Ahmed Hussein 56988e88f7 MAPREDUCE-7208. Tuning TaskRuntimeEstimator. (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit ed302f1fed)
 2019-11-05 15:21:51 -06:00
Jonathan Hung ad3c98456d YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 
(cherry picked from commit eebd313d76ed742fe82292bd8c0184970cdc5692)
 2019-10-08 11:17:05 -07:00
Rohith Sharma K S 7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
Tao Yang c002186ee1 YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph. 2019-09-07 07:40:04 +08:00
Akira Ajisaka 2d8799f4bc HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2019-08-27 19:08:39 +00:00
Szilard Nemeth faafab9388 MAPREDUCE-7230. TestHSWebApp.testLogsViewSingle fails. Contributed by Prabhu Joseph 
(cherry picked from commit 3f4f097f1f)
 2019-08-15 17:08:28 +02:00
Szilard Nemeth e611fb878b MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko. 2019-08-01 16:05:28 +02:00
pingsutw 772cacdbdd
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build 
This closes #1089

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ee87e9a42e)
 2019-07-23 10:35:36 +09:00
Sean Mackrory e0b3cbd221 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:53:40 -06:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Wanqiang Ji e4011e6886
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165)
 2019-05-22 14:25:34 +09:00
Adam Antal ddf8859655 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 856cbf62d3)
 2019-04-01 10:38:58 -07:00
Steve Loughran 60c9042286
HADOOP-16058. S3A tests to include Terasort. 
Contributed by Steve Loughran.

This includes
 - HADOOP-15890. Some S3A committer tests don't match ITest* pattern; don't run in maven
 - MAPREDUCE-7090. BigMapOutput example doesn't work with paths off cluster fs
 - MAPREDUCE-7091. Terasort on S3A to switch to new committers
 - MAPREDUCE-7092. MR examples to work better against cloud stores
 2019-03-29 15:25:45 +00:00
David Mollitor 9a449ac075
HADOOP-16196. Path Parameterize Comparable. 
Author:    David Mollitor <david.mollitor@cloudera.com>

(cherry picked from commit 246ab77f28)
 2019-03-22 10:27:17 +00:00