Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,878 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2840 Commits

Author SHA1 Message Date
Duo Zhang 57fd58bc3c HADOOP-16664 Backport "HADOOP-16561 [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes" to all active branches 2022-05-29 17:58:31 +09:00
Ashutosh Gupta ca9f18b7d6 MAPREDUCE-7246. In MapredAppMasterRest#Mapreduce_Application_Master_Info_API, updating the datatype of appId to "string". (#4223) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fb13c1e4a8)
 2022-04-25 14:31:39 +09:00
Kengo Seki 5c6547d794 MAPREDUCE-7373. Building MapReduce NativeTask fails on Fedora 34+ (#4120) 
(cherry picked from commit dc4a680da8)
 2022-03-30 22:51:40 +09:00
Masatake Iwasaki e13408d9d2
HADOOP-18088. Replace log4j 1.x with reload4j. (#4084) 
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-03-23 12:57:27 +09:00
lzx404243 8f70d48d14
MAPREDUCE-7311. Clear filesystem statistics after tests in TestTaskProgressReporter (#2500) 
Co-authored-by: Zhengxi Li <zli89@illinois.edu>
(cherry picked from commit 6187f76f11)
 2021-09-01 17:20:48 +09:00
lzx404243 5c9ab0ce09
MAPREDUCE-7342. Stop RMService in TestClientRedirect.testRedirect() (#2968) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 7b5be74228)
 2021-08-30 08:48:11 +09:00
Masatake Iwasaki 50c7fec545 HADOOP-14922. Build of Mapreduce Native Task module fails with unknown opcode "bswap". Contributed by Anup Halarnkar. 
(cherry picked from commit 0d59500e8c)
 2021-08-25 10:57:22 +09:00
Brahma Reddy Battula 3cf2479c04 Preparing for 3.2.4 development. 2021-08-08 18:35:16 +05:30
jenny 18ba3d6b1e
MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGroup (#1808) 
Co-authored-by: chenjuanni <chenjuanni@inspur.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e31169c864)
 2021-08-02 15:40:14 +09:00
Eric Payne ab7d2f1f3a MAPREDUCE-7353: Mapreduce job fails when NM is stopped. Contributed by Bilwa S T (BilwaST) 
(cherry picked from commit 7581413156)
 2021-07-07 21:14:47 +00:00
Jim Brennan 9c4ac62d68 YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T. 
(cherry picked from commit 7c7d02edbd)
 2021-06-25 20:50:11 +00:00
Eric Badger 7ad844ee9e MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka. 
(cherry picked from commit da93cd962c)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/ReadaheadPool.java
 2021-04-23 18:53:09 +00:00
Takanobu Asanuma 9060972172
HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. (#2155) (#2806) 
(cherry picked from commit 1b29c9bfee)

Co-authored-by: Masatake Iwasaki <iwasakims@apache.org>
 2021-03-24 17:09:43 +09:00
Jim Brennan 96fe9d2282 MAPREDUCE-7325. Intermediate data encryption is broken in LocalJobRunner. Contributed by Ahmed Hussein 
(cherry picked from commit ede490d131)
 2021-03-22 19:07:53 +00:00
Jim Brennan 38f86cc8c7 MAPREDUCE-7322. revisiting TestMRIntermediateDataEncryption. Contributed by Ahmed Hussein. 2021-03-16 19:17:18 +00:00
Akira Ajisaka 0b278502e2
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 23b343aed1)

 Conflicts:
	dev-support/docker/Dockerfile
	hadoop-project/pom.xml

(cherry picked from commit de2904f123)

 Conflicts:
	dev-support/docker/Dockerfile_aarch64
	hadoop-cloud-storage-project/hadoop-cos/pom.xml
	hadoop-common-project/hadoop-registry/pom.xml
	hadoop-project/pom.xml
 2021-03-11 15:37:56 +09:00
Eric Badger 86b387622a [MAPREDUCE-7234] ClientHSSecurityInfo class is in wrong META-INF file. 
Contributed by Eric Badger.

(cherry picked from commit a17dc4e821)
 2021-03-02 21:49:40 +00:00
Ahmed Hussein e4c638eb21 MAPREDUCE-7320. organize test directories for ClusterMapReduceTestCase (#2722). Contributed by Ahmed Hussein 
(cherry picked from commit e04bcb3a06)
 2021-02-26 20:38:13 +00:00
Akira Ajisaka 0172ce763a
MAPREDUCE-7323. Remove job_history_summary.py. (#2712) 
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 8bca105655)
 2021-02-22 23:16:32 +09:00
Eric Badger dbd4ec5e2d MAPREDUCE-7319. Log list of mappers at trace level in ShuffleHandler audit log. 
Contributed by Jim Brennan.

(cherry picked from commit 462561654b)
 2021-02-09 19:11:45 +00:00
lzx404243 53460b664d
MAPREDUCE-7310. Clear the fileMap in JHEventHandlerForSigtermTest (#2499) 
Co-authored-by: Zhengxi Li <zli89@illinois.edu>
Reviewed-by: Wanqiang Ji <jiwq@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ca7dd5fad3)
 2021-01-12 15:55:44 +09:00
He Xiaoqiao a17793534c
Make upstream aware of 3.2.2 release. 2021-01-09 18:07:46 +08:00
Akira Ajisaka 066d6f5d0d
MAPREDUCE-7284. TestCombineFileInputFormat#testMissingBlocks fails (#2136) 
(cherry picked from commit 3e7b65c3e5)
 2020-12-04 10:08:22 +09:00
Peter Bacsko d727ab4906 MAPREDUCE-7309. Improve performance of reading resource request for mapper/reducers from config. Contributed by Peter Bacsko & Wangda Tan. 2020-11-25 11:44:48 +01:00
Peter Bacsko 82db198d24 MAPREDUCE-7304. Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class. Contributed by Zoltan Erdmann 2020-11-20 13:15:47 +01:00
He Xiaoqiao 3274fd139d
Preparing for 3.2.3 development 2020-10-16 14:52:41 +08:00
Swaroopa Kadam 55f01bda8e
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:09:50 -07:00
Wanqiang Ji 578370d6c8
MAPREDUCE-7289. Fix wrong comment in LongLong.java (#2338) 
(cherry picked from commit 143bdd4188)
 2020-09-29 23:08:30 +09:00
zz 0873555f04
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache (#2223) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-19 23:35:24 -07:00
Wanqiang Ji ea7e41e29e
MAPREDUCE-7288. Fix TestLongLong#testRightShift (#2183) 
(cherry picked from commit dc5470ae86)
 2020-08-06 09:37:03 +09:00
ywheel 3a5f48acab
MAPREDUCE-7051. Fix typo in MultipleOutputFormat (#338) 
(cherry picked from commit cf4eb75608)
 2020-07-30 13:30:46 +09:00
Ayush Saxena 27a97e4f28 HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein. 2020-07-22 18:39:49 +05:30
Ahmed Hussein 43a865dc07 HADOOP-17101. Replace Guava Function with Java8+ Function 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 98fcffe93f)
 2020-07-15 10:18:47 -05:00
Eric Badger d07e00008c Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)." 
This reverts commit 6bf255b4ea.
 2020-06-08 20:19:14 +00:00
Wilfred Spiegelenburg e2713d5788
MAPREDUCE-7278. Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false 
Contributed by Tarun Parimi.

(cherry picked from commit 10db97df1c)
 2020-05-28 13:39:57 +10:00
Eric E Payne 6bf255b4ea MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 
(cherry picked from commit e2322e1117)
 2020-04-27 19:56:48 +00:00
Eric E Payne 9c53616190 MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit 11d17417ce)
 2020-04-13 19:07:24 +00:00
Szilard Nemeth 8daa8556d6 MAPREDUCE-7266. historyContext doesn't need to be a class attribute inside JobHistoryServer. Contributed by Siddharth Ahuja 
(cherry picked from commit e2c87b7917)
 2020-04-09 12:42:21 +02:00
Szilard Nemeth 6aec712c6c YARN-10101. Support listing of aggregated logs for containers belonging to an application attempt. Contributed by Adam Antal 2020-02-11 09:18:44 +01:00
Eric E Payne f62c472944 MAPREDUCE-7079: JobHistory#ServiceStop implementation is incorrect. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit b897f6834b)
 2020-01-29 17:10:13 +00:00
Szilard Nemeth 351c64f327 MAPREDUCE-7260. Cross origin request support for Job history server web UI. Contributed by Adam Antal 2020-01-29 14:47:10 +01:00
Ahmed Hussein 96caaf7185 MAPREDUCE-7259. testSpeculateSuccessfulWithUpdateEvents fails Intermittently 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 08251538fe)
 2020-01-28 10:58:42 -06:00
Ahmed Hussein 6538e57c41 MAPREDUCE-7262. MRApp helpers block for long intervals (500ms) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 3f01c48106)
 2020-01-27 16:10:42 -06:00
Szilard Nemeth da416c826f YARN-10083. Provide utility to ask whether an application is in final status. Contributed by Adam Antal 2020-01-22 16:18:35 +01:00
kevinzhao1661 45b23fd74a
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684) 
(cherry picked from commit b8518e6dab)
 2020-01-16 12:58:07 +09:00
Szilard Nemeth 0c2e312fef YARN-10028. Integrate the new abstract log servlet to the JobHistory server. Contributed by Adam Antal 2020-01-15 10:51:02 +01:00
Masatake Iwasaki 483d54f1a7
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804) 
(cherry picked from commit a2fdd7c2b5)
 2020-01-15 12:09:59 +09:00
Eric E Payne 3ba0fd1e50 YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e)
 2020-01-09 17:22:10 +00:00
Ahmed Hussein ee7c56c6de MAPREDUCE-7252. Handling 0 progress in SimpleExponential task runtime estimator 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit cdd6efd3ab)
 2020-01-08 11:09:58 -06:00
Sergey Pogorelov d7cc897bdc
MAPREDUCE-7255. Fix typo in MapReduce documentaion example (#1793) 
(cherry picked from commit b343e1533b)
 2020-01-06 12:40:29 +09:00