Commit Graph

813 Commits

Author SHA1 Message Date
Jason Lowe 9dacb9fb27 MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao
(cherry picked from commit 32fa3a63e0)
2018-03-09 10:44:49 -06:00
Weiwei Yang 5db733fb09 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-02 10:43:03 +08:00
Steve Loughran c960ae0b9a MAPREDUCE-7060. Cherry Pick PathOutputCommitter class/factory to branch-3.0.
Contributed by Steve Loughran.
2018-03-01 18:39:54 +00:00
Jason Lowe bc2d4ba507 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko
(cherry picked from commit 87e2570a14)
2018-02-12 14:49:35 -06:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Eric Payne 1d53c9d3a9 MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe
(cherry picked from commit 5a725bb886)
2018-01-31 16:01:46 -06:00
Jason Lowe d7acc45ce4 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko
(cherry picked from commit 6eef3d7f1a)
2018-01-26 15:37:16 -06:00
Chris Douglas 03f0b8b3f2 MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec
(cherry picked from commit 782ba3bf9d)
2017-11-21 20:42:59 -08:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Naganarasimha 8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph.
(cherry picked from commit 61bbdc511e)
2017-11-06 12:59:52 +05:30
Akira Ajisaka 12f92e636e
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling.
(cherry picked from commit 178751ed8c)
2017-11-02 17:43:55 +09:00
Daniel Templeton 7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e810838)
2017-10-31 15:06:18 -07:00
Akira Ajisaka a887016e1d
MAPREDUCE-6972. Enable try-with-resources for RecordReader. Contributed by Zoltan Haindrich.
(cherry picked from commit 2523e1cce5)
2017-10-18 11:19:00 +09:00
Chris Trezzo 24b03eb79f MAPREDUCE-5951. Add support for the YARN Shared Cache.
(cherry picked from commit e46d5bb962)
2017-10-12 11:15:46 -07:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Jason Lowe bf03363d83 MAPREDUCE-6968. Staging directory erasure coding config property has a typo. Contributed by Jason Lowe
(cherry picked from commit 9df05005ac)
2017-09-26 14:53:47 -05:00
Robert Kanter 917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter)
(cherry picked from commit 0adc0471d0)
2017-09-18 10:42:01 -07:00
Robert Kanter 3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
This reverts commit 293ab432e1.
2017-09-18 10:41:46 -07:00
Robert Kanter 293ab432e1 YARN-7162. Remove XML excludes file format (rkanter)
(cherry picked from commit 3a8d57a0a2)
2017-09-15 12:01:41 -07:00
Steve Loughran b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
Contributed by Steve Loughran

(cherry picked from commit 11390c2d11)
2017-09-15 17:00:16 +01:00
Jason Lowe 12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim
(cherry picked from commit 4d98936eec)
2017-09-13 17:22:29 -05:00
Ray Chiang 656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)
(cherry picked from commit 556812c179)
2017-09-12 10:19:58 -07:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
Haibo Chen d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Sangjin Lee 092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Mingliang Liu ae7abad3d9 MAPREDUCE-6945. TestMapFileOutputFormat missing @after annotation. Contributed by Ajay Kumar 2017-08-27 15:19:55 -07:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Jason Lowe 0acc5e0036 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:34:06 -05:00
Haibo Chen a32e0138fb MAPREDUCE-6870. Add configuration for MR job to finish when all reducers are complete. (Peter Bacsko via Haibo Chen) 2017-08-10 15:17:36 -07:00
Yufei Gu b8e8241854 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:44:34 -07:00
Junping Du fbb7d6bcbb Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)"
This reverts commit e8a31f2e1c.
2017-07-31 14:09:16 -07:00
Junping Du 713349a9af Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)"
This reverts commit 390a7c12f5.
2017-07-28 14:21:04 -07:00
Junping Du e15f928295 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu."
This reverts commit 433542904a.
2017-07-28 14:06:59 -07:00
Sunil G fb3b5d33ff MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 2017-07-17 13:35:15 +05:30
Junping Du f484a6ff60 MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 2017-07-07 13:26:16 -07:00
Andrew Wang af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Xiao Chen 4ebc23ba7b HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:33 -07:00
Sean Busbey f5517a8200
HADOOP-13921. Remove log4j classes from JobConf.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-06-01 13:35:14 +09:00
Akira Ajisaka d4015f8628
MAPREDUCE-6887. Modifier 'static' is redundant for inner enums. Contributed by ZhangBing Lin. 2017-05-30 14:48:58 +09:00
Andrew Wang 16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Sunil G b6f66b0da1 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-22 14:10:06 +05:30
Jason Lowe 1e71fe8c42 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 2017-05-10 09:57:41 -05:00
Chris Douglas 45382bf047 AuditLogger and TestAuditLogger are dead code. Contributed by Vrushali C 2017-05-09 10:31:52 -07:00
Akira Ajisaka 3ed3062fe3
MAPREDUCE-6881. Fix warnings from Spotbugs in hadoop-mapreduce. Contributed by Weiwei Yang. 2017-04-27 15:45:33 +09:00
Robert Kanter 3721cfe1fb MAPREDUCE-6871. Allow users to specify racks and nodes for strict locality for AMs (rkanter) 2017-04-21 16:12:01 -07:00
Daniel Templeton fc0885da29 MAPREDUCE-6846. Fragments specified for libjar paths are not handled correctly
(Contributed by Chris Trezzo via Daniel Templeton)
2017-04-05 17:25:40 -07:00
Chris Douglas 6eba79232f HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang 2017-04-03 20:13:14 -07:00
Chris Douglas 7297e82cb4 HADOOP-14250. Correct spelling of 'separate' and variants. Contributed by Doris Gu 2017-03-30 10:17:11 -07:00