Commit Graph

2663 Commits

Author SHA1 Message Date
Haibo Chen fc669778be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen)
(cherry picked from commit cce71dceef)
2018-01-19 13:05:40 -08:00
Jason Lowe 2a870d91fe MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas
(cherry picked from commit fe35103591)
2018-01-03 11:03:16 -06:00
Junping Du 53b9d81f46 Add 2.8.3 release jdiff files.
(cherry picked from commit c89f99aade)
2017-12-18 22:40:04 -08:00
Jason Lowe 0dd7f1f4e8 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko
(cherry picked from commit 21d3627355)
2017-12-01 14:10:02 -06:00
Chris Douglas 03f0b8b3f2 MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec
(cherry picked from commit 782ba3bf9d)
2017-11-21 20:42:59 -08:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Arun Suresh 72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh)
(cherry picked from commit 5eb7dbe9b3)
2017-11-09 15:16:55 -08:00
Akira Ajisaka a536655c63
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling.
(cherry picked from commit e4c220ee4f)
2017-11-08 19:28:59 +09:00
Akira Ajisaka d77f17458c
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák.
(cherry picked from commit ffee10b68e)
2017-11-08 19:23:51 +09:00
Naganarasimha 8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph.
(cherry picked from commit 61bbdc511e)
2017-11-06 12:59:52 +05:30
Xiao Chen 1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED.
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
2017-11-02 21:27:43 -07:00
Akira Ajisaka 12f92e636e
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling.
(cherry picked from commit 178751ed8c)
2017-11-02 17:43:55 +09:00
Daniel Templeton 7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e810838)
2017-10-31 15:06:18 -07:00
Wangda Tan 4af3deaeb0 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda)
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541a)
2017-10-31 15:06:17 -07:00
Jason Lowe 75605766fd HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe
(cherry picked from commit 2fae63aa60)
2017-10-31 09:32:02 -05:00
Jason Lowe 00345eed50 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla
(cherry picked from commit 665bb147aa)
2017-10-27 17:00:31 -05:00
Junping Du ce10c1ef05 Add several jdiff xml files for 2.8.2 release.
(cherry picked from commit a25b5aa0cf)
2017-10-26 23:18:50 -07:00
Akira Ajisaka a887016e1d
MAPREDUCE-6972. Enable try-with-resources for RecordReader. Contributed by Zoltan Haindrich.
(cherry picked from commit 2523e1cce5)
2017-10-18 11:19:00 +09:00
Chris Trezzo 24b03eb79f MAPREDUCE-5951. Add support for the YARN Shared Cache.
(cherry picked from commit e46d5bb962)
2017-10-12 11:15:46 -07:00
Akira Ajisaka 47aa044fcc
MAPREDUCE-6971. Moving logging APIs over to slf4j in hadoop-mapreduce-client-app. Contributed by Jinjiang Ling.
(cherry picked from commit 453d48bdfb)
2017-10-03 12:15:43 +09:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Jason Lowe bf03363d83 MAPREDUCE-6968. Staging directory erasure coding config property has a typo. Contributed by Jason Lowe
(cherry picked from commit 9df05005ac)
2017-09-26 14:53:47 -05:00
Jason Lowe 5eb1f9f7b3 MAPREDUCE-6965. QuasiMonteCarlo should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
(cherry picked from commit 908d8e914e)
2017-09-22 09:36:29 -05:00
Akira Ajisaka cbb3294e2d
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari.
(cherry picked from commit c71d137960)
2017-09-22 15:10:01 +09:00
Akira Ajisaka 0af3aec502
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergely Novák.
(cherry picked from commit 53be075241)
2017-09-22 13:30:22 +09:00
Jason Lowe 6c07077837 MAPREDUCE-6964. BaileyBorweinPlouffe should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
(cherry picked from commit b9db0afa1e)
2017-09-21 09:39:01 -05:00
Akira Ajisaka 835717210a
Revert "MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák."
This reverts commit 9c90400c96.
2017-09-21 11:17:35 +09:00
Andrew Wang 5c8331c230 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong.
(cherry picked from commit 51edaacd09)
2017-09-19 13:48:25 -07:00
Eric Payne 839d5b124d MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure.
(cherry picked from commit 595d478408)
2017-09-19 10:40:41 -05:00
Jason Lowe 731a622627 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit 3a20debdde)
2017-09-19 09:14:12 -05:00
Jason Lowe db88b6179d Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe"
This reverts commit b3d61304f2.

(cherry picked from commit ea845ba58c)
2017-09-19 08:50:20 -05:00
Akira Ajisaka 9c90400c96
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák.
(cherry picked from commit 2018538fdb)
2017-09-19 11:07:08 +09:00
Andrew Wang 531290c2c8 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory.
(cherry picked from commit 3cf3540f19)
2017-09-18 15:13:45 -07:00
Jason Lowe 8894654fde MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit b3d61304f2)
2017-09-18 17:06:02 -05:00
Robert Kanter 917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter)
(cherry picked from commit 0adc0471d0)
2017-09-18 10:42:01 -07:00
Robert Kanter 3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
This reverts commit 293ab432e1.
2017-09-18 10:41:46 -07:00
Robert Kanter 293ab432e1 YARN-7162. Remove XML excludes file format (rkanter)
(cherry picked from commit 3a8d57a0a2)
2017-09-15 12:01:41 -07:00
Steve Loughran b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
Contributed by Steve Loughran

(cherry picked from commit 11390c2d11)
2017-09-15 17:00:16 +01:00
Andrew Wang ad0fe570d9 HADOOP-14089. Automated checking for malformed client. Contributed by Sean Busbey.
(cherry picked from commit c3f35c422b)
2017-09-13 16:57:53 -07:00
Jason Lowe 12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim
(cherry picked from commit 4d98936eec)
2017-09-13 17:22:29 -05:00
Ray Chiang 656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)
(cherry picked from commit 556812c179)
2017-09-12 10:19:58 -07:00
Robert Kanter f5f343a665 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter)
(cherry picked from commit a323f73bae)
2017-09-08 13:14:52 -07:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
Konstantin V Shvachko 3e0e2033cc MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 2017-08-30 14:07:23 -07:00
Haibo Chen d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Jian He 08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S 7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Sangjin Lee 684de1a902 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-08-30 11:29:52 +05:30