Commit Graph

1828 Commits

Author SHA1 Message Date
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Arun Suresh 72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh)
(cherry picked from commit 5eb7dbe9b3)
2017-11-09 15:16:55 -08:00
Akira Ajisaka a536655c63
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling.
(cherry picked from commit e4c220ee4f)
2017-11-08 19:28:59 +09:00
Akira Ajisaka d77f17458c
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák.
(cherry picked from commit ffee10b68e)
2017-11-08 19:23:51 +09:00
Naganarasimha 8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph.
(cherry picked from commit 61bbdc511e)
2017-11-06 12:59:52 +05:30
Xiao Chen 1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED.
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
2017-11-02 21:27:43 -07:00
Akira Ajisaka 12f92e636e
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling.
(cherry picked from commit 178751ed8c)
2017-11-02 17:43:55 +09:00
Daniel Templeton 7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e810838)
2017-10-31 15:06:18 -07:00
Wangda Tan 4af3deaeb0 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda)
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541a)
2017-10-31 15:06:17 -07:00
Jason Lowe 75605766fd HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe
(cherry picked from commit 2fae63aa60)
2017-10-31 09:32:02 -05:00
Jason Lowe 00345eed50 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla
(cherry picked from commit 665bb147aa)
2017-10-27 17:00:31 -05:00
Akira Ajisaka a887016e1d
MAPREDUCE-6972. Enable try-with-resources for RecordReader. Contributed by Zoltan Haindrich.
(cherry picked from commit 2523e1cce5)
2017-10-18 11:19:00 +09:00
Chris Trezzo 24b03eb79f MAPREDUCE-5951. Add support for the YARN Shared Cache.
(cherry picked from commit e46d5bb962)
2017-10-12 11:15:46 -07:00
Akira Ajisaka 47aa044fcc
MAPREDUCE-6971. Moving logging APIs over to slf4j in hadoop-mapreduce-client-app. Contributed by Jinjiang Ling.
(cherry picked from commit 453d48bdfb)
2017-10-03 12:15:43 +09:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Jason Lowe bf03363d83 MAPREDUCE-6968. Staging directory erasure coding config property has a typo. Contributed by Jason Lowe
(cherry picked from commit 9df05005ac)
2017-09-26 14:53:47 -05:00
Eric Payne 839d5b124d MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure.
(cherry picked from commit 595d478408)
2017-09-19 10:40:41 -05:00
Jason Lowe 731a622627 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit 3a20debdde)
2017-09-19 09:14:12 -05:00
Jason Lowe db88b6179d Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe"
This reverts commit b3d61304f2.

(cherry picked from commit ea845ba58c)
2017-09-19 08:50:20 -05:00
Andrew Wang 531290c2c8 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory.
(cherry picked from commit 3cf3540f19)
2017-09-18 15:13:45 -07:00
Jason Lowe 8894654fde MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit b3d61304f2)
2017-09-18 17:06:02 -05:00
Robert Kanter 917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter)
(cherry picked from commit 0adc0471d0)
2017-09-18 10:42:01 -07:00
Robert Kanter 3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
This reverts commit 293ab432e1.
2017-09-18 10:41:46 -07:00
Robert Kanter 293ab432e1 YARN-7162. Remove XML excludes file format (rkanter)
(cherry picked from commit 3a8d57a0a2)
2017-09-15 12:01:41 -07:00
Steve Loughran b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
Contributed by Steve Loughran

(cherry picked from commit 11390c2d11)
2017-09-15 17:00:16 +01:00
Andrew Wang ad0fe570d9 HADOOP-14089. Automated checking for malformed client. Contributed by Sean Busbey.
(cherry picked from commit c3f35c422b)
2017-09-13 16:57:53 -07:00
Jason Lowe 12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim
(cherry picked from commit 4d98936eec)
2017-09-13 17:22:29 -05:00
Ray Chiang 656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)
(cherry picked from commit 556812c179)
2017-09-12 10:19:58 -07:00
Robert Kanter f5f343a665 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter)
(cherry picked from commit a323f73bae)
2017-09-08 13:14:52 -07:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
Konstantin V Shvachko 3e0e2033cc MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 2017-08-30 14:07:23 -07:00
Haibo Chen d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Jian He 08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S 7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Sangjin Lee 684de1a902 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-08-30 11:29:52 +05:30
Sangjin Lee 8499299740 MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu. 2017-08-30 11:29:51 +05:30
Sangjin Lee 092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Mingliang Liu ae7abad3d9 MAPREDUCE-6945. TestMapFileOutputFormat missing @after annotation. Contributed by Ajay Kumar 2017-08-27 15:19:55 -07:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Robert Kanter ab051bd42e MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter) 2017-08-16 16:14:04 -07:00
Jason Lowe 0acc5e0036 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:34:06 -05:00
Chris Douglas 645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Lei Xu ce797a1706 HDFS-12221. Replace xcerces in XmlEditsVisitor. (Ajay Kumar via lei) 2017-08-14 10:27:47 -07:00
Allen Wittenauer 4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Haibo Chen a32e0138fb MAPREDUCE-6870. Add configuration for MR job to finish when all reducers are complete. (Peter Bacsko via Haibo Chen) 2017-08-10 15:17:36 -07:00
Ravi Prakash ac7d0604bc MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke. 2017-08-09 15:39:52 -07:00
Jason Lowe 735fce5bec MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 2017-08-08 14:46:47 -05:00