Commit Graph

2646 Commits

Author SHA1 Message Date
Akira Ajisaka a887016e1d
MAPREDUCE-6972. Enable try-with-resources for RecordReader. Contributed by Zoltan Haindrich.
(cherry picked from commit 2523e1cce5)
2017-10-18 11:19:00 +09:00
Chris Trezzo 24b03eb79f MAPREDUCE-5951. Add support for the YARN Shared Cache.
(cherry picked from commit e46d5bb962)
2017-10-12 11:15:46 -07:00
Akira Ajisaka 47aa044fcc
MAPREDUCE-6971. Moving logging APIs over to slf4j in hadoop-mapreduce-client-app. Contributed by Jinjiang Ling.
(cherry picked from commit 453d48bdfb)
2017-10-03 12:15:43 +09:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Jason Lowe bf03363d83 MAPREDUCE-6968. Staging directory erasure coding config property has a typo. Contributed by Jason Lowe
(cherry picked from commit 9df05005ac)
2017-09-26 14:53:47 -05:00
Jason Lowe 5eb1f9f7b3 MAPREDUCE-6965. QuasiMonteCarlo should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
(cherry picked from commit 908d8e914e)
2017-09-22 09:36:29 -05:00
Akira Ajisaka cbb3294e2d
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari.
(cherry picked from commit c71d137960)
2017-09-22 15:10:01 +09:00
Akira Ajisaka 0af3aec502
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergely Novák.
(cherry picked from commit 53be075241)
2017-09-22 13:30:22 +09:00
Jason Lowe 6c07077837 MAPREDUCE-6964. BaileyBorweinPlouffe should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
(cherry picked from commit b9db0afa1e)
2017-09-21 09:39:01 -05:00
Akira Ajisaka 835717210a
Revert "MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák."
This reverts commit 9c90400c96.
2017-09-21 11:17:35 +09:00
Andrew Wang 5c8331c230 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong.
(cherry picked from commit 51edaacd09)
2017-09-19 13:48:25 -07:00
Eric Payne 839d5b124d MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure.
(cherry picked from commit 595d478408)
2017-09-19 10:40:41 -05:00
Jason Lowe 731a622627 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit 3a20debdde)
2017-09-19 09:14:12 -05:00
Jason Lowe db88b6179d Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe"
This reverts commit b3d61304f2.

(cherry picked from commit ea845ba58c)
2017-09-19 08:50:20 -05:00
Akira Ajisaka 9c90400c96
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák.
(cherry picked from commit 2018538fdb)
2017-09-19 11:07:08 +09:00
Andrew Wang 531290c2c8 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory.
(cherry picked from commit 3cf3540f19)
2017-09-18 15:13:45 -07:00
Jason Lowe 8894654fde MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit b3d61304f2)
2017-09-18 17:06:02 -05:00
Robert Kanter 917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter)
(cherry picked from commit 0adc0471d0)
2017-09-18 10:42:01 -07:00
Robert Kanter 3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
This reverts commit 293ab432e1.
2017-09-18 10:41:46 -07:00
Robert Kanter 293ab432e1 YARN-7162. Remove XML excludes file format (rkanter)
(cherry picked from commit 3a8d57a0a2)
2017-09-15 12:01:41 -07:00
Steve Loughran b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
Contributed by Steve Loughran

(cherry picked from commit 11390c2d11)
2017-09-15 17:00:16 +01:00
Andrew Wang ad0fe570d9 HADOOP-14089. Automated checking for malformed client. Contributed by Sean Busbey.
(cherry picked from commit c3f35c422b)
2017-09-13 16:57:53 -07:00
Jason Lowe 12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim
(cherry picked from commit 4d98936eec)
2017-09-13 17:22:29 -05:00
Ray Chiang 656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)
(cherry picked from commit 556812c179)
2017-09-12 10:19:58 -07:00
Robert Kanter f5f343a665 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter)
(cherry picked from commit a323f73bae)
2017-09-08 13:14:52 -07:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
Konstantin V Shvachko 3e0e2033cc MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 2017-08-30 14:07:23 -07:00
Haibo Chen d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Jian He 08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S 7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Sangjin Lee 684de1a902 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-08-30 11:29:52 +05:30
Sangjin Lee 8499299740 MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu. 2017-08-30 11:29:51 +05:30
Sangjin Lee 092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Mingliang Liu ae7abad3d9 MAPREDUCE-6945. TestMapFileOutputFormat missing @after annotation. Contributed by Ajay Kumar 2017-08-27 15:19:55 -07:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Andrew Wang dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Robert Kanter ab051bd42e MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter) 2017-08-16 16:14:04 -07:00
Jason Lowe 0acc5e0036 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:34:06 -05:00
Chris Douglas 645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Lei Xu ce797a1706 HDFS-12221. Replace xcerces in XmlEditsVisitor. (Ajay Kumar via lei) 2017-08-14 10:27:47 -07:00
Allen Wittenauer 4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Haibo Chen a32e0138fb MAPREDUCE-6870. Add configuration for MR job to finish when all reducers are complete. (Peter Bacsko via Haibo Chen) 2017-08-10 15:17:36 -07:00
Ravi Prakash ac7d0604bc MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke. 2017-08-09 15:39:52 -07:00
Jason Lowe 735fce5bec MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 2017-08-08 14:46:47 -05:00
Arun Suresh 35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Yufei Gu b8e8241854 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:44:34 -07:00
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Akira Ajisaka ceacadc51e
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. 2017-08-01 14:56:42 +09:00