d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-31 18:31:10 +09:00
88cbe57c06
MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan.
2018-05-28 14:05:49 +05:30
a2cdffb95a
MAPREDUCE-7094. LocalDistributedCacheManager leaves classloaders open, which leaks FDs. Contributed by Adam Szita.
2018-05-17 11:16:04 -07:00
274eee3284
MAPREDUCE-7095. Race conditions in closing FadvisedChunkedFile. (Miklos Szegedi via Haibo Chen)
2018-05-10 10:46:55 -07:00
c8b53c4364
YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter)
2018-05-10 09:31:59 -07:00
e9159db623
MAPREDUCE-7093. Use assertEquals instead of assertTrue(a == b) in TestMapReduceJobControlWithMocks
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-07 01:15:14 +09:00
8cdb032aff
YARN-8223. Improved yarn auxiliary service to load jar file from HDFS.
...
Contributed by Zian Chen
2018-05-04 12:36:31 -04:00
1a95a4524a
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-02 16:14:28 +09:00
68c6ec719d
MAPREDUCE-7086. Add config to allow FileInputFormat to ignore directories when recursive=false. Contributed by Sergey Shelukhin
2018-05-01 16:19:53 -05:00
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-27 19:21:37 +09:00
83e60cd2db
MAPREDUCE-7042. Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled. Contributed by Rohith Sharma K S.
2018-04-26 13:54:40 -07:00
42e248372c
Revert "HDFS-12515. Ozone: mvn package compilation fails on HDFS-7240. Contributed by Anu Engineer."
...
This fix was need because of change in Trunk. That patch has been reverted and change proposed by this
patch will be applied in trunk. So reverting now to match the changes in trunk. Please see the JIRA for
details.
This reverts commit 244e7a5f65
2018-04-26 05:36:04 -07:00
c14e8b17ec
HDFS-12515. Ozone: mvn package compilation fails on HDFS-7240. Contributed by Anu Engineer.
2018-04-26 05:36:04 -07:00
1b9ecc264a
MAPREDUCE-7072. mapred job -history prints duplicate counter in human output (wilfreds via rkanter)
2018-04-24 11:30:38 -07:00
e4c39f3247
HADOOP-15396. Some java source files are executable
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-19 08:35:38 +09:00
bb92bfb4ef
YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)
2018-04-17 11:15:20 -07:00
86dfb70875
Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)"
...
This reverts commit 0c14b6f0d8
2018-04-17 06:04:38 -07:00
0c14b6f0d8
YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)
2018-04-17 05:59:03 -07:00
995cba65fe
MAPREDUCE-7077. Pipe mapreduce job fails with Permission denied for jobTokenPassword. (Akira Ajisaka via wangda)
...
Change-Id: Ie8f01425d58409fa3661f768205b7616128c8aa4
(cherry picked from commit 035e0f97ea
2018-04-13 13:52:10 -07:00
1a407bc990
HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of format option. Contributed by Erik Krogen
2018-04-12 23:27:51 -07:00
4571351ccc
MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan
2018-04-12 11:12:46 -05:00
6bb128dfb8
YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe.
2018-04-12 16:04:23 +00:00
6cc59a09e7
MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri.
2018-04-11 09:45:39 +05:30
6cf023f9b7
Added CHANGES/RELEASES/Jdiff for 3.1.0 release
...
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
2018-04-05 15:52:39 -07:00
345e7624d5
YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S
2018-04-04 15:08:03 -07:00
dde1579096
HADOOP-15146. Remove DataOutputByteBuffer.
...
Contributed by BELUGA BEHR.
2018-04-02 15:52:40 +01:00
edb202e493
MAPREDUCE-6441. Improve temporary directory name generation in LocalDistributedCacheManager for concurrent processes (wattsinabox, rchiang, haibochen via rkanter)
2018-03-26 14:55:53 -07:00
86816da5b4
MAPREDUCE-7066. TestQueue fails on Java9
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-19 09:39:49 +09:00
2626ec35e9
MAPREDUCE-7064. Flaky test TestTaskAttempt#testReducerCustomResourceTypes. Contributed by Peter Bacsko
2018-03-14 15:05:24 -05:00
3a8dade9b1
Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu."
...
This reverts commit 172b23af33
2018-03-09 22:46:30 -08:00
32fa3a63e0
MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao
2018-03-09 10:41:16 -06:00
4971276631
HADOOP-15271. Remove unicode multibyte characters from JavaDoc
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-05 15:07:58 -08:00
6e6945cd78
MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang.
2018-03-01 10:18:53 +08:00
7dd385098c
MAPREDUCE-7010. Make Job History File Permissions configurable. Contributed by Gergely Novák
2018-02-26 14:32:46 -08:00
2fa7963c3d
HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.
2018-02-24 14:41:55 -08:00
2bc3351eaf
HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel.
2018-02-21 12:57:14 -07:00
0898ff42e9
HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka
2018-02-16 14:49:00 -06:00
82f029f7b5
MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe.
2018-02-16 08:15:09 -06:00
a53d62ab26
MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko
2018-02-15 15:12:57 -06:00
87e2570a14
MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko
2018-02-12 13:21:09 -06:00
60f9e60b3b
Preparing for 3.2.0 development
...
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
2018-02-11 11:17:38 +08:00
4304fcd5bd
HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen.
2018-02-06 13:43:45 -08:00
5a725bb886
MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe
2018-01-31 15:45:30 -06:00
38af237969
YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh)
2018-01-31 01:30:17 -08:00
56feaa40bb
MAPREDUCE-7036. ASF License warning in hadoop-mapreduce-client
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-01-30 00:39:39 +09:00
6eef3d7f1a
MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko
2018-01-26 15:31:43 -06:00
a37e7f0ad8
MAPREDUCE-7022. Fast fail rogue jobs based on task scratch dir size. Contributed by Johan Gustavsson
2018-01-26 14:36:45 -06:00
cc10852252
MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen)
2018-01-25 16:11:30 -08:00
cff9edd4b5
MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko
2018-01-24 14:44:07 -06:00
836643d793
MAPREDUCE-6995. Uploader tool for Distributed Cache Deploy documentation (miklos.szegedi@cloudera.com via rkanter)
2018-01-19 17:57:54 -08:00
c191538ed1
HADOOP-15166 CLI MiniCluster fails with ClassNotFoundException o.a.h.yarn.server.timelineservice.collector.TimelineCollectorManager. Contributed by Gera Shegalov
2018-01-19 16:15:55 -08:00
cce71dceef
MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen)
2018-01-19 12:56:17 -08:00
6e42d05829
MAPREDUCE-7029. FileOutputCommitter is slow on filesystems lacking recursive delete. Contributed by Karthik Palaniappan
2018-01-17 08:14:11 -06:00
d716084f45
MAPREDUCE-7032. Add the ability to specify a delayed replication count (miklos.szegedi@cloudera.com via rkanter)
2018-01-16 10:45:45 -08:00
1a9c5d479e
MAPREDUCE-7034. Moving logging APIs over to slf4j the rest of all in hadoop-mapreduce
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-01-15 15:38:03 +09:00
e404650f48
MAPREDUCE-7030. Uploader tool should ignore symlinks to the same directory (miklos.szegedi@cloudera.com via rkanter)
2018-01-12 14:18:01 -08:00
fe35103591
MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas
2018-01-03 11:01:38 -06:00
a7f8caf58e
Add 2.8.3 release jdiff files.
...
(cherry picked from commit c89f99aade
2017-12-18 22:39:01 -08:00
c7a4dda3c5
HADOOP-15109. TestDFSIO -read -random doesn't work on file sized 4GB. Contributed by Ajay Kumar.
2017-12-18 13:25:47 -08:00
2316f52690
MAPREDUCE-7018. Apply erasure coding properly to framework tarball and support plain tar (miklos.szegedi@cloudera.com via rkanter)
2017-12-11 14:00:42 -08:00
c2e8a5c229
MAPREDUCE-7000. Moving logging APIs over to slf4j in hadoop-mapreduce-client-nativetask. Contributed by Jinjiang Ling.
2017-12-07 16:27:08 +09:00
d4cae977a2
MAPREDUCE-6998. Moving logging APIs over to slf4j in hadoop-mapreduce-client-jobclient. Contributed by Gergely Novák.
2017-12-07 16:21:25 +09:00
e00c7f78c1
HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts.
2017-12-04 21:02:04 -08:00
3b78607a02
MAPREDUCE-6994. Uploader tool for Distributed Cache Deploy code changes (miklos.szegedi@cloudera.com via rkanter)
2017-12-01 12:12:15 -08:00
21d3627355
MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko
2017-12-01 14:04:25 -06:00
3cd75845da
MAPREDUCE-7014. Fix java doc errors in jdk1.8. Contributed by Steve Loughran.
2017-11-27 22:01:00 +05:30
de8b6ca5ef
HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints.
...
Contributed by Steve Loughran and Ryan Blue.
2017-11-22 15:28:12 +00:00
782ba3bf9d
MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec
2017-11-21 20:42:28 -08:00
5eb7dbe9b3
Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh)
2017-11-09 15:15:51 -08:00
e4c220ee4f
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling.
2017-11-08 19:28:08 +09:00
ffee10b68e
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák.
2017-11-08 19:21:43 +09:00
61bbdc511e
MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph.
2017-11-06 12:58:38 +05:30
e565b5277d
HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED.
2017-11-02 21:27:35 -07:00
178751ed8c
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling.
2017-11-02 17:43:08 +09:00
2fae63aa60
HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe
2017-10-31 09:30:13 -05:00
9a7e810838
YARN-6927. Add support for individual resource types requests in MapReduce
...
(Contributed by Gergo Repas via Daniel Templeton)
2017-10-30 11:04:22 -07:00
665bb147aa
YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla
2017-10-27 16:56:05 -05:00
de09716fb8
MAPREDUCE-6977 Move logging APIs over to slf4j in hadoop-mapreduce-client-common.
...
Contributed by Jinjiang Ling.
2017-10-27 10:45:03 +01:00
a25b5aa0cf
Add several jdiff xml files for 2.8.2 release.
2017-10-26 19:30:48 -07:00
2523e1cce5
MAPREDUCE-6972. Enable try-with-resources for RecordReader. Contributed by Zoltan Haindrich.
2017-10-18 11:18:39 +09:00
e46d5bb962
MAPREDUCE-5951. Add support for the YARN Shared Cache.
2017-10-12 10:59:20 -07:00
639f98cc9d
MAPREDUCE-6951. Improve exception message when mapreduce.jobhistory.webapp.address is in wrong format. Contributed by Prabhu Joseph.
2017-10-11 14:27:38 +05:30
453d48bdfb
MAPREDUCE-6971. Moving logging APIs over to slf4j in hadoop-mapreduce-client-app. Contributed by Jinjiang Ling.
2017-10-03 12:14:54 +09:00
9df05005ac
MAPREDUCE-6968. Staging directory erasure coding config property has a typo. Contributed by Jason Lowe
2017-09-26 14:52:27 -05:00
908d8e914e
MAPREDUCE-6965. QuasiMonteCarlo should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
2017-09-22 09:34:53 -05:00
c71d137960
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari.
2017-09-22 15:07:59 +09:00
53be075241
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergely Novák.
2017-09-22 13:27:59 +09:00
b9db0afa1e
MAPREDUCE-6964. BaileyBorweinPlouffe should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari
2017-09-21 09:37:19 -05:00
53047f934e
Revert "MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák."
...
This reverts commit 2018538fdb
2017-09-21 11:16:05 +09:00
51edaacd09
HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong.
2017-09-19 13:48:23 -07:00
595d478408
MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure.
2017-09-19 10:35:15 -05:00
3a20debdde
MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
2017-09-19 09:13:17 -05:00
ea845ba58c
Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe"
...
This reverts commit b3d61304f2
2017-09-19 08:45:05 -05:00
2018538fdb
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák.
2017-09-19 11:05:54 +09:00
3cf3540f19
HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory.
2017-09-18 15:13:42 -07:00
b3d61304f2
MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
2017-09-18 17:04:43 -05:00
0adc0471d0
MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter)
2017-09-18 10:40:06 -07:00
5f496683fb
Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
...
This reverts commit 3a8d57a0a2
2017-09-18 10:32:08 -07:00
3a8d57a0a2
YARN-7162. Remove XML excludes file format (rkanter)
2017-09-15 12:00:26 -07:00
11390c2d11
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
...
Contributed by Steve Loughran
2017-09-15 16:59:04 +01:00
Takanobu Asanuma
Rohith Sharma K S
Miklos Szegedi
Haibo Chen
Robert Kanter
Abhishek Modi
Eric Yang
Bibin A Chundatt
Jason Lowe
Ewan Higgs
Sunil G
Anu Engineer
Shashikant Banerjee
Wangda Tan
Erik Krogen
Eric E Payne
Vrushali C
Steve Loughran
Owen O'Malley
Weiwei Yang
Billie Rinaldi
Arpit Agarwal
Sean Mackrory
Eric Payne
Arun Suresh
Junping Du
Chen Liang
Akira Ajisaka
Chris Douglas
Naganarasimha
Xiao Chen
Daniel Templeton
Chris Trezzo
Andrew Wang