e0b3cbd221
HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota.
2019-06-13 07:53:40 -06:00
a9a3450560
HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:34:16 +09:00
855dc997d6
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:27:33 +09:00
e4011e6886
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event.
...
This closes #821
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165
2019-05-22 14:25:34 +09:00
ddf8859655
MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 856cbf62d3
2019-04-01 10:38:58 -07:00
60c9042286
HADOOP-16058. S3A tests to include Terasort.
...
Contributed by Steve Loughran.
This includes
- HADOOP-15890. Some S3A committer tests don't match ITest* pattern; don't run in maven
- MAPREDUCE-7090. BigMapOutput example doesn't work with paths off cluster fs
- MAPREDUCE-7091. Terasort on S3A to switch to new committers
- MAPREDUCE-7092. MR examples to work better against cloud stores
2019-03-29 15:25:45 +00:00
9a449ac075
HADOOP-16196. Path Parameterize Comparable.
...
Author: David Mollitor <david.mollitor@cloudera.com>
(cherry picked from commit 246ab77f28
2019-03-22 10:27:17 +00:00
3d86dd8931
MAPREDUCE-7192. JobHistoryServer attempts page support jump to containers log page in NM when logAggregation is disable. Contributed by Jiandan Yang.
...
(cherry picked from commit 159a715eef
2019-03-13 17:41:31 +08:00
9a2b24642d
MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang.
...
(cherry picked from commit f0605146b3
2019-03-11 16:03:38 +08:00
649da5af04
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa
2019-02-05 14:05:51 -08:00
97fa326b79
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin.
...
(cherry picked from commit c991e2ca5a
2019-02-04 14:04:07 +09:00
0d78725fa8
MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui.
...
(cherry picked from commit 9e7196749e
2019-01-02 21:48:57 +08:00
495285e50e
MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin.
...
(cherry picked from commit e0d75088f2
2018-12-13 16:27:52 +09:00
298a1f29e8
MAPREDUCE-7159. FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used. Contributed by Peter Bacsko
...
(cherry picked from commit 9886c27c71
2018-12-06 14:57:45 -06:00
7a78bdf7bb
MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla
...
(cherry picked from commit 4d8de7ab69
2018-11-28 15:54:59 -06:00
2e6b9b174e
MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin.
...
(cherry picked from commit 1aad99a718
2018-11-27 20:02:11 +09:00
f214af74b2
MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda)
...
Change-Id: I99ace87980da03bb35a8012cea7218d602a8817a
(cherry picked from commit 762a56cc64
2018-11-13 11:35:13 -08:00
ff5a2cb5c9
MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko
...
(cherry picked from commit ba1f9d66d9
2018-11-06 17:57:09 -06:00
9265934201
YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
...
(cherry picked from commit ab6aa4c726
2018-11-06 08:47:30 -06:00
aed71316e5
MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T.
...
(cherry picked from commit 74a5e683fe
2018-10-25 00:18:39 +05:30
bde4fd5ed9
Preparing for 3.2.0 release
2018-10-18 17:07:45 +05:30
424c99681b
MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko.
...
(cherry picked from commit 25f8fcb064
2018-10-16 10:24:15 -07:00
230d90c5b9
MAPREDUCE-7125. JobResourceUploader creates LocalFileSystem when it's not necessary. (Peter Cseh via wangda)
...
Change-Id: I1aa720ed03739f6f4abeec46f6068e2ab332987a
2018-09-24 13:52:53 -07:00
8382b860d4
MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko
2018-09-18 17:06:32 -05:00
34b2237e42
MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko
2018-09-18 16:56:31 -05:00
2cf8927683
MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe
2018-09-18 20:26:28 +00:00
488806baca
MAPREDUCE-7140. Refactoring TaskAttemptInfo to separate Map and Reduce tasks. Contributed by Oleksandr Shevchenko
2018-09-14 15:10:27 -05:00
2886024ac3
MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko
2018-09-13 14:41:38 -05:00
8cf6a9a2bd
YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R.
2018-09-12 16:01:00 +05:30
5dc7d6e0f3
YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
0a01b1350d
YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt.
2018-09-12 16:01:00 +05:30
598380ad5a
MAPREDUCE-7136. TestMRAppMetrics should shutdown DefaultMetricsSystem after completion. Contributed by Oleksandr Shevchenko
2018-09-11 13:10:50 -05:00
7ce997af97
MAPREDUCE-7135. TestTaskAttemptContainerRequest should reset UserGroupInformation. Contributed by Oleksandr Shevchenko
2018-09-11 11:16:33 -05:00
eb0b5a844f
MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu
2018-09-06 14:13:29 -05:00
5a0babf765
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs.
...
Contributed by Steve Loughran.
2018-08-30 14:49:53 +01:00
84973d1049
MAPREDUCE-6861. Add metrics tags for ShuffleClientMetrics. (Contributed by Zoltan Siegl)
2018-08-27 16:53:06 +02:00
77b015000a
HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor.
2018-08-17 11:10:29 +02:00
b3b4d4ccb5
MAPREDUCE-7118. Distributed cache conflicts breaks backwards compatability. (Jason Lowe via wangda)
...
Change-Id: I89ab4852b4ad305fec19812e8931c59d96581376
2018-07-19 14:26:05 -07:00
88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma.
2018-07-13 11:42:12 -04:00
7a3c6e9c3c
HADOOP-15550. Avoid static initialization of ObjectMappers
2018-06-25 15:36:45 -07:00
ae055622ed
MAPREDUCE-7114. Make FrameworkUploader symlink ignore improvement. Contributed by Gergo Repas.
2018-06-22 13:05:41 -07:00
d6ee4290df
MAPREDUCE-7113. Typos in test names in TestTaskAttempt: "testAppDiognostic". Contributed by Szilard Nemeth.
2018-06-20 12:04:44 -07:00
1e94e5977f
MAPREDUCE-7063. Fix log level inconsistency in CombineFileInputFormat.java
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-06-18 14:25:11 -07:00
5670e89b2e
MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via asuresh)
2018-06-12 15:36:52 -07:00
04b74eddc6
MAPREDUCE-7108. TestFileOutputCommitter fails on Windows. (Zuoming Zhang via gera)
2018-06-12 11:21:51 -07:00
0afc036deb
MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola.
2018-06-05 12:24:57 -07:00
6d5e87aec2
HADOOP-15507. Add MapReduce counters about EC bytes read.
2018-06-04 21:13:47 -07:00
dad1bb868f
MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola.
2018-06-04 12:55:54 -07:00
a804b7c9d2
MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola.
2018-06-02 20:25:42 -07:00
d5e69d8994
MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user(addendum). Contributed by Sunil Govindan.
2018-06-02 09:27:03 +05:30
d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-31 18:31:10 +09:00
88cbe57c06
MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan.
2018-05-28 14:05:49 +05:30
a2cdffb95a
MAPREDUCE-7094. LocalDistributedCacheManager leaves classloaders open, which leaks FDs. Contributed by Adam Szita.
2018-05-17 11:16:04 -07:00
274eee3284
MAPREDUCE-7095. Race conditions in closing FadvisedChunkedFile. (Miklos Szegedi via Haibo Chen)
2018-05-10 10:46:55 -07:00
c8b53c4364
YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter)
2018-05-10 09:31:59 -07:00
e9159db623
MAPREDUCE-7093. Use assertEquals instead of assertTrue(a == b) in TestMapReduceJobControlWithMocks
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-07 01:15:14 +09:00
8cdb032aff
YARN-8223. Improved yarn auxiliary service to load jar file from HDFS.
...
Contributed by Zian Chen
2018-05-04 12:36:31 -04:00
1a95a4524a
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-02 16:14:28 +09:00
68c6ec719d
MAPREDUCE-7086. Add config to allow FileInputFormat to ignore directories when recursive=false. Contributed by Sergey Shelukhin
2018-05-01 16:19:53 -05:00
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-27 19:21:37 +09:00
83e60cd2db
MAPREDUCE-7042. Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled. Contributed by Rohith Sharma K S.
2018-04-26 13:54:40 -07:00
42e248372c
Revert "HDFS-12515. Ozone: mvn package compilation fails on HDFS-7240. Contributed by Anu Engineer."
...
This fix was need because of change in Trunk. That patch has been reverted and change proposed by this
patch will be applied in trunk. So reverting now to match the changes in trunk. Please see the JIRA for
details.
This reverts commit 244e7a5f65
2018-04-26 05:36:04 -07:00
c14e8b17ec
HDFS-12515. Ozone: mvn package compilation fails on HDFS-7240. Contributed by Anu Engineer.
2018-04-26 05:36:04 -07:00
1b9ecc264a
MAPREDUCE-7072. mapred job -history prints duplicate counter in human output (wilfreds via rkanter)
2018-04-24 11:30:38 -07:00
e4c39f3247
HADOOP-15396. Some java source files are executable
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-19 08:35:38 +09:00
bb92bfb4ef
YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)
2018-04-17 11:15:20 -07:00
86dfb70875
Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)"
...
This reverts commit 0c14b6f0d8
2018-04-17 06:04:38 -07:00
0c14b6f0d8
YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)
2018-04-17 05:59:03 -07:00
995cba65fe
MAPREDUCE-7077. Pipe mapreduce job fails with Permission denied for jobTokenPassword. (Akira Ajisaka via wangda)
...
Change-Id: Ie8f01425d58409fa3661f768205b7616128c8aa4
(cherry picked from commit 035e0f97ea
2018-04-13 13:52:10 -07:00
1a407bc990
HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of format option. Contributed by Erik Krogen
2018-04-12 23:27:51 -07:00
4571351ccc
MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan
2018-04-12 11:12:46 -05:00
6bb128dfb8
YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe.
2018-04-12 16:04:23 +00:00
6cc59a09e7
MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri.
2018-04-11 09:45:39 +05:30
6cf023f9b7
Added CHANGES/RELEASES/Jdiff for 3.1.0 release
...
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
2018-04-05 15:52:39 -07:00
345e7624d5
YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S
2018-04-04 15:08:03 -07:00
dde1579096
HADOOP-15146. Remove DataOutputByteBuffer.
...
Contributed by BELUGA BEHR.
2018-04-02 15:52:40 +01:00
edb202e493
MAPREDUCE-6441. Improve temporary directory name generation in LocalDistributedCacheManager for concurrent processes (wattsinabox, rchiang, haibochen via rkanter)
2018-03-26 14:55:53 -07:00
86816da5b4
MAPREDUCE-7066. TestQueue fails on Java9
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-19 09:39:49 +09:00
2626ec35e9
MAPREDUCE-7064. Flaky test TestTaskAttempt#testReducerCustomResourceTypes. Contributed by Peter Bacsko
2018-03-14 15:05:24 -05:00
3a8dade9b1
Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu."
...
This reverts commit 172b23af33
2018-03-09 22:46:30 -08:00
32fa3a63e0
MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao
2018-03-09 10:41:16 -06:00
4971276631
HADOOP-15271. Remove unicode multibyte characters from JavaDoc
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-05 15:07:58 -08:00
6e6945cd78
MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang.
2018-03-01 10:18:53 +08:00
7dd385098c
MAPREDUCE-7010. Make Job History File Permissions configurable. Contributed by Gergely Novák
2018-02-26 14:32:46 -08:00
2fa7963c3d
HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi.
2018-02-24 14:41:55 -08:00
2bc3351eaf
HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel.
2018-02-21 12:57:14 -07:00
0898ff42e9
HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka
2018-02-16 14:49:00 -06:00
82f029f7b5
MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe.
2018-02-16 08:15:09 -06:00
a53d62ab26
MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko
2018-02-15 15:12:57 -06:00
87e2570a14
MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko
2018-02-12 13:21:09 -06:00
60f9e60b3b
Preparing for 3.2.0 development
...
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
2018-02-11 11:17:38 +08:00
4304fcd5bd
HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen.
2018-02-06 13:43:45 -08:00
5a725bb886
MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe
2018-01-31 15:45:30 -06:00
38af237969
YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh)
2018-01-31 01:30:17 -08:00
56feaa40bb
MAPREDUCE-7036. ASF License warning in hadoop-mapreduce-client
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-01-30 00:39:39 +09:00
6eef3d7f1a
MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko
2018-01-26 15:31:43 -06:00
a37e7f0ad8
MAPREDUCE-7022. Fast fail rogue jobs based on task scratch dir size. Contributed by Johan Gustavsson
2018-01-26 14:36:45 -06:00
cc10852252
MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen)
2018-01-25 16:11:30 -08:00
cff9edd4b5
MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko
2018-01-24 14:44:07 -06:00
836643d793
MAPREDUCE-6995. Uploader tool for Distributed Cache Deploy documentation (miklos.szegedi@cloudera.com via rkanter)
2018-01-19 17:57:54 -08:00
Sean Mackrory
Takanobu Asanuma
Akira Ajisaka
Wanqiang Ji
Adam Antal
Steve Loughran
David Mollitor
Weiwei Yang
Wangda Tan
Jason Lowe
bibinchundatt
Sunil G
Xiao Chen
Eric E Payne
Naganarasimha
Haibo Chen
Márton Elek
Todd Lipcon
Miklos Szegedi
Vidura Mudalige
Arun Suresh
Gera Shegalov
Inigo Goiri
Rohith Sharma K S
Takanobu Asanuma
Miklos Szegedi
Robert Kanter
Abhishek Modi
Eric Yang
Ewan Higgs
Anu Engineer
Shashikant Banerjee
Erik Krogen
Vrushali C
Owen O'Malley
Billie Rinaldi
Arpit Agarwal
Eric Payne