Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,170 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2755 Commits

Author SHA1 Message Date
Szilard Nemeth 639763c561 MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko. 2019-08-01 16:08:14 +02:00
pingsutw 1a2aba85d8
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build 
This closes #1089

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ee87e9a42e)
 2019-07-23 10:36:01 +09:00
Sean Mackrory fee1e67453 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:38:43 -06:00
Takanobu Asanuma 8098ddaf40 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:39:49 +09:00
Akira Ajisaka f8bd5deec1
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:21 +09:00
Wanqiang Ji 6690955358
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165)
 2019-05-22 14:41:20 +09:00
Adam Antal 52536ff887 MAPREDUCE-7190. Add SleepJob additional parameter to make parallel runs distinguishable. Contributed by Adam Antal. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 856cbf62d3)
(cherry picked from commit ddf8859655)
 2019-04-01 10:39:18 -07:00
Weiwei Yang fdd56b5b05 MAPREDUCE-7191. JobHistoryServer should log exception when loading/parsing history file failed. Contributed by Jiandan Yang. 
(cherry picked from commit f0605146b3)
 2019-03-11 16:40:01 +08:00
Wangda Tan a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa)
(cherry picked from commit 649da5af04)
 2019-02-05 14:09:33 -08:00
Akira Ajisaka 72b3baf207
MAPREDUCE-7177. Disable speculative execution in TestDFSIO. Contributed by Zhaohui Xin. 
(cherry picked from commit c991e2ca5a)
 2019-02-04 14:04:26 +09:00
Haibo Chen 61a6cc8d23 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 
(cherry picked from commit bb92bfb4ef)
 2019-01-24 15:58:37 -08:00
Wangda Tan 31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Weiwei Yang 2a5b2eda7c MAPREDUCE-7174. Mapreduce example wordmedian should handle generic options. Contributed by Fei Hui. 
(cherry picked from commit 9e7196749e)
 2019-01-02 21:56:52 +08:00
Akira Ajisaka 51b7376a8d
MAPREDUCE-7170. Doc typo in PluggableShuffleAndPluggableSort.md. Contributed by Zhaohui Xin. 
(cherry picked from commit e0d75088f2)
 2018-12-13 16:28:18 +09:00
Jason Lowe 6e6f43afd8 MAPREDUCE-7159. FrameworkUploader: ensure proper permissions of generated framework tar.gz if restrictive umask is used. Contributed by Peter Bacsko 
(cherry picked from commit 9886c27c71)
 2018-12-06 15:02:00 -06:00
Jason Lowe e7fa638fe8 MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla 
(cherry picked from commit 4d8de7ab69)
 2018-11-28 16:01:05 -06:00
Akira Ajisaka 5e2f123748
MAPREDUCE-7162. TestEvents#testEvents fails. Contributed by Zhaohui Xin. 
(cherry picked from commit 1aad99a718)
 2018-11-27 20:02:34 +09:00
Wangda Tan abd9d93a55 MAPREDUCE-7158. Inefficient Flush Logic in JobHistory EventWriter. (Zichen Sun via wangda) 
Change-Id: I99ace87980da03bb35a8012cea7218d602a8817a
(cherry picked from commit 762a56cc64)
 2018-11-13 11:36:04 -08:00
Jason Lowe afbcb51ac1 MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko 
(cherry picked from commit ba1f9d66d9)
 2018-11-06 18:07:57 -06:00
Jason Lowe 7335d940de YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c726)
 2018-11-06 08:52:29 -06:00
bibinchundatt 4167275e89 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T. 
(cherry picked from commit 74a5e683fe)
 2018-10-25 00:27:40 +05:30
Xiao Chen 30526f2c5c MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb064)
 2018-10-16 10:24:24 -07:00
Todd Lipcon 89d4481020 HADOOP-15550. Avoid static initialization of ObjectMappers 
(cherry picked from commit 7a3c6e9c3c)
 2018-10-09 13:50:00 -05:00
Wangda Tan daa14bf06a MAPREDUCE-7125. JobResourceUploader creates LocalFileSystem when it's not necessary. (Peter Cseh via wangda) 
Change-Id: I1aa720ed03739f6f4abeec46f6068e2ab332987a
(cherry picked from commit 230d90c5b9)
 2018-09-24 13:53:51 -07:00
Jason Lowe 3a13fa1b7b MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 8382b860d4)
 2018-09-18 17:12:11 -05:00
Jason Lowe 85ae097f3c MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 34b2237e42)
 2018-09-18 17:08:35 -05:00
Eric E Payne af2390cf49 MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 
(cherry picked from commit 2cf8927683)
 2018-09-18 20:31:33 +00:00
Jason Lowe 1105f54634 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 2886024ac3)
 2018-09-13 14:43:17 -05:00
Jason Lowe 4a55092c6d MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu 
(cherry picked from commit eb0b5a844f)
 2018-09-06 14:29:51 -05:00
Steve Loughran a0766bf66a
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs. 
Contributed by Steve Loughran.

(cherry picked from commit 5a0babf765)
 2018-08-30 15:23:08 +01:00
Wangda Tan 5583711419 Preparing for 3.1.2 release 
Change-Id: If2793e2ed2b5b349a9e1f98f78df43f309dcfcbd
 2018-07-31 13:08:55 -07:00
Wangda Tan 23624c9248 MAPREDUCE-7118. Distributed cache conflicts breaks backwards compatability. (Jason Lowe via wangda) 
Change-Id: I89ab4852b4ad305fec19812e8931c59d96581376
(cherry picked from commit b3b4d4ccb5)
 2018-07-19 14:26:40 -07:00
Wangda Tan 332139e928 MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via wangda) 
Change-Id: I8d8e71c44469fb48f1f5108620843e35e0100411
 2018-06-15 13:18:17 -07:00
Gera Shegalov f2bcb8163a MAPREDUCE-7108. TestFileOutputCommitter fails on Windows. (Zuoming Zhang via gera) 
(cherry picked from commit 04b74eddc6)
 2018-06-12 13:02:41 -07:00
Inigo Goiri b0283fc193 MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 0afc036deb)
 2018-06-05 12:25:29 -07:00
Inigo Goiri 10e35ed322 MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit dad1bb868f)
 2018-06-04 12:56:39 -07:00
Inigo Goiri fce54631c5 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit a804b7c9d2)
 2018-06-02 20:26:17 -07:00
Rohith Sharma K S 61ae43f725 MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user(addendum). Contributed by Sunil Govindan. 
(cherry picked from commit d5e69d8994)
 2018-06-02 09:57:20 +05:30
Rohith Sharma K S f19e8e2246 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 
(cherry picked from commit 88cbe57c06)
 2018-05-28 14:06:50 +05:30
Robert Kanter 0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c4364)
 2018-05-10 09:52:11 -07:00
Eric Yang a984abc472 YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen

(cherry picked from commit 8cdb032aff)
 2018-05-04 12:38:28 -04:00
Jason Lowe daa739fa13 MAPREDUCE-7086. Add config to allow FileInputFormat to ignore directories when recursive=false. Contributed by Sergey Shelukhin 
(cherry picked from commit 68c6ec719d)
 2018-05-03 12:40:26 -05:00
Bibin A Chundatt 254933cf31
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1a95a4524a)
 2018-05-02 16:15:18 +09:00
Sunil G 099e9fb226 MAPREDUCE-7042. Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit 583a2f4edfdf71c32001cc2f2a03d16f9371d533)
 2018-04-26 19:07:48 +05:30
Shashikant Banerjee a13b3edfd8
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247)
 2018-04-19 08:36:32 +09:00
Takanobu Asanuma 077eda66ad MAPREDUCE-7066. TestQueue fails on Java9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-13 10:17:34 -07:00
Erik Krogen e343be46f8 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of format option. Contributed by Erik Krogen 
(cherry picked from commit 1a407bc990)
 2018-04-12 23:31:18 -07:00
Wangda Tan 035e0f97ea MAPREDUCE-7077. Pipe mapreduce job fails with Permission denied for jobTokenPassword. (Akira Ajisaka via wangda) 
Change-Id: Ie8f01425d58409fa3661f768205b7616128c8aa4
 2018-04-12 14:33:33 -07:00
Eric E Payne f955d52f39 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb8)
 2018-04-12 16:11:57 +00:00
Rohith Sharma K S cb6555df24 MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri. 
(cherry picked from commit 6cc59a09e7)
 2018-04-11 09:47:15 +05:30