Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,585 Commits 326 Branches 371 Tags
Commit Graph

1890 Commits

Author SHA1 Message Date
Wangda Tan
5583711419 Preparing for 3.1.2 release 
Change-Id: If2793e2ed2b5b349a9e1f98f78df43f309dcfcbd
 2018-07-31 13:08:55 -07:00
Wangda Tan
23624c9248 MAPREDUCE-7118. Distributed cache conflicts breaks backwards compatability. (Jason Lowe via wangda) 
Change-Id: I89ab4852b4ad305fec19812e8931c59d96581376
(cherry picked from commit b3b4d4ccb53fdf8dacc66e912822b34f8b3bf215)
 2018-07-19 14:26:40 -07:00
Wangda Tan
332139e928 MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via wangda) 
Change-Id: I8d8e71c44469fb48f1f5108620843e35e0100411
 2018-06-15 13:18:17 -07:00
Gera Shegalov
f2bcb8163a MAPREDUCE-7108. TestFileOutputCommitter fails on Windows. (Zuoming Zhang via gera) 
(cherry picked from commit 04b74eddc67888142879ed114d21348e8a4aad78)
 2018-06-12 13:02:41 -07:00
Inigo Goiri
b0283fc193 MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 0afc036deb35df7e86ede3dcebc430c8f05ed368)
 2018-06-05 12:25:29 -07:00
Inigo Goiri
10e35ed322 MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit dad1bb868f7a28a486fd500f102c8c174e6dd272)
 2018-06-04 12:56:39 -07:00
Inigo Goiri
fce54631c5 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit a804b7c9d2986556c91c1741c1ae7ac2b9c579f3)
 2018-06-02 20:26:17 -07:00
Rohith Sharma K S
61ae43f725 MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user(addendum). Contributed by Sunil Govindan. 
(cherry picked from commit d5e69d89942fd5dcfafd56bf653565557ff85501)
 2018-06-02 09:57:20 +05:30
Rohith Sharma K S
f19e8e2246 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 
(cherry picked from commit 88cbe57c069a1d2dd3bfb32e3ad742566470a10b)
 2018-05-28 14:06:50 +05:30
Robert Kanter
0506c762b2 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 
(cherry picked from commit c8b53c43644b4ad22d5385c22cad8ed573c0b1ba)
 2018-05-10 09:52:11 -07:00
Eric Yang
a984abc472 YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen

(cherry picked from commit 8cdb032aff4237d8d3970057d82290e4e32c4040)
 2018-05-04 12:38:28 -04:00
Jason Lowe
daa739fa13 MAPREDUCE-7086. Add config to allow FileInputFormat to ignore directories when recursive=false. Contributed by Sergey Shelukhin 
(cherry picked from commit 68c6ec719da8e79ada31c8f3a82124f90b9a71fd)
 2018-05-03 12:40:26 -05:00
Bibin A Chundatt
254933cf31
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1a95a4524a8c6c7be601ce8b92640a6a76164a2c)
 2018-05-02 16:15:18 +09:00
Sunil G
099e9fb226 MAPREDUCE-7042. Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit 583a2f4edfdf71c32001cc2f2a03d16f9371d533)
 2018-04-26 19:07:48 +05:30
Shashikant Banerjee
a13b3edfd8
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247da77d03c6015de4f18be75924fcb22)
 2018-04-19 08:36:32 +09:00
Takanobu Asanuma
077eda66ad MAPREDUCE-7066. TestQueue fails on Java9 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-13 10:17:34 -07:00
Erik Krogen
e343be46f8 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of format option. Contributed by Erik Krogen 
(cherry picked from commit 1a407bc9906306801690bc75ff0f0456f8f265fd)
 2018-04-12 23:31:18 -07:00
Wangda Tan
035e0f97ea MAPREDUCE-7077. Pipe mapreduce job fails with Permission denied for jobTokenPassword. (Akira Ajisaka via wangda) 
Change-Id: Ie8f01425d58409fa3661f768205b7616128c8aa4
 2018-04-12 14:33:33 -07:00
Eric E Payne
f955d52f39 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb893cf0e4aa2d3ecc65440668a1fc8d7)
 2018-04-12 16:11:57 +00:00
Rohith Sharma K S
cb6555df24 MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri. 
(cherry picked from commit 6cc59a09e7330dc893b386d84c8f2aa86c02eace)
 2018-04-11 09:47:15 +05:30
Vrushali C
67cd7acf99 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 
(cherry picked from commit 345e7624d58a058a1bad666bd1e5ce4b346a9056)
 2018-04-11 09:41:04 +05:30
Wangda Tan
25bf1d3ac6 Updated branch-3.1 to track 3.1.1 
Change-Id: Idd55c79921b8acacc8a8a3258450e476f3abb706
 2018-04-09 10:58:50 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
0bd99621c5 Preparing for 3.1.0 development 2018-03-19 15:16:54 -07:00
Jason Lowe
2626ec35e9 MAPREDUCE-7064. Flaky test TestTaskAttempt#testReducerCustomResourceTypes. Contributed by Peter Bacsko 2018-03-14 15:05:24 -05:00
Owen O'Malley
3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33554b7d58fd41b022d983bcc2433da7.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Jason Lowe
32fa3a63e0 MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao 2018-03-09 10:41:16 -06:00
Weiwei Yang
6e6945cd78 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-01 10:18:53 +08:00
Billie Rinaldi
7dd385098c MAPREDUCE-7010. Make Job History File Permissions configurable. Contributed by Gergely Novák 2018-02-26 14:32:46 -08:00
Arpit Agarwal
2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Sean Mackrory
2bc3351eaf HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel. 2018-02-21 12:57:14 -07:00
Jason Lowe
0898ff42e9 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 2018-02-16 14:49:00 -06:00
Eric Payne
82f029f7b5 MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 2018-02-16 08:15:09 -06:00
Jason Lowe
a53d62ab26 MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 2018-02-15 15:12:57 -06:00
Jason Lowe
87e2570a14 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 2018-02-12 13:21:09 -06:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Anu Engineer
4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Eric Payne
5a725bb886 MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe 2018-01-31 15:45:30 -06:00
Arun Suresh
38af237969 YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Takanobu Asanuma
56feaa40bb
MAPREDUCE-7036. ASF License warning in hadoop-mapreduce-client 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-30 00:39:39 +09:00
Jason Lowe
6eef3d7f1a MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 2018-01-26 15:31:43 -06:00
Jason Lowe
a37e7f0ad8 MAPREDUCE-7022. Fast fail rogue jobs based on task scratch dir size. Contributed by Johan Gustavsson 2018-01-26 14:36:45 -06:00
Haibo Chen
cc10852252 MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 2018-01-25 16:11:30 -08:00
Jason Lowe
cff9edd4b5 MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 2018-01-24 14:44:07 -06:00
Robert Kanter
836643d793 MAPREDUCE-6995. Uploader tool for Distributed Cache Deploy documentation (miklos.szegedi@cloudera.com via rkanter) 2018-01-19 17:57:54 -08:00
Haibo Chen
cce71dceef MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 2018-01-19 12:56:17 -08:00
Jason Lowe
6e42d05829 MAPREDUCE-7029. FileOutputCommitter is slow on filesystems lacking recursive delete. Contributed by Karthik Palaniappan 2018-01-17 08:14:11 -06:00
Robert Kanter
d716084f45 MAPREDUCE-7032. Add the ability to specify a delayed replication count (miklos.szegedi@cloudera.com via rkanter) 2018-01-16 10:45:45 -08:00
Takanobu Asanuma
1a9c5d479e
MAPREDUCE-7034. Moving logging APIs over to slf4j the rest of all in hadoop-mapreduce 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-01-15 15:38:03 +09:00
Robert Kanter
e404650f48 MAPREDUCE-7030. Uploader tool should ignore symlinks to the same directory (miklos.szegedi@cloudera.com via rkanter) 2018-01-12 14:18:01 -08:00
Jason Lowe
fe35103591 MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas 2018-01-03 11:01:38 -06:00