Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,789 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2696 Commits

Author SHA1 Message Date
Xiao Chen 99b447f612 MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb064)
 2018-10-16 10:24:33 -07:00
Jason Lowe f5ebb2ccf5 MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 8382b860d4)
 2018-09-18 17:15:17 -05:00
Jason Lowe 4a2a7d37dd MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 34b2237e42)
 2018-09-18 17:13:00 -05:00
Eric E Payne 373c7a0ceb MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 
(cherry picked from commit 2cf8927683)
 2018-09-18 20:42:35 +00:00
Jason Lowe f918e4d894 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 2886024ac3)
 2018-09-13 14:47:37 -05:00
Jason Lowe 09e1851b0d MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu 
(cherry picked from commit eb0b5a844f)
 2018-09-06 15:14:38 -05:00
Inigo Goiri 49cfb11d8c MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 0afc036deb)
 2018-06-05 12:25:59 -07:00
Inigo Goiri 9fe265f44e MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit dad1bb868f)
 2018-06-04 12:57:11 -07:00
Inigo Goiri 77d98ae1e6 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit a804b7c9d2)
 2018-06-02 20:26:38 -07:00
Yongjun Zhang f0de11ba98 Preparing for 3.0.4 development 2018-05-29 23:40:26 -07:00
Bibin A Chundatt e6e2146ae0
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1a95a4524a)
 2018-05-02 16:35:18 +09:00
Sunil G 8e0e33b0fd MAPREDUCE-7042. Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit 583a2f4edfdf71c32001cc2f2a03d16f9371d533)
 2018-04-26 19:08:20 +05:30
Shashikant Banerjee 2c4b1c2b15
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e4c39f3247)
 2018-04-19 08:36:59 +09:00
Erik Krogen bce57a139b HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of format option. Contributed by Erik Krogen 
(cherry picked from commit 1a407bc990)
 2018-04-12 23:35:58 -07:00
Lei Xu 3717df89ee Preparing for 3.0.3 development 2018-04-12 13:57:46 -07:00
Eric E Payne 081ea1ec39 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb8)
 2018-04-12 16:22:48 +00:00
Rohith Sharma K S 7cca3486b7 MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri. 
(cherry picked from commit 6cc59a09e7)
 2018-04-11 09:49:38 +05:30
Vrushali C ec258b8ed1 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 
(cherry picked from commit 345e7624d5)
 2018-04-11 09:49:25 +05:30
Jason Lowe 31eef823f0 MAPREDUCE-7064. Flaky test TestTaskAttempt#testReducerCustomResourceTypes. Contributed by Peter Bacsko 
(cherry picked from commit 2626ec35e9)
 2018-03-14 15:21:09 -05:00
Owen O'Malley 0f106df4d6 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-10 13:33:52 -08:00
Jason Lowe 9dacb9fb27 MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao 
(cherry picked from commit 32fa3a63e0)
 2018-03-09 10:44:49 -06:00
Weiwei Yang 5db733fb09 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-02 10:43:03 +08:00
Steve Loughran c960ae0b9a MAPREDUCE-7060. Cherry Pick PathOutputCommitter class/factory to branch-3.0. 
Contributed by Steve Loughran.
 2018-03-01 18:39:54 +00:00
Jason Lowe f173e9cc04 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 14:54:55 -06:00
Eric Payne cf933e17bc MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 82f029f7b5)
 2018-02-16 08:25:52 -06:00
Jason Lowe b9ce463fbd MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26)
 2018-02-15 15:21:05 -06:00
Jason Lowe bc2d4ba507 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 87e2570a14)
 2018-02-12 14:49:35 -06:00
Anu Engineer 2d6ab20007 HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 
(cherry picked from commit 4304fcd5bd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java

(cherry picked from commit 9264f10bb3)
 2018-02-07 11:16:53 -08:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Eric Payne 1d53c9d3a9 MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe 
(cherry picked from commit 5a725bb886)
 2018-01-31 16:01:46 -06:00
Jason Lowe d7acc45ce4 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 
(cherry picked from commit 6eef3d7f1a)
 2018-01-26 15:37:16 -06:00
Haibo Chen e6c66baba9 MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cc10852252)
 2018-01-25 17:14:25 -08:00
Jason Lowe 5281c188df MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 
(cherry picked from commit cff9edd4b5)
 2018-01-24 14:45:46 -06:00
Haibo Chen fc669778be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef)
 2018-01-19 13:05:40 -08:00
Jason Lowe 2a870d91fe MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas 
(cherry picked from commit fe35103591)
 2018-01-03 11:03:16 -06:00
Junping Du 53b9d81f46 Add 2.8.3 release jdiff files. 
(cherry picked from commit c89f99aade)
 2017-12-18 22:40:04 -08:00
Jason Lowe 0dd7f1f4e8 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 
(cherry picked from commit 21d3627355)
 2017-12-01 14:10:02 -06:00
Chris Douglas 03f0b8b3f2 MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 
(cherry picked from commit 782ba3bf9d)
 2017-11-21 20:42:59 -08:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Arun Suresh 72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 5eb7dbe9b3)
 2017-11-09 15:16:55 -08:00
Akira Ajisaka a536655c63
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling. 
(cherry picked from commit e4c220ee4f)
 2017-11-08 19:28:59 +09:00
Akira Ajisaka d77f17458c
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák. 
(cherry picked from commit ffee10b68e)
 2017-11-08 19:23:51 +09:00
Naganarasimha 8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e)
 2017-11-06 12:59:52 +05:30
Xiao Chen 1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
 2017-11-02 21:27:43 -07:00
Akira Ajisaka 12f92e636e
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling. 
(cherry picked from commit 178751ed8c)
 2017-11-02 17:43:55 +09:00
Daniel Templeton 7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce 
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e810838)
 2017-10-31 15:06:18 -07:00
Wangda Tan 4af3deaeb0 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541a)
 2017-10-31 15:06:17 -07:00
Jason Lowe 75605766fd HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60)
 2017-10-31 09:32:02 -05:00
Jason Lowe 00345eed50 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 
(cherry picked from commit 665bb147aa)
 2017-10-27 17:00:31 -05:00
Junping Du ce10c1ef05 Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
 2017-10-26 23:18:50 -07:00