Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,453 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1850 Commits

Author SHA1 Message Date
Eric E Payne 081ea1ec39 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 
(cherry picked from commit 6bb128dfb8)
 2018-04-12 16:22:48 +00:00
Rohith Sharma K S 7cca3486b7 MAPREDUCE-7062. Update mapreduce.job.tags description for making use for ATSv2 purpose. Contributed by Charan Hebri. 
(cherry picked from commit 6cc59a09e7)
 2018-04-11 09:49:38 +05:30
Vrushali C ec258b8ed1 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 
(cherry picked from commit 345e7624d5)
 2018-04-11 09:49:25 +05:30
Jason Lowe 31eef823f0 MAPREDUCE-7064. Flaky test TestTaskAttempt#testReducerCustomResourceTypes. Contributed by Peter Bacsko 
(cherry picked from commit 2626ec35e9)
 2018-03-14 15:21:09 -05:00
Owen O'Malley 0f106df4d6 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-10 13:33:52 -08:00
Jason Lowe 9dacb9fb27 MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao 
(cherry picked from commit 32fa3a63e0)
 2018-03-09 10:44:49 -06:00
Weiwei Yang 5db733fb09 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-02 10:43:03 +08:00
Steve Loughran c960ae0b9a MAPREDUCE-7060. Cherry Pick PathOutputCommitter class/factory to branch-3.0. 
Contributed by Steve Loughran.
 2018-03-01 18:39:54 +00:00
Jason Lowe f173e9cc04 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 14:54:55 -06:00
Eric Payne cf933e17bc MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 82f029f7b5)
 2018-02-16 08:25:52 -06:00
Jason Lowe b9ce463fbd MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26)
 2018-02-15 15:21:05 -06:00
Jason Lowe bc2d4ba507 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 87e2570a14)
 2018-02-12 14:49:35 -06:00
Anu Engineer 2d6ab20007 HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 
(cherry picked from commit 4304fcd5bd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java

(cherry picked from commit 9264f10bb3)
 2018-02-07 11:16:53 -08:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Eric Payne 1d53c9d3a9 MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe 
(cherry picked from commit 5a725bb886)
 2018-01-31 16:01:46 -06:00
Jason Lowe d7acc45ce4 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 
(cherry picked from commit 6eef3d7f1a)
 2018-01-26 15:37:16 -06:00
Haibo Chen e6c66baba9 MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cc10852252)
 2018-01-25 17:14:25 -08:00
Jason Lowe 5281c188df MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 
(cherry picked from commit cff9edd4b5)
 2018-01-24 14:45:46 -06:00
Haibo Chen fc669778be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef)
 2018-01-19 13:05:40 -08:00
Jason Lowe 2a870d91fe MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas 
(cherry picked from commit fe35103591)
 2018-01-03 11:03:16 -06:00
Jason Lowe 0dd7f1f4e8 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 
(cherry picked from commit 21d3627355)
 2017-12-01 14:10:02 -06:00
Chris Douglas 03f0b8b3f2 MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 
(cherry picked from commit 782ba3bf9d)
 2017-11-21 20:42:59 -08:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Arun Suresh 72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 5eb7dbe9b3)
 2017-11-09 15:16:55 -08:00
Akira Ajisaka a536655c63
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling. 
(cherry picked from commit e4c220ee4f)
 2017-11-08 19:28:59 +09:00
Akira Ajisaka d77f17458c
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák. 
(cherry picked from commit ffee10b68e)
 2017-11-08 19:23:51 +09:00
Naganarasimha 8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e)
 2017-11-06 12:59:52 +05:30
Xiao Chen 1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
 2017-11-02 21:27:43 -07:00
Akira Ajisaka 12f92e636e
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling. 
(cherry picked from commit 178751ed8c)
 2017-11-02 17:43:55 +09:00
Daniel Templeton 7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce 
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e810838)
 2017-10-31 15:06:18 -07:00
Wangda Tan 4af3deaeb0 YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
(cherry picked from commit df3855541a)
 2017-10-31 15:06:17 -07:00
Jason Lowe 75605766fd HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60)
 2017-10-31 09:32:02 -05:00
Jason Lowe 00345eed50 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 
(cherry picked from commit 665bb147aa)
 2017-10-27 17:00:31 -05:00
Akira Ajisaka a887016e1d
MAPREDUCE-6972. Enable try-with-resources for RecordReader. Contributed by Zoltan Haindrich. 
(cherry picked from commit 2523e1cce5)
 2017-10-18 11:19:00 +09:00
Chris Trezzo 24b03eb79f MAPREDUCE-5951. Add support for the YARN Shared Cache. 
(cherry picked from commit e46d5bb962)
 2017-10-12 11:15:46 -07:00
Akira Ajisaka 47aa044fcc
MAPREDUCE-6971. Moving logging APIs over to slf4j in hadoop-mapreduce-client-app. Contributed by Jinjiang Ling. 
(cherry picked from commit 453d48bdfb)
 2017-10-03 12:15:43 +09:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Jason Lowe bf03363d83 MAPREDUCE-6968. Staging directory erasure coding config property has a typo. Contributed by Jason Lowe 
(cherry picked from commit 9df05005ac)
 2017-09-26 14:53:47 -05:00
Eric Payne 839d5b124d MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408)
 2017-09-19 10:40:41 -05:00
Jason Lowe 731a622627 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 
(cherry picked from commit 3a20debdde)
 2017-09-19 09:14:12 -05:00
Jason Lowe db88b6179d Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe" 
This reverts commit b3d61304f2.

(cherry picked from commit ea845ba58c)
 2017-09-19 08:50:20 -05:00
Andrew Wang 531290c2c8 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory. 
(cherry picked from commit 3cf3540f19)
 2017-09-18 15:13:45 -07:00
Jason Lowe 8894654fde MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 
(cherry picked from commit b3d61304f2)
 2017-09-18 17:06:02 -05:00
Robert Kanter 917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter) 
(cherry picked from commit 0adc0471d0)
 2017-09-18 10:42:01 -07:00
Robert Kanter 3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message 
This reverts commit 293ab432e1.
 2017-09-18 10:41:46 -07:00
Robert Kanter 293ab432e1 YARN-7162. Remove XML excludes file format (rkanter) 
(cherry picked from commit 3a8d57a0a2)
 2017-09-15 12:01:41 -07:00
Steve Loughran b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter. 
Contributed by Steve Loughran

(cherry picked from commit 11390c2d11)
 2017-09-15 17:00:16 +01:00
Andrew Wang ad0fe570d9 HADOOP-14089. Automated checking for malformed client. Contributed by Sean Busbey. 
(cherry picked from commit c3f35c422b)
 2017-09-13 16:57:53 -07:00
Jason Lowe 12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)
 2017-09-13 17:22:29 -05:00
Ray Chiang 656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179)
 2017-09-12 10:19:58 -07:00