Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,929 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

17929 Commits

Author SHA1 Message Date
Wangda Tan ac9e5ee568 HADOOP-14875. Create end user documentation from the compatibility guidelines. (Daniel Templeton via wangda) 
Change-Id: I5d66d0f034874623af22aecae0a0b858b65396ed
(cherry picked from commit 31db977cb3)
 2018-02-17 13:03:58 +08:00
Jason Lowe 03aba69835 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 14:50:14 -06:00
Arpit Agarwal c880a87101 HDFS-13058. Fix dfs.namenode.shared.edits.dir in TestJournalNode. Contributed by Bharat Viswanadham. 2018-02-16 10:29:29 -08:00
Arpit Agarwal f43f32bd31 HDFS-13153. Enable HDFS diskbalancer by default. Contributed by Ajay Kumar. 2018-02-16 09:38:51 -08:00
Steve Loughran 27a9640036 HADOOP-14507. Extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key. 
Contributed by Steve Loughran.

(cherry picked from commit 7ac88244c5)
 2018-02-16 16:39:26 +00:00
Eric Payne 02836d3282 MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 82f029f7b5)
 2018-02-16 08:19:03 -06:00
Sunil G d6eb001da9 YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan. 
(cherry picked from commit aae629913c)
 2018-02-16 13:56:44 +05:30
Bharat Viswanadham 1fb2f020e2
HDFS-13151. Fix the javadoc error in ReplicaInfo 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a1e05e0292)
 2018-02-16 16:23:29 +09:00
Jason Lowe 6f64530fc3 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-02-15 17:14:39 -06:00
Konstantinos Karanasos 41708402a1 YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan. 
(cherry picked from commit 0b489e564c)
 2018-02-15 14:25:34 -08:00
Kihwal Lee 4cdc57f6a4 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e5)
 2018-02-15 15:34:46 -06:00
Jason Lowe 50de2676a1 MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26)
 2018-02-15 15:16:39 -06:00
Kihwal Lee 8b9376a089 xattr api cleanup 
(cherry picked from commit da59acd8ca)
 2018-02-15 11:13:56 -06:00
Steve Loughran 050f5287b7 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.
 2018-02-15 16:25:55 +00:00
Steve Loughran 96c047fbb9 HADOOP-15176. Enhance IAM Assumed Role support in S3A client. 
Contributed by Steve Loughran
 2018-02-15 15:56:10 +00:00
Steve Loughran c761e658f6 HADOOP-15076. Enhance S3A troubleshooting documents and add a performance document. 
Contributed by Steve Loughran.
 2018-02-15 14:56:32 +00:00
Steve Loughran 58a2120e8a HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.
 2018-02-15 14:24:36 +00:00
Aaron Fabbri bfdf99eb35
HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE bug spams logs w/ Yarn Log Aggregation 2018-02-14 18:21:32 -08:00
Anu Engineer 26f16f6a64 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 
(cherry picked from commit 8f66affd62)
 2018-02-14 13:29:56 -08:00
Eric Payne c81415dd2a Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit 1fcc3eac88.
 2018-02-14 14:34:07 -06:00
Xiao Chen 562274a20b Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit e88265ea56.
 2018-02-14 10:26:54 -08:00
Steve Loughran 401a0ee38d HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.

(cherry picked from commit f20dc0d577)
 2018-02-14 16:21:22 +00:00
Wangda Tan 4eb98f70ee YARN-7789. Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator. (Zian Chen via wangda) 
Change-Id: I1f6a451fe16758def6f47c046a9b8a67ad7255af
(cherry picked from commit 042ef2fa7b)
 2018-02-14 23:21:22 +08:00
Arpit Agarwal e88265ea56 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:38 -08:00
Eric Payne 1fcc3eac88 YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 
(cherry picked from commit c5e6e3de1c)
 2018-02-13 10:20:35 -06:00
Chris Douglas 08053c4ead HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 
(cherry picked from commit 0c5d7d71a8)
 2018-02-12 21:07:25 -08:00
Chris Douglas 1f6b1894fc Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine" 
This reverts commit d51e4affba.
 2018-02-12 21:06:21 -08:00
Chris Douglas d51e4affba HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 
(cherry picked from commit 5b88cb3398)
 2018-02-12 21:03:00 -08:00
Jason Lowe 5f7ef73b7a YARN-7914. Fix exit code handling for short lived Docker containers. Contributed by Shane Kumpf 
(cherry picked from commit 5a1db60ab1)
 2018-02-12 15:51:32 -06:00
Jason Lowe 600e919469 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 87e2570a14)
 2018-02-12 13:26:33 -06:00
Brahma Reddy Battula fb088564a2 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351ca)
 2018-02-12 22:16:41 +05:30
Jason Lowe aad3958e50 YARN-7917. Fix failing test TestDockerContainerRuntime#testLaunchContainerWithDockerTokens. Contributed by Shane Kumpf 
(cherry picked from commit 3414fd1243)
 2018-02-12 09:30:06 -06:00
Arpit Agarwal e82abfd64b HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:17:50 -08:00
Steve Loughran bd5ab59125 HADOOP-15187. Remove ADL mock test dependency on REST call invoked from Java SDK. 
Contributed by Vishwajeet Dusane.
 2018-02-12 15:11:17 +00:00
Wangda Tan 101cf2599e YARN-7739. DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation. (wangda) 
Change-Id: I10cc9341237d9a2fc0f8c855efb98a36b91389e2
(cherry picked from commit d02e42cee4)
 2018-02-12 10:39:42 +08:00
Wangda Tan 3b20413acf YARN-7697. NM goes down with OOM due to leak in log-aggregation. (Xuan Gong via wangda) 
Change-Id: Ie4fc7979d834e25f37a033c314f3efceeb8f4a9e
(cherry picked from commit d4c98579e3)
 2018-02-12 10:39:34 +08:00
Wangda Tan 1e6d2972b8 YARN-5848. Remove unnecessary public/crossdomain.xml from YARN UIv2 sub project. (Sunil G via wangda) 
Change-Id: Ie295f88232192e6b520c335b0332383cc6a232c0
(cherry picked from commit 789a185c16)
 2018-02-12 10:39:27 +08:00
Wangda Tan 47a2e18670 YARN-7906. Fix mvn site fails with error: Multiple sources of package comments found for package o.a.h.y.client.api.impl. (Akira Ajisaka via wangda) 
Change-Id: I20221d97446e97f208d587eacbc60448c11ffd48
(cherry picked from commit e795833d8c)
 2018-02-12 10:39:21 +08:00
Weiwei Yang 388f4d11d7 YARN-7838. Support AND/OR constraints in Distributed Shell. Contributed by Weiwei Yang. 2018-02-11 14:21:55 +08:00
Eric Yang c97d5bceb2 YARN-7909. Add charset to YARN Service REST API. (Contributed by Eric Yang) 2018-02-09 20:01:25 -05:00
Yiqun Lin 543f3abbee HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 2018-02-09 13:57:42 +08:00
Jian He ddec08d7cc YARN-7827. Stop and Delete Yarn Service from RM UI fails with HTTP ERROR 404. Contributed by Sunil G 2018-02-08 21:32:40 -08:00
Yufei Gu 1bc03ddf97 YARN-7655. Avoid AM preemption caused by RRs for specific nodes or racks. Contributed by Steven Rand. 2018-02-08 12:32:43 -08:00
Jian He eb2449d539 YARN-5428. Allow for specifying the docker client configuration directory. Contributed by Shane Kumpf 2018-02-08 11:35:30 -08:00
Steve Loughran 996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0. 
Contributed by Igor Dvorzhak
 2018-02-08 10:55:54 -08:00
Xiaoyu Yao 8faf0b50d4 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 2018-02-08 08:59:48 -08:00
Yongjun Zhang f491f717e9 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 2018-02-07 12:58:09 -08:00
Jitendra Pandey b061215ecf HDFS-11701. NPE from Unresolved Host causes permanent DFSInputStream failures. Contributed by Lokesh Jain. 2018-02-07 11:22:36 -08:00
Jason Lowe 456705a07c YARN-7815. Make the YARN mounts added to Docker containers more restrictive. Contributed by Shane Kumpf 2018-02-07 13:09:08 -06:00
Brahma Reddy Battula 01bd6ab18f HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-07 23:10:33 +05:30