Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,891 Commits 326 Branches 371 Tags
Commit Graph

17891 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Steve Loughran
958b445047 HADOOP-15247. Move commons-net up to 3.6. 
Contributed by Steve Loughran.

(cherry picked from commit 004b722372de67635a24e71b264b3b604df4b693)
 2018-02-21 10:41:20 +00:00
Sunil G
e53da3914a YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 
(cherry picked from commit bdd2a184d78379d99c802a43ebec7d2cef0bbaf7)
 2018-02-21 14:57:43 +05:30
Sunil G
21af27995d YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan. 
(cherry picked from commit 86b227a1fbe26b992c5498cfdd3b1691b4362ee9)
 2018-02-21 14:17:18 +05:30
Anu Engineer
4aa4bb46af HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.

(cherry picked from commit 121e1e1280c7b019f6d2cc3ba9eae1ead0dd8408)
 2018-02-20 19:58:58 -08:00
Eric Yang
7506e100fc YARN-7940. Fixed a bug in ServiceAM ZooKeeper initialization. 
(Contributed by Billie Rinaldi)

(cherry-picked from commit 7280c5af82d36a9be15448293210d871f680f55e)
 2018-02-20 14:16:46 -05:00
Yiqun Lin
538c473f49 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 
(cherry picked from commit 8896d20b91520053a6bbfb680adb345cd24f4142)
 2018-02-20 10:14:09 +08:00
Jason Lowe
94def88889 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 
(cherry picked from commit 949721508467968d5f46170353716ad04349cd6f)
 2018-02-19 14:22:52 -06:00
Jason Lowe
1481adbabe Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 6f64530fc35bbc954a84d64f349379dc15fe0898.
 2018-02-19 08:23:29 -06:00
fang zhenyi
55669df4f9
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d4dde5112e9ee6b37cbdea17104c5a4c6870bd5)
 2018-02-18 22:22:06 +09:00
Rohith Sharma K S
7ee3713f73 YARN-7937. Fix http method name in Cluster Application Timeout Update API example request. Contributed by Charan Hebri. 
(cherry picked from commit 87bdde69431c19a22d79a767071f6ea47e1ceb3d)
 2018-02-18 14:05:59 +05:30
Arun Suresh
950566230c YARN-7918. Fix TestAMRMClientPlacementConstraints. (Gergely Novák via asuresh) 
(cherry picked from commit a1e56a62863d8d494af309ec5f476c4b7e4d5ef9)
 2018-02-17 03:26:13 -08:00
Wangda Tan
f878ddb067 YARN-7328. ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores. (lovekesh bansal via wangda) 
Change-Id: Ibb1faf0beefec079dae2208986976b2f64650672
(cherry picked from commit ca1043ab9030339d7cdd3275c3f8f4713b8bff59)
 2018-02-17 13:03:58 +08:00
Wangda Tan
ac9e5ee568 HADOOP-14875. Create end user documentation from the compatibility guidelines. (Daniel Templeton via wangda) 
Change-Id: I5d66d0f034874623af22aecae0a0b858b65396ed
(cherry picked from commit 31db977cb33ac7dde04d6001b9ad69012df4e3a0)
 2018-02-17 13:03:58 +08:00
Jason Lowe
03aba69835 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9e5c53f2fce7ccdeb4e1cd7d0f123b3)
 2018-02-16 14:50:14 -06:00
Arpit Agarwal
c880a87101 HDFS-13058. Fix dfs.namenode.shared.edits.dir in TestJournalNode. Contributed by Bharat Viswanadham. 2018-02-16 10:29:29 -08:00
Arpit Agarwal
f43f32bd31 HDFS-13153. Enable HDFS diskbalancer by default. Contributed by Ajay Kumar. 2018-02-16 09:38:51 -08:00
Steve Loughran
27a9640036 HADOOP-14507. Extend per-bucket secret key config with explicit getPassword() on fs.s3a.$bucket.secret.key. 
Contributed by Steve Loughran.

(cherry picked from commit 7ac88244c54ce483729af3d2736d9f4731e230ca)
 2018-02-16 16:39:26 +00:00
Eric Payne
02836d3282 MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 82f029f7b50679ea477a3a898e4ee400fa394adf)
 2018-02-16 08:19:03 -06:00
Sunil G
d6eb001da9 YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan. 
(cherry picked from commit aae629913cee0157c945a2c7384c7bf398f10616)
 2018-02-16 13:56:44 +05:30
Bharat Viswanadham
1fb2f020e2
HDFS-13151. Fix the javadoc error in ReplicaInfo 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a1e05e02927f29e5598fdc665ac997667e6b00b1)
 2018-02-16 16:23:29 +09:00
Jason Lowe
6f64530fc3 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-02-15 17:14:39 -06:00
Konstantinos Karanasos
41708402a1 YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan. 
(cherry picked from commit 0b489e564ce5a50324a530e29c18aa8a75276c50)
 2018-02-15 14:25:34 -08:00
Kihwal Lee
4cdc57f6a4 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:34:46 -06:00
Jason Lowe
50de2676a1 MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26e170a0338f93e228718da52e9196e4)
 2018-02-15 15:16:39 -06:00
Kihwal Lee
8b9376a089 xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:13:56 -06:00
Steve Loughran
050f5287b7 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.
 2018-02-15 16:25:55 +00:00
Steve Loughran
96c047fbb9 HADOOP-15176. Enhance IAM Assumed Role support in S3A client. 
Contributed by Steve Loughran
 2018-02-15 15:56:10 +00:00
Steve Loughran
c761e658f6 HADOOP-15076. Enhance S3A troubleshooting documents and add a performance document. 
Contributed by Steve Loughran.
 2018-02-15 14:56:32 +00:00
Steve Loughran
58a2120e8a HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.
 2018-02-15 14:24:36 +00:00
Aaron Fabbri
bfdf99eb35
HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE bug spams logs w/ Yarn Log Aggregation 2018-02-14 18:21:32 -08:00
Anu Engineer
26f16f6a64 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 
(cherry picked from commit 8f66affd6265c9e4231e18d7ca352fb3035dae9a)
 2018-02-14 13:29:56 -08:00
Eric Payne
c81415dd2a Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit 1fcc3eac885af23d79b170b5b80f271d7877641a.
 2018-02-14 14:34:07 -06:00
Xiao Chen
562274a20b Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit e88265ea56c7ca3abfbe9c78819c9df9f789af39.
 2018-02-14 10:26:54 -08:00
Steve Loughran
401a0ee38d HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.

(cherry picked from commit f20dc0d5770d3876954faf0a6e8dcce6539ffc23)
 2018-02-14 16:21:22 +00:00
Wangda Tan
4eb98f70ee YARN-7789. Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator. (Zian Chen via wangda) 
Change-Id: I1f6a451fe16758def6f47c046a9b8a67ad7255af
(cherry picked from commit 042ef2fa7bcc22e3ca4eb8205c34d83e594bc7de)
 2018-02-14 23:21:22 +08:00
Arpit Agarwal
e88265ea56 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:38 -08:00
Eric Payne
1fcc3eac88 YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 
(cherry picked from commit c5e6e3de1c31eda052f89eddd7bba288625936b9)
 2018-02-13 10:20:35 -06:00
Chris Douglas
08053c4ead HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 
(cherry picked from commit 0c5d7d71a80bccd4ad7eab269d0727b999606a7e)
 2018-02-12 21:07:25 -08:00
Chris Douglas
1f6b1894fc Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine" 
This reverts commit d51e4affbacad720793a42794aa16343a4e36a66.
 2018-02-12 21:06:21 -08:00
Chris Douglas
d51e4affba HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 
(cherry picked from commit 5b88cb339898f82519223bcd07e1caedff02d051)
 2018-02-12 21:03:00 -08:00
Jason Lowe
5f7ef73b7a YARN-7914. Fix exit code handling for short lived Docker containers. Contributed by Shane Kumpf 
(cherry picked from commit 5a1db60ab1e8b28cd73367c69970513de88cf4dd)
 2018-02-12 15:51:32 -06:00
Jason Lowe
600e919469 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 87e2570a1419d3616de2de3b553108ad1a8af425)
 2018-02-12 13:26:33 -06:00
Brahma Reddy Battula
fb088564a2 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
 2018-02-12 22:16:41 +05:30
Jason Lowe
aad3958e50 YARN-7917. Fix failing test TestDockerContainerRuntime#testLaunchContainerWithDockerTokens. Contributed by Shane Kumpf 
(cherry picked from commit 3414fd1243c0b0dcbb9cea986af5f096300eba97)
 2018-02-12 09:30:06 -06:00
Arpit Agarwal
e82abfd64b HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:17:50 -08:00
Steve Loughran
bd5ab59125 HADOOP-15187. Remove ADL mock test dependency on REST call invoked from Java SDK. 
Contributed by Vishwajeet Dusane.
 2018-02-12 15:11:17 +00:00
Wangda Tan
101cf2599e YARN-7739. DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation. (wangda) 
Change-Id: I10cc9341237d9a2fc0f8c855efb98a36b91389e2
(cherry picked from commit d02e42cee4a08a47ed2835f7a4a100daaa95833f)
 2018-02-12 10:39:42 +08:00
Wangda Tan
3b20413acf YARN-7697. NM goes down with OOM due to leak in log-aggregation. (Xuan Gong via wangda) 
Change-Id: Ie4fc7979d834e25f37a033c314f3efceeb8f4a9e
(cherry picked from commit d4c98579e36df7eeb788352d7b76cd2c7448c511)
 2018-02-12 10:39:34 +08:00
Wangda Tan
1e6d2972b8 YARN-5848. Remove unnecessary public/crossdomain.xml from YARN UIv2 sub project. (Sunil G via wangda) 
Change-Id: Ie295f88232192e6b520c335b0332383cc6a232c0
(cherry picked from commit 789a185c16351d2343e075413a50eb3e5849cc5f)
 2018-02-12 10:39:27 +08:00
Wangda Tan
47a2e18670 YARN-7906. Fix mvn site fails with error: Multiple sources of package comments found for package o.a.h.y.client.api.impl. (Akira Ajisaka via wangda) 
Change-Id: I20221d97446e97f208d587eacbc60448c11ffd48
(cherry picked from commit e795833d8c1981cab85a10b4e516cd0c5423c792)
 2018-02-12 10:39:21 +08:00