Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,394 Commits 326 Branches 371 Tags
Commit Graph

17394 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
283d48a4f7 HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell. 
(cherry picked from commit 92c15e19272a8bb59f8704c44d4cd713d3a58bcb)
 2018-03-06 09:58:09 -08:00
Steve Loughran
a5cb7f3616 HADOOP-15289. FileStatus.readFields() assertion incorrect. 
Contributed by Steve Loughran.

(cherry picked from commit 12ecb55ffee2b89108eea2565e7ad955ea1fabe7)
 2018-03-06 10:08:28 +00:00
Sen Zhao
4df200bc6d
YARN-7736. Fix itemization in YARN federation document 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 245751ffdc4229715a0c031f57f20748ed16d8a6)
 2018-03-05 11:55:01 -08:00
Xiao Chen
d28db463c2 HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 
(cherry picked from commit c75105f07b4cdbc2773435fc1125446233113c15)
 2018-03-02 16:47:36 -08:00
Gergely Novák
8b9a7b2e48
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e99a1e77671fa0e74a652bc2221bbe01d1a579d3)
 2018-03-02 16:22:06 -08:00
Jason Lowe
7c224656ce YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 
(cherry picked from commit 83798f15f8602ef580a7885876de114b2425da89)
 2018-03-02 10:03:51 -06:00
Weiwei Yang
5db733fb09 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-02 10:43:03 +08:00
Steve Loughran
c960ae0b9a MAPREDUCE-7060. Cherry Pick PathOutputCommitter class/factory to branch-3.0. 
Contributed by Steve Loughran.
 2018-03-01 18:39:54 +00:00
Nanda kumar
7193c95584
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 55d04a6db10d0e58cfe0ff82fa646bb6ba57aaac)
 2018-03-01 19:04:41 +09:00
Xiaoyu Yao
908ca63c03 HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 
(cherry picked from commit f20e10b2dd59f99d9af009bbbb60a067b9893e69)
 2018-02-28 09:48:12 -08:00
Xiaoyu Yao
ce356dcbf6 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 
(cherry picked from commit 90fbe5c6834a6685e65542fa69e396fad2678927)
 2018-02-28 08:44:14 -08:00
Steve Loughran
3c03672e87 HADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake. 
Contributed by Steve Loughran

(cherry picked from commit edc9f1451b4af79069cd4b1d608b0fa68648ee72)
(cherry picked from commit abfe1eaf29a31d96f63022a59e29392b773e6bc1)
 2018-02-28 15:33:21 +00:00
Steve Loughran
0dcd0ead91 HADOOP-15264. AWS "shaded" SDK 1.271 is pulling in netty 4.1.17. 
Contributed by Steve Loughran.

(cherry picked from commit b580196a16dcafe26d6069267a5e2242917aad84)
 2018-02-28 15:33:08 +00:00
Inigo Goiri
0eff245d3d HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:41:42 -08:00
Arpit Agarwal
e22a731070 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:28:03 -08:00
Yiqun Lin
09017aee90 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:52:49 +08:00
Chao Sun
26395aef67 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 
(cherry picked from commit ae290a4bb4e514e2fe9b40d28426a7589afe2a3f)
 2018-02-26 16:18:51 -08:00
Wei Yan
1fb87df87e HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:29:11 -08:00
Kihwal Lee
1087b9af8c HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d8798624ae2a144bc58fa41db826704)
 2018-02-26 10:30:50 -06:00
Xiaoyu Yao
21d4b5fd24 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 
(cherry picked from commit 1e84e46f1621fe694f806bfc41d3b2a06c9500b6)
 2018-02-23 19:41:28 -08:00
Xiao Chen
1fbd419957 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d323359587dca7831f74c9d065c2fccc60d)
 2018-02-23 14:00:25 -08:00
Kihwal Lee
107efcdc5d HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 
(cherry picked from commit 1f0952bfc734f82339d555e764e454d5594bb84f)
 2018-02-23 13:14:44 -06:00
Xiaoyu Yao
ec1b445c62 HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 
(cherry picked from commit e978e1d9064fc794ea83217d0af2315d4167211e)
 2018-02-22 20:18:02 -08:00
Steve Loughran
c2bbe22c5a HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-21 11:11:28 +00:00
Steve Loughran
e011650090 HADOOP-15247. Move commons-net up to 3.6. 
Contributed by Steve Loughran.

(cherry picked from commit 004b722372de67635a24e71b264b3b604df4b693)
 2018-02-21 10:42:07 +00:00
Sunil G
09bb378c10 YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 
(cherry picked from commit bdd2a184d78379d99c802a43ebec7d2cef0bbaf7)
 2018-02-21 15:06:28 +05:30
Anu Engineer
c0d8103ba2 HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.

(cherry picked from commit 121e1e1280c7b019f6d2cc3ba9eae1ead0dd8408)
 2018-02-20 19:59:31 -08:00
Yiqun Lin
7601489995 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 
(cherry picked from commit 8896d20b91520053a6bbfb680adb345cd24f4142)
 2018-02-20 13:06:26 -08:00
Jason Lowe
85c611ad7d YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:38:49 -06:00
Rohith Sharma K S
313c38b502 YARN-7937. Fix http method name in Cluster Application Timeout Update API example request. Contributed by Charan Hebri. 
(cherry picked from commit 87bdde69431c19a22d79a767071f6ea47e1ceb3d)
 2018-02-18 14:06:29 +05:30
Jason Lowe
f173e9cc04 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9e5c53f2fce7ccdeb4e1cd7d0f123b3)
 2018-02-16 14:54:55 -06:00
Eric Payne
cf933e17bc MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 82f029f7b50679ea477a3a898e4ee400fa394adf)
 2018-02-16 08:25:52 -06:00
Kihwal Lee
7b83fca453 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:35:27 -06:00
Jason Lowe
b9ce463fbd MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26e170a0338f93e228718da52e9196e4)
 2018-02-15 15:21:05 -06:00
Kihwal Lee
11ff6327af xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:14:42 -06:00
Steve Loughran
b302f728f8 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b79324b7f6231b879c0bfc608203b980)
 2018-02-15 16:29:30 +00:00
Steve Loughran
1888b94806 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a31307f19551f87be4e81d4fb626de1)
 2018-02-15 14:26:18 +00:00
Anu Engineer
4b4157441f HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 
(cherry picked from commit 8f66affd6265c9e4231e18d7ca352fb3035dae9a)
 2018-02-14 13:30:33 -08:00
Eric Payne
55ded70716 Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit 36c9dda07f9d0ac14f0c979c7d519536fc9b3d28.
 2018-02-14 14:38:31 -06:00
Xiao Chen
2f3415a4d6 Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit 16b2cad8e8f5215325c51eb582f58640a386b06b.
 2018-02-14 10:25:15 -08:00
Eric Payne
36c9dda07f YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 2018-02-14 08:58:28 -06:00
Aaron Fabbri
e6f6c0bfc2
HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE bug spams logs w/ Yarn Log Aggregation 2018-02-13 20:20:46 -08:00
Arpit Agarwal
16b2cad8e8 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:29 -08:00
Jason Lowe
bc2d4ba507 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 87e2570a1419d3616de2de3b553108ad1a8af425)
 2018-02-12 14:49:35 -06:00
Brahma Reddy Battula
055d5e0b85 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
 2018-02-12 22:17:35 +05:30
Arpit Agarwal
41cf2b8e7c HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:16:48 -08:00
Yiqun Lin
38febc0ae2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 13:59:38 +08:00
Xiaoyu Yao
49ab09f00b HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d435039f69ea35f592856ca04d378809)
 2018-02-08 10:18:35 -08:00
Manoj Govindassamy
bef769e436 HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 
(cherry picked from commit 73ff09b79a5cf9932edc21c58f3a730f7379086b)

(cherry picked from commit 821729905ed8439ee5f15372faaf6b40aff7f155)
 2018-02-07 19:49:27 -08:00
Yongjun Zhang
5105b60614 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java

(cherry picked from commit 60d82ac61f2afa2a61c8eeabf20e45b5a89311d0)
 2018-02-07 16:49:26 -08:00