Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,389 Commits 326 Branches 371 Tags
Commit Graph

6760 Commits

Author SHA1 Message Date
Inigo Goiri
c6b1125a24 HDFS-12886. Ignore minReplication for block recovery. Contributed by Lukas Majercak. 
(cherry picked from commit 08ff1586d5d3e39f546200f9e696f62ea4cf000d)
 2018-03-16 10:30:34 -07:00
Inigo Goiri
ba9a656ffa HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 
(cherry picked from commit e71bc00a471422ddb26dd54e706f09f0fe09925c)
 2018-03-15 10:35:20 -07:00
Inigo Goiri
6b8a60b62a HDFS-12723. TestReadStripedFileWithMissingBlocks#testReadFileWithMissingBlocks failing consistently. Contributed by Ajay Kumar. 
(cherry picked from commit 6de135169eaaba9a4707d2bef380793ef91478d7)
 2018-03-15 10:15:46 -07:00
Inigo Goiri
cfa7cc2aa4 HDFS-13198. RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router. Contributed by Wei Yan. 
(cherry picked from commit 41fc7f80beb32f4a1fac73faf7c9116ad5ee3420)
 2018-03-14 13:15:51 -07:00
Sammi Chen
e97234f274 HDFS-11600. Refactor TestDFSStripedOutputStreamWithFailure test classes. Contributed by Sammi Chen. 
(cherry picked from commit ad1b988a828608b12cafb6382436cd17f95bfcc5)
 2018-03-14 14:51:51 +08:00
Yiqun Lin
a105069729 HDFS-12773. RBF: Improve State Store FS implementation. Contributed by Inigo Goiri. 
(cherry picked from commit 76be6cbf6c33f866794f27ca2560ca7c7b2fa0e7)
 2018-03-14 11:31:35 +08:00
Xiao Chen
fc6f3e2a6f HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 
(cherry picked from commit 8211a3d4693fea46cff11c5883c16a9b4df7b4de)
 2018-03-13 11:18:22 -07:00
Xiao Chen
f23ef858ff Revert "HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma." 
This reverts commit 752e116f413214cfe8d9feae6caef4c5b21db2da.
 2018-03-13 10:39:50 -07:00
Xiao Chen
c0aa249a1d HDFS-12587. Use Parameterized tests in TestBlockInfoStriped and TestLowRedundancyBlockQueues to test all EC policies. Contributed by Takanobu Asanuma. 
(cherry picked from commit 3b8dbc2cb766ba9fc1d655c891d32f5b4e4aa9c8)
 2018-03-13 10:14:39 -07:00
Xiao Chen
8f4bfb3ed5 HDFS-13239. Fix non-empty dir warning message when setting default EC policy. Contributed by Bharat Viswanadham. 
(cherry picked from commit d6931c30c5a643ca192109b05a9c44da42a6318c)
 2018-03-13 10:08:09 -07:00
Xiao Chen
752e116f41 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 
(cherry picked from commit 84c10955863eca1e300aeeac1d9cd7a1186144b6)
 2018-03-13 09:58:50 -07:00
Xiaoyu Yao
2a93ba690c HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 
(cherry picked from commit 107c177782a24a16c66113841f2fc5144f56207b)
 2018-03-12 20:23:52 -07:00
Yiqun Lin
840c83bf7a HDFS-13226. RBF: Throw the exception if mount table entry validated failed. Contributed by maobaolong. 2018-03-13 11:13:15 +08:00
Chris Douglas
a03c8ea61f HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc04a86be91a9b72073cdc30bec04d3b)
(cherry picked from commit 09940b1eb3b7ed764149f4a993c1857e9c6ad938)
 2018-03-12 20:06:22 -07:00
Yiqun Lin
430cdfefc0 HDFS-13253. RBF: Quota management incorrect parent-child relationship judgement. Contributed by Yiqun Lin. 2018-03-13 10:41:44 +08:00
Akira Ajisaka
89105a623b
HDFS-12156. TestFSImage fails without -Pnative 
(cherry picked from commit 319defafc105c0d0b69b83828b578d9c453036f5)
 2018-03-13 11:27:46 +09:00
Inigo Goiri
177dd6580d HDFS-13241. RBF: TestRouterSafemode failed if the port 8888 is in use. Contributed by maobaolong. 
(cherry picked from commit 91c82c90f05ea75fe59c6ffad3dc3fcac1429e9e)
 2018-03-12 17:30:00 -07:00
Chris Douglas
2210593dad HADOOP-14742. Document multi-URI replication Inode for ViewFS. Contributed by Gera Shegalov 
(cherry picked from commit ddb67ca707de896cd0ba5cda3c0d1a2d9edca968)
(cherry picked from commit c9364b3bce30946a6c0154974d0adc6accb3bea3)
 2018-03-12 13:55:10 -07:00
Inigo Goiri
29cf29a853 HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 
(cherry picked from commit ac627f561f0946e98a650850fb507536cbd2f2c4)
 2018-03-12 10:05:43 -07:00
Yiqun Lin
2e607f2103 HDFS-13240. RBF: Update some inaccurate document descriptions. Contributed by Yiqun Lin. 
(cherry picked from commit 4743d4a2c70a213a41804a24c776e6db00e1b90d)
 2018-03-10 11:32:12 +08:00
Inigo Goiri
465c3f0e24 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 
(cherry picked from commit 8133cd5305d7913453abb2d48da12f672c0ce334)
 2018-03-09 18:27:17 -08:00
Inigo Goiri
c453fcb23f HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 
(cherry picked from commit afe1a3ccd56a12fec900360a8a2855c080728e65)
 2018-03-09 17:20:29 -08:00
Hanisha Koneru
bd2ebf031c HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham. 
(cherry picked from commit 4eeff62f6925991bca725b1ede5308055817de80)
 2018-03-09 15:49:09 -08:00
Lokesh Jain
80aabd72c2
HDFS-13190. Document WebHDFS support for snapshot diff 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 7b0dc310208ee5bc191c9accb3d1312513145653)
 2018-03-09 15:07:34 -08:00
Hanisha Koneru
dec53bb9e1 HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 
(cherry picked from commit 9a082fbe6e302df7139b65a23be9a39acd87715d)
 2018-03-09 13:39:49 -08:00
Yiqun Lin
f879504fe1 HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:48:36 +08:00
Inigo Goiri
68cff8c235 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 
(cherry picked from commit 0c2b969e0161a068bf9ae013c4b95508dfb90a8a)
 2018-03-08 09:33:24 -08:00
Hanisha Koneru
0b1e9665df HDFS-13225. StripeReader#checkMissingBlocks() 's IOException info is incomplete. Contributed by lufei. 
(cherry picked from commit 19ae4429f9a2807548bb4b9ff296521fbdbbb2bc)
 2018-03-07 14:22:01 -08:00
Yiqun Lin
253ef33f80 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 16:07:05 +08:00
Xiaoyu Yao
b74df5523c HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 
(cherry picked from commit edf9445708ffb7a9e59cb933e049b540f99add1e)
 2018-03-06 18:30:10 -08:00
Xiao Chen
283d48a4f7 HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell. 
(cherry picked from commit 92c15e19272a8bb59f8704c44d4cd713d3a58bcb)
 2018-03-06 09:58:09 -08:00
Xiao Chen
d28db463c2 HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 
(cherry picked from commit c75105f07b4cdbc2773435fc1125446233113c15)
 2018-03-02 16:47:36 -08:00
Xiaoyu Yao
908ca63c03 HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 
(cherry picked from commit f20e10b2dd59f99d9af009bbbb60a067b9893e69)
 2018-02-28 09:48:12 -08:00
Xiaoyu Yao
ce356dcbf6 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 
(cherry picked from commit 90fbe5c6834a6685e65542fa69e396fad2678927)
 2018-02-28 08:44:14 -08:00
Inigo Goiri
0eff245d3d HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:41:42 -08:00
Yiqun Lin
09017aee90 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:52:49 +08:00
Chao Sun
26395aef67 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 
(cherry picked from commit ae290a4bb4e514e2fe9b40d28426a7589afe2a3f)
 2018-02-26 16:18:51 -08:00
Wei Yan
1fb87df87e HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:29:11 -08:00
Kihwal Lee
1087b9af8c HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d8798624ae2a144bc58fa41db826704)
 2018-02-26 10:30:50 -06:00
Xiaoyu Yao
21d4b5fd24 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 
(cherry picked from commit 1e84e46f1621fe694f806bfc41d3b2a06c9500b6)
 2018-02-23 19:41:28 -08:00
Xiao Chen
1fbd419957 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d323359587dca7831f74c9d065c2fccc60d)
 2018-02-23 14:00:25 -08:00
Xiaoyu Yao
ec1b445c62 HDFS-13136. Avoid taking FSN lock while doing group member lookup for FSD permission check. Contributed by Xiaoyu Yao. 
(cherry picked from commit e978e1d9064fc794ea83217d0af2315d4167211e)
 2018-02-22 20:18:02 -08:00
Steve Loughran
c2bbe22c5a HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions. 
Contributed by Andras Bokor.
 2018-02-21 11:11:28 +00:00
Anu Engineer
c0d8103ba2 HDFS-13175. Add more information for checking argument in DiskBalancerVolume. 
Contributed by  Lei (Eddy) Xu.

(cherry picked from commit 121e1e1280c7b019f6d2cc3ba9eae1ead0dd8408)
 2018-02-20 19:59:31 -08:00
Yiqun Lin
7601489995 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 
(cherry picked from commit 8896d20b91520053a6bbfb680adb345cd24f4142)
 2018-02-20 13:06:26 -08:00
Kihwal Lee
7b83fca453 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:35:27 -06:00
Kihwal Lee
11ff6327af xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:14:42 -06:00
Brahma Reddy Battula
055d5e0b85 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
 2018-02-12 22:17:35 +05:30
Arpit Agarwal
41cf2b8e7c HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:16:48 -08:00
Yiqun Lin
38febc0ae2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 13:59:38 +08:00