Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,344 Commits 326 Branches 371 Tags
Commit Graph

4800 Commits

Author SHA1 Message Date
Inigo Goiri
972c322301 HDFS-12886. Ignore minReplication for block recovery. Contributed by Lukas Majercak. 
(cherry picked from commit 08ff1586d5d3e39f546200f9e696f62ea4cf000d)
 2018-03-16 10:31:02 -07:00
Inigo Goiri
4615c0bba2 HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 
(cherry picked from commit e71bc00a471422ddb26dd54e706f09f0fe09925c)
 2018-03-16 09:15:23 -07:00
Inigo Goiri
3a5f50fa6e Revert "HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri." 
This reverts commit ccf07fdafd74d5d5782deaed9839a4122e463d4b.
 2018-03-16 09:12:30 -07:00
Inigo Goiri
b1b10007a4 HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-03-15 11:30:52 -07:00
Inigo Goiri
ccf07fdafd HDFS-13224. RBF: Resolvers to support mount points across multiple subclusters. Contributed by Inigo Goiri. 
(cherry picked from commit e71bc00a471422ddb26dd54e706f09f0fe09925c)
 2018-03-15 10:51:04 -07:00
Yiqun Lin
3fad968624 HDFS-13261. Fix incorrect null value check. Contributed by Jianfei Jiang. 2018-03-15 11:12:25 +08:00
Inigo Goiri
299b40ff8b HDFS-13198. RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router. Contributed by Wei Yan. 
(cherry picked from commit 41fc7f80beb32f4a1fac73faf7c9116ad5ee3420)
 2018-03-14 13:16:32 -07:00
Yiqun Lin
b3d56cb835 HDFS-12773. RBF: Improve State Store FS implementation. Contributed by Inigo Goiri. 
(cherry picked from commit 76be6cbf6c33f866794f27ca2560ca7c7b2fa0e7)
 2018-03-14 11:25:08 +08:00
Chris Douglas
340cd5f1b1 HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc04a86be91a9b72073cdc30bec04d3b)
(cherry picked from commit 09940b1eb3b7ed764149f4a993c1857e9c6ad938)
(cherry picked from commit a03c8ea61f6e30a9d462571ace23858b6e0fd1c9)
 2018-03-12 20:08:46 -07:00
Yiqun Lin
38c21ec651 HDFS-13226. RBF: Throw the exception if mount table entry validated failed. Contributed by maobaolong. 
(cherry picked from commit 19292bc264cada5117ec76063d36cc88159afdf4)
 2018-03-13 11:06:53 +08:00
Yiqun Lin
5edbd1a258 HDFS-13253. RBF: Quota management incorrect parent-child relationship judgement. Contributed by Yiqun Lin. 
(cherry picked from commit 7fab787de72756863a91c2358da5c611afdb80e9)
 2018-03-13 10:33:12 +08:00
Akira Ajisaka
b0c80f1c81
HDFS-12156. TestFSImage fails without -Pnative 
(cherry picked from commit 319defafc105c0d0b69b83828b578d9c453036f5)
 2018-03-13 11:29:44 +09:00
Yiqun Lin
2ca6cf3170 HDFS-10803. TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available. Contributed by Yiqun Lin. 
(cherry picked from commit 4afd50b10650a72162c40cf86dea44676013f262)
 2018-03-13 10:18:23 +08:00
Inigo Goiri
7fded36e4c HDFS-13241. RBF: TestRouterSafemode failed if the port 8888 is in use. Contributed by maobaolong. 
(cherry picked from commit 91c82c90f05ea75fe59c6ffad3dc3fcac1429e9e)
 2018-03-12 17:30:45 -07:00
Inigo Goiri
737dca1959 HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 2018-03-12 12:54:42 -07:00
Anu Engineer
88c749a903 HDFS-10618. TestPendingReconstruction#testPendingAndInvalidate is flaky due to race condition. Contributed by Eric Badger. 2018-03-12 12:14:21 -07:00
Inigo Goiri
457895d50f Revert "HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin." 
This reverts commit ab0f12708452f22d3780a30b354f7f8a24366500.
 2018-03-12 11:44:06 -07:00
Inigo Goiri
ab0f127084 HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 
(cherry picked from commit ac627f561f0946e98a650850fb507536cbd2f2c4)
 2018-03-12 10:04:51 -07:00
Yiqun Lin
3a34924896 HDFS-13240. RBF: Update some inaccurate document descriptions. Contributed by Yiqun Lin. 
(cherry picked from commit 4743d4a2c70a213a41804a24c776e6db00e1b90d)
 2018-03-10 11:33:44 +08:00
Inigo Goiri
0dadcf9ea3 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 
(cherry picked from commit 8133cd5305d7913453abb2d48da12f672c0ce334)
 2018-03-09 18:28:59 -08:00
Inigo Goiri
4b6b07690e HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 
(cherry picked from commit afe1a3ccd56a12fec900360a8a2855c080728e65)
 2018-03-09 17:21:10 -08:00
Yiqun Lin
aa748c6bca HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 
(cherry picked from commit 122805b43acff2b094bd984fa76dbc8d2e110edd)
 2018-03-09 15:51:56 +08:00
Kihwal Lee
0aa52d4085 HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. Contributed by Manoj Govindassamy. 2018-03-08 14:22:17 -06:00
Inigo Goiri
95f34ce462 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 
(cherry picked from commit 0c2b969e0161a068bf9ae013c4b95508dfb90a8a)
(cherry picked from commit 44abf77d9b371b0d06569bd4881356ac157b490e)
 2018-03-08 10:24:41 -08:00
Yiqun Lin
a5d9691b54 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 
(cherry picked from commit 58ea2d7a65ccd8b7775021bae1d24b9e5561e67b)
 2018-03-07 15:51:42 +08:00
Xiaoyu Yao
313fdb9968 HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 
(cherry picked from commit edf9445708ffb7a9e59cb933e049b540f99add1e)
 2018-03-06 20:01:51 -08:00
Xiaoyu Yao
4f3c4e4514 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d435039f69ea35f592856ca04d378809)
 2018-03-06 18:49:13 -08:00
Xiao Chen
0882725c88 HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 2018-03-05 09:44:42 -08:00
Yiqun Lin
08cd911b69 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 
(cherry picked from commit a9c14b11193adeaa31389578f4cb90fa79cad8c3)
 2018-02-28 10:25:49 +08:00
Inigo Goiri
2f18785f39 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:42:28 -08:00
Yiqun Lin
1b0aa0e87a HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:54:37 +08:00
Wei Yan
54803ebe49 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:32:27 -08:00
Kihwal Lee
4b43f2aa56 HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d8798624ae2a144bc58fa41db826704)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java
 2018-02-26 10:58:07 -06:00
Arpit Agarwal
79af42f095 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-24 15:12:36 -08:00
Xiao Chen
80f7165374 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d323359587dca7831f74c9d065c2fccc60d)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
 2018-02-23 14:00:33 -08:00
Gabor Bota
696e8e0299
HADOOP-15236. Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c36b4aa31ce25fbe5fa173bce36da2950d74a475)
(cherry picked from commit b8942cb3bf4a458c78654bc166447806523409b5)
 2018-02-23 13:58:08 +09:00
Xiao Chen
49ed7d7fc9 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Gabor Bota, Wei-Chiu Chuang. 2018-02-19 19:15:02 -08:00
Yiqun Lin
a278ad6b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:58:28 +08:00
Kihwal Lee
fe044e69c3 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:38:49 -06:00
Kihwal Lee
b725fd6924 xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:28:45 -06:00
Brahma Reddy Battula
24af0bc144 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
 2018-02-12 22:25:44 +05:30
Arpit Agarwal
82530c966a HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:02:42 -08:00
Yiqun Lin
d49074f0a2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 14:00:55 +08:00
Brahma Reddy Battula
9873eb63a7 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-08 18:33:11 +05:30
Yongjun Zhang
b7f7fb003e HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
(cherry picked from commit f491f717e9ee6b75ad5cfca48da9c6297e94a8f7)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
(cherry picked from commit 60d82ac61f2afa2a61c8eeabf20e45b5a89311d0)
 2018-02-07 15:35:41 -08:00
Yongjun Zhang
6dfea05049 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16e81b4bd6fae443a997ef24d588a6e8)
 2018-02-02 23:33:10 -08:00
Yiqun Lin
7cd3770fee HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit b0627c891b0e90e29dab2bec64a01c2c2ffe4ed0)
 2018-02-02 17:38:32 +08:00
Inigo Goiri
7b8cc048cc HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204cebf4b7060696a07b1777723a6dfa7d9d)
 2018-02-01 10:38:39 -08:00
Yiqun Lin
e5cac88c82 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit dbb9dded33b3cff3b630e98300d30515a9d1eec4)
 2018-01-31 09:14:31 -08:00
Weiwei Yang
0c2c4c20cb HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 
(cherry picked from commit 2e7331ca264dd366b975f3c8e610cf84eb8cc155)
 2018-01-30 21:40:07 -08:00