Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,286 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7868 Commits

Author SHA1 Message Date
Uma Maheswara Rao G 80111fe5bb HDFS-15427. Merged ListStatus with Fallback target filesystem and InternalDirViewFS. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 7c02d1889b)
 2020-06-27 15:55:41 -07:00
Uma Maheswara Rao G 6c22210baa HDFS-15418. ViewFileSystemOverloadScheme should represent mount links as non symlinks. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit b27810aa60)
 2020-06-27 15:55:11 -07:00
Uma Maheswara Rao G aa8de2f43b HDFS-15387. FSUsage#DF should consider ViewFSOverloadScheme in processPath. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 785b1def95)
(cherry picked from commit 120ee793fc)
 2020-06-27 15:53:07 -07:00
Ayush Saxena e80421820a HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena 
(cherry picked from commit cc671b16f7)
(cherry picked from commit bee2846bee)
 2020-06-27 15:52:47 -07:00
Uma Maheswara Rao G e7cead1143 HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 76fa0222f0)
(cherry picked from commit 418580446b)
 2020-06-27 15:52:18 -07:00
Uma Maheswara Rao G 852ee71354 HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit ed83c865dd)
(cherry picked from commit 0b5e202614)
 2020-06-27 15:51:51 -07:00
Uma Maheswara Rao G 84ceb6d520 HDFS-15322. Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 4734c77b4b)
(cherry picked from commit 8e71e85af7)
 2020-06-27 15:51:28 -07:00
Abhishek Das 7cf9601987 HADOOP-17024. ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). Contributed by Abhishek Das. 
(cherry picked from commit ce4ec74453)
(cherry picked from commit 5b248de42d)
 2020-06-27 15:50:41 -07:00
Uma Maheswara Rao G 3dcc9aed4d HDFS-15306. Make mount-table to read from central place ( Let's say from HDFS). Contributed by Uma Maheswara Rao G. 
(cherry picked from commit ac4a2e11d9)
(cherry picked from commit 544996c857)
 2020-06-27 15:49:48 -07:00
S O'Donnell 240974932b HDFS-15406. Improve the speed of Datanode Block Scan. Contributed by hemanthboyina 
(cherry picked from commit 123777823e)
 2020-06-18 12:44:42 +01:00
Takanobu Asanuma 695d20615d HDFS-15403. NPE in FileIoProvider#transferToSocketFully. Contributed by hemanthboyina. 
(cherry picked from commit f41a144077)
 2020-06-15 09:18:22 +09:00
Ayush Saxena 452d801f1b HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena. 
*Added missed test file.
 2020-06-09 18:48:48 +05:30
Toshihiro Suzuki e5a02a7499 HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories (#2052) 
Contributed by Toshihiro Suzuki.

(cherry picked from commit 545a0a147c)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
 2020-06-05 12:26:52 +01:00
Takanobu Asanuma d6de530bbb HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina. 
(cherry picked from commit 4d22d1c58f)
 2020-05-23 17:31:59 +09:00
Chen Liang 1813d25bf2 HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang 2020-05-18 12:05:47 -07:00
Ayush Saxena b706ac11f7 HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei. 2020-05-12 18:44:56 +05:30
Uma Maheswara Rao G 0d0aac1a2a HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 9c8236d04d)
 2020-05-11 15:29:26 -07:00
Virajith Jalaparti e5e824bd6f HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2020-05-11 12:50:14 -07:00
Ayush Saxena c3fffa8451 HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole. 2020-05-08 00:57:24 +05:30
Inigo Goiri 53f6d26500 HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao. 
(cherry picked from commit 4a212242d9)
 2020-05-04 18:04:07 -07:00
Konstantin V Shvachko e95a4ddbb1 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ebb878bab9)
 2020-05-04 10:47:07 -07:00
Ayush Saxena d451c9e2a4 Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida. 2020-05-03 19:17:44 +05:30
Akira Ajisaka 5154f6eef4
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina. 
(cherry picked from commit 5e0eda5d5f)
 2020-04-28 06:19:17 +09:00
Mingliang Liu 57c1e7ad9b
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:44:27 -07:00
Dhiraj 6225fce50d
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:08:32 -07:00
Wei-Chiu Chuang 9e3b4c9ebd HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun. 
(cherry picked from commit 453771f170)
(cherry picked from commit d6c48f8100)
 2020-04-24 11:54:20 -07:00
Mingliang Liu e154d9d9f7
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 13:56:15 -07:00
Inigo Goiri 7d64d44515 HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena. 
(cherry picked from commit efdfe679d6)
 2020-04-21 09:50:20 -07:00
Stephen O'Donnell 041c93a26c HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a)
(cherry picked from commit aaad947c74)
 2020-04-16 20:19:14 -07:00
Eric Badger 17fc8e4a64 Revert "HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago." 
This reverts commit 26b51f3e22.
 2020-04-02 19:24:42 +00:00
Chen Liang 10fe37b704 HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand. 2020-03-27 13:01:00 -07:00
Ayush Saxena 6cd9290401 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li. 
(cherry picked from commit 62deab17a3)
 2020-03-25 17:12:52 -07:00
Eric Yang ba6b3a3848 HDFS-14434. Ignore user.name query parameter in secure WebHDFS. 
Contributed by KWON BYUNGCHANG

(cherry picked from commit d78854b928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java
 2020-03-25 17:12:01 -07:00
Giovanni Matteo Fumarola eca7bc7ac4 HDFS-14006. Refactor name node to allow different token verification implementations. Contributed by CR Hota. 
(cherry picked from commit 00d5e631b5)
 2020-03-25 17:06:17 -07:00
Ayush Saxena 1e3b0df6ab HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun. 
(cherry picked from commit 6191d4b4a0)
 2020-03-25 16:41:48 -07:00
Yiqun Lin 26b51f3e22 HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 
(cherry picked from commit 2b452b4e60)
 2020-03-25 16:31:42 -07:00
Ayush Saxena 853eafa81a HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena. 
(cherry picked from commit bb41ddaf1e)
 2020-03-25 16:31:34 -07:00
Wei-Chiu Chuang 1d55e1e229 HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs. 
Contributed by Wei-Chiu Chuang.

Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
(cherry picked from commit 096533c2dc)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
 2020-03-18 09:58:35 -07:00
S O'Donnell b7f33e2e37 HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree. 
(cherry picked from commit 25497799cb)
 2020-03-13 15:34:20 +00:00
Surendra Singh Lilhore 269aa25a73 HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree. 2020-03-12 19:10:27 +05:30
Surendra Singh Lilhore 57f887a529 HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree. 2020-03-10 23:56:42 +05:30
Inigo Goiri 1d89ba72f6 HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit 1d4d0fcbe1)
 2020-03-04 10:04:22 -08:00
Akshay Nehe 369f4f9c58
HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. (#1871) 
(cherry picked from commit bbd704bb82)
 2020-03-04 11:12:00 +09:00
Surendra Singh Lilhore d33aec93f1 HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena. 2020-02-28 18:43:57 +05:30
Masatake Iwasaki e1ff3b2f97 HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 037ec8cfb1)
 2020-02-27 16:36:46 -08:00
Konstantin V Shvachko c82e20f9f8 HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ecbcb058b8)
 2020-02-27 15:29:31 -08:00
Kihwal Lee 27cfda708e HDFS-15147. LazyPersistTestCase wait logic is error-prone. Contributed 
by Ahmed Hussein.
 2020-02-27 09:17:44 -06:00
Kihwal Lee de7edf58bd Revert "HADOOP-16888. [JDK11] Support JDK11 in the precommit job. Contributed by" 
Incorrect commit message

This reverts commit 749d7c0027.
 2020-02-27 09:16:55 -06:00
Kihwal Lee 749d7c0027 HADOOP-16888. [JDK11] Support JDK11 in the precommit job. Contributed by 
Ahmed Hussein.
 2020-02-27 09:13:20 -06:00
Ayush Saxena b182dd809c HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li. 2020-02-27 04:02:55 +05:30