Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,232 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

8287 Commits

Author SHA1 Message Date
Eric Yang 9db6e96db3
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454)
 2020-05-02 21:51:41 +09:00
Akira Ajisaka 5154f6eef4
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina. 
(cherry picked from commit 5e0eda5d5f)
 2020-04-28 06:19:17 +09:00
Mingliang Liu 57c1e7ad9b
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:44:27 -07:00
Dhiraj 6225fce50d
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:08:32 -07:00
Wei-Chiu Chuang 9e3b4c9ebd HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun. 
(cherry picked from commit 453771f170)
(cherry picked from commit d6c48f8100)
 2020-04-24 11:54:20 -07:00
Mingliang Liu e154d9d9f7
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 13:56:15 -07:00
Inigo Goiri 7d64d44515 HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena. 
(cherry picked from commit efdfe679d6)
 2020-04-21 09:50:20 -07:00
Stephen O'Donnell 041c93a26c HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a)
(cherry picked from commit aaad947c74)
 2020-04-16 20:19:14 -07:00
Eric Badger 17fc8e4a64 Revert "HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago." 
This reverts commit 26b51f3e22.
 2020-04-02 19:24:42 +00:00
Chen Liang 10fe37b704 HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand. 2020-03-27 13:01:00 -07:00
Ayush Saxena 6cd9290401 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li. 
(cherry picked from commit 62deab17a3)
 2020-03-25 17:12:52 -07:00
Eric Yang ba6b3a3848 HDFS-14434. Ignore user.name query parameter in secure WebHDFS. 
Contributed by KWON BYUNGCHANG

(cherry picked from commit d78854b928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java
 2020-03-25 17:12:01 -07:00
Giovanni Matteo Fumarola eca7bc7ac4 HDFS-14006. Refactor name node to allow different token verification implementations. Contributed by CR Hota. 
(cherry picked from commit 00d5e631b5)
 2020-03-25 17:06:17 -07:00
Isa Hekmatizadeh 152cbc6457 HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error (#1902). Contributed by zhengchenyu. 
(cherry picked from commit d9c4f1129c)
 2020-03-25 16:52:52 -07:00
Ayush Saxena 1e3b0df6ab HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun. 
(cherry picked from commit 6191d4b4a0)
 2020-03-25 16:41:48 -07:00
Yiqun Lin 26b51f3e22 HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 
(cherry picked from commit 2b452b4e60)
 2020-03-25 16:31:42 -07:00
Ayush Saxena 853eafa81a HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena. 
(cherry picked from commit bb41ddaf1e)
 2020-03-25 16:31:34 -07:00
Akira Ajisaka 1bbbef38d0
HDFS-15232. Fix libhdfspp test failures with GCC 7. (#1906) 
(cherry picked from commit f59f6891c8)
 2020-03-24 11:38:12 +09:00
Wei-Chiu Chuang 1d55e1e229 HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs. 
Contributed by Wei-Chiu Chuang.

Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
(cherry picked from commit 096533c2dc)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
 2020-03-18 09:58:35 -07:00
Surendra Singh Lilhore e9916052b1 HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena. 2020-03-15 20:53:45 +05:30
Ayush Saxena 87532b6e3f HDFS-15197. [SBN read] Change ObserverRetryOnActiveException log to debug. Contributed by Chen Liang. 2020-03-15 16:31:20 +05:30
S O'Donnell b7f33e2e37 HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree. 
(cherry picked from commit 25497799cb)
 2020-03-13 15:34:20 +00:00
Surendra Singh Lilhore 269aa25a73 HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree. 2020-03-12 19:10:27 +05:30
Surendra Singh Lilhore 57f887a529 HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree. 2020-03-10 23:56:42 +05:30
Inigo Goiri 1d89ba72f6 HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit 1d4d0fcbe1)
 2020-03-04 10:04:22 -08:00
Akshay Nehe 369f4f9c58
HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. (#1871) 
(cherry picked from commit bbd704bb82)
 2020-03-04 11:12:00 +09:00
Surendra Singh Lilhore d33aec93f1 HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena. 2020-02-28 18:43:57 +05:30
Ayush Saxena 9acbbd885c HDFS-12999. When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. Contributed by lufei and Fei Hui. 
(cherry picked from commit df622cf4a3)
 2020-02-27 16:36:53 -08:00
Masatake Iwasaki e1ff3b2f97 HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 037ec8cfb1)
 2020-02-27 16:36:46 -08:00
Konstantin V Shvachko c82e20f9f8 HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ecbcb058b8)
 2020-02-27 15:29:31 -08:00
Istvan Fajth e42ac486e7 HDFS-14668 Support Fuse with Users from multiple Security Realms (#1739) 
(cherry picked from commit 57aa048516)
 2020-02-27 07:49:31 -08:00
Kihwal Lee 27cfda708e HDFS-15147. LazyPersistTestCase wait logic is error-prone. Contributed 
by Ahmed Hussein.
 2020-02-27 09:17:44 -06:00
Kihwal Lee de7edf58bd Revert "HADOOP-16888. [JDK11] Support JDK11 in the precommit job. Contributed by" 
Incorrect commit message

This reverts commit 749d7c0027.
 2020-02-27 09:16:55 -06:00
Kihwal Lee 749d7c0027 HADOOP-16888. [JDK11] Support JDK11 in the precommit job. Contributed by 
Ahmed Hussein.
 2020-02-27 09:13:20 -06:00
Ayush Saxena b182dd809c HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li. 2020-02-27 04:02:55 +05:30
Stephen O'Donnell 2377649cdb HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 900430b990)
 2020-02-25 13:29:03 -08:00
Wei-Chiu Chuang f9437b1817 HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun. 
(cherry picked from commit cb4daeaad95470125c59ccb80b988261ae83d534)
 2020-02-24 13:37:48 -08:00
Ayush Saxena af46658baf HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li. 2020-02-24 20:17:31 +05:30
Konstantin V Shvachko 769397201a HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 6f84269bcd)
 2020-02-21 10:54:20 -08:00
Masatake Iwasaki 610805ec72 HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. (#1758) 
(cherry picked from commit 2338d25dc7)

 Conflicts:
        hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
 2020-02-21 13:40:11 +09:00
xiaoxiaopan118 b5321f1600
HDFS-15173. RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' (#1849) 
(cherry picked from commit 439d935e1d)
 2020-02-17 15:57:15 +09:00
Ayush Saxena 9a351e837f HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena. 2020-02-14 22:15:12 +05:30
Surendra Singh Lilhore e13ea1b094 HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina. 2020-02-13 22:52:50 +05:30
Ayush Saxena 9a1ce410f6 HDFS-15161. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close(). Contributed by Lisheng Sun 2020-02-12 20:29:09 +05:30
Kihwal Lee f1840669cf HDFS-14758. Make lease hard limit configurable and reduce the default. 
Contributed by hemanthboyina.

(cherry picked from commit 9b8a78d97b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/DfsClientConf.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend4.java
 2020-02-11 14:50:10 -06:00
Ayush Saxena 7be104a9be HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang 2020-02-08 10:41:15 +05:30
Chen Liang 226cd3977b HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 2020-02-04 12:34:19 -08:00
Kihwal Lee 5257afb153 HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 10a60fbe20)
 2020-02-04 12:24:12 -06:00
Stephen O'Donnell e89436b183 HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1e3a0b0d93)
 2020-01-31 16:13:42 -08:00
Inigo Goiri 2c8c6e8e22 HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit 1839c467f6)
 2020-01-31 11:24:37 -08:00