Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,720 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7586 Commits

Author SHA1 Message Date
Uma Maheswara Rao G fda93334dc HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G. 
(cherry picked from commit 9c8236d04d)
 2020-05-12 21:33:49 -07:00
Virajith Jalaparti bec5d7baa0 HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun. 2020-05-12 19:37:40 -07:00
Ayush Saxena d24e1bc388 HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei. 2020-05-12 18:45:52 +05:30
Babneet Singh 42997d14a9 HDFS-15270. Account for *env == NULL in hdfsThreadDestructor (#1951) 
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.

The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.

Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
(cherry picked from commit 1996351b0b)
(cherry picked from commit d1e5e393c3)
(cherry picked from commit 44fdee351d)
 2020-05-11 10:46:21 -07:00
Ayush Saxena 86c15b8287 HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest. 2020-05-10 11:45:13 +05:30
Ayush Saxena 8988f33ac3 HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole. 2020-05-08 00:57:41 +05:30
He Xiaoqiao dbb2809026 HDFS-15272. Backport HDFS-12862 to branch-3.1. Contributed by Xiaoqiao He, Wang XL. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2020-05-04 18:13:01 -07:00
Inigo Goiri 5612537df0 HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao. 
(cherry picked from commit 4a212242d9)
 2020-05-04 18:03:57 -07:00
Konstantin V Shvachko 8358d7c175 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ebb878bab9)
 2020-05-04 10:50:14 -07:00
Ayush Saxena 69163f5e5d Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida. 2020-05-03 19:18:04 +05:30
Eric Yang 4b7d6667fd
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454)
 2020-05-02 21:52:06 +09:00
Gabor Bota ec6d2a8402 Preparing for 3.1.5 development 
Change-Id: Iabc64aba7392e3b6f9e4e18109fcaa2cfc01d1f9
 2020-04-29 11:18:18 +02:00
Akira Ajisaka 74ec3c001d
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina. 
(cherry picked from commit 5e0eda5d5f)
 2020-04-28 06:30:38 +09:00
Mingliang Liu 75354285e6
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:49:29 -07:00
Dhiraj 68dcdad666
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:53:32 -07:00
Wei-Chiu Chuang 6f7a30fd71 HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun. 
(cherry picked from commit 453771f170)
(cherry picked from commit d6c48f8100)
(cherry picked from commit 9e3b4c9ebd)
 2020-04-24 11:54:29 -07:00
Mingliang Liu 03ff1d3bd8
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 14:14:19 -07:00
Inigo Goiri 3938aa422e HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena. 
(cherry picked from commit efdfe679d6)
(cherry picked from commit 7d64d44515)
 2020-04-21 09:50:31 -07:00
Stephen O'Donnell 0ea8f3a19c HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a)
(cherry picked from commit aaad947c74)
(cherry picked from commit 041c93a26c)
 2020-04-16 20:19:26 -07:00
Ayush Saxena 6a4c3fad66 HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li. 
(cherry picked from commit 62deab17a3)
(cherry picked from commit 6cd9290401)
 2020-03-25 17:14:24 -07:00
Eric Yang b837431a08 HDFS-14434. Ignore user.name query parameter in secure WebHDFS. 
Contributed by KWON BYUNGCHANG

(cherry picked from commit d78854b928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsTokens.java

(cherry picked from commit ba6b3a3848)
 2020-03-25 17:14:18 -07:00
Giovanni Matteo Fumarola 4aa7734fc7 HDFS-14006. Refactor name node to allow different token verification implementations. Contributed by CR Hota. 
(cherry picked from commit 00d5e631b5)
(cherry picked from commit eca7bc7ac4)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
 2020-03-25 17:13:58 -07:00
Isa Hekmatizadeh d64f688198 HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error (#1902). Contributed by zhengchenyu. 
(cherry picked from commit d9c4f1129c)
 2020-03-25 16:52:42 -07:00
Ayush Saxena 4891f24a94 HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun. 
(cherry picked from commit 6191d4b4a0)
(cherry picked from commit 1e3b0df6ab)
 2020-03-25 16:42:09 -07:00
Yiqun Lin f1a19b7a3f HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 
(cherry picked from commit 2b452b4e60)
 2020-03-25 16:31:54 -07:00
Wei-Chiu Chuang 61915fbd83 HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs. 
Contributed by Wei-Chiu Chuang.

Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
(cherry picked from commit 096533c2dc)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties

(cherry picked from commit 1d55e1e229)
 2020-03-18 09:59:12 -07:00
Surendra Singh Lilhore a3224ff799 HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena. 2020-03-15 20:59:00 +05:30
Ayush Saxena 8566c65917 HDFS-15197. [SBN read] Change ObserverRetryOnActiveException log to debug. Contributed by Chen Liang. 2020-03-15 16:31:58 +05:30
S O'Donnell 4e1021975b HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree. 2020-03-13 15:47:57 +00:00
Surendra Singh Lilhore 64eb5b2709 HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena. 2020-02-28 18:47:54 +05:30
Ayush Saxena 354e63b366 HDFS-12999. When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. Contributed by lufei and Fei Hui. 
(cherry picked from commit df622cf4a3)
 2020-02-27 16:37:30 -08:00
Masatake Iwasaki abee7402e4 HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 037ec8cfb1)
 2020-02-27 16:37:23 -08:00
Konstantin V Shvachko 4af188587f HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ecbcb058b8)
 2020-02-27 16:02:30 -08:00
Ayush Saxena 7bfa123d58 HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li. 
(cherry picked from commit b182dd809c)
 2020-02-27 14:55:50 -08:00
Istvan Fajth b5022b0515 HDFS-14668 Support Fuse with Users from multiple Security Realms (#1739) 
(cherry picked from commit 57aa048516)
(cherry picked from commit e42ac486e7)
 2020-02-27 07:49:45 -08:00
Kihwal Lee b92477c638 HDFS-15147. LazyPersistTestCase wait logic is error-prone. Contributed 
by Ahmed Hussein.

(cherry picked from commit 27cfda708e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/LazyPersistTestCase.java
 2020-02-27 09:45:12 -06:00
Stephen O'Donnell 8aaa8d1b71 HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 900430b990)
(cherry picked from commit 2377649cdb)
 2020-02-25 13:29:15 -08:00
Wei-Chiu Chuang 268e35d3ae HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun. 
(cherry picked from commit cb4daeaad95470125c59ccb80b988261ae83d534)
(cherry picked from commit f9437b1817)
 2020-02-24 13:37:59 -08:00
Ayush Saxena d318b48e12 HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li. 2020-02-24 20:17:52 +05:30
Konstantin V Shvachko cd0525054d HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 6f84269bcd)
 2020-02-21 10:56:31 -08:00
Masatake Iwasaki 8095e3b149 HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. (#1758) 
(cherry picked from commit 2338d25dc7)

 Conflicts:
        hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java

(cherry picked from commit 610805ec72)
 2020-02-21 13:51:26 +09:00
Kihwal Lee c54051def5 HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
 2020-02-17 16:07:03 -06:00
xiaoxiaopan118 b59fa3e571
HDFS-15173. RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' (#1849) 
(cherry picked from commit 439d935e1d)
 2020-02-17 15:58:35 +09:00
Ayush Saxena 2190fb1fcf HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang 2020-02-16 10:09:40 +05:30
Ayush Saxena cf5a7186e5 HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena. 2020-02-14 22:11:39 +05:30
Surendra Singh Lilhore 6a4297adc4 HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina. 2020-02-13 23:04:28 +05:30
Ayush Saxena 83ab853733 HDFS-15161. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close(). Contributed by Lisheng Sun 2020-02-12 20:30:13 +05:30
Kihwal Lee bf09bfa2db HDFS-14758. Make lease hard limit configurable and reduce the default. 
Contributed by hemanthboyina.

(cherry picked from commit 9b8a78d97b)
(cherry picked from commit f1840669cf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
 2020-02-11 14:57:49 -06:00
Ayush Saxena 547f1798aa HDFS-15157. Fix compilation failure for branch-3.1. Contributed by Ayush Saxena 2020-02-09 08:14:33 +05:30
Chen Liang 466f77a5cf HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang. 
(cherry picked from commit 226cd3977b)
 2020-02-04 12:36:46 -08:00
Kihwal Lee a55a0a1f6d HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 10a60fbe20)
 2020-02-04 12:24:58 -06:00
Stephen O'Donnell 8ea4787f9f HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1e3a0b0d93)
(cherry picked from commit e89436b183)
 2020-01-31 16:13:53 -08:00
Inigo Goiri ffaf0d3c82 HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein. 
(cherry picked from commit 1839c467f6)
 2020-01-31 11:25:36 -08:00
Chen Liang 5e1d89b74d [SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang 2020-01-29 11:15:35 -08:00
Kihwal Lee b8d87cfc32 HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails 
intermittently. Contributed by Ahmed Hussein.

(cherry picked from commit 799d4c1cf4)
 2020-01-29 11:01:51 -06:00
Ayush Saxena 1293199532 HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun. 2020-01-28 23:17:21 +05:30
Kihwal Lee 7dd43cec7f HDFS-15119. Allow expiration of cached locations in DFSInputStream. 
Contributed by Ahmed Hussein.

(cherry picked from commit d10f77e3c9)
(cherry picked from commit 1bb9667137)
 2020-01-24 09:53:26 -06:00
Kihwal Lee b4e9725955 HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein. 
(cherry picked from commit bd03053ea2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java

(cherry picked from commit 4842708320)
 2020-01-22 09:28:01 -06:00
Akira Ajisaka 2c84ea96e3
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li. 
(cherry picked from commit f6d20daf40)
 2020-01-21 18:05:13 +09:00
Wei-Chiu Chuang 286c94f13d HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu. 
(cherry picked from commit 6b86a5110e)
(cherry picked from commit aa3c96474f)
 2020-01-13 10:49:39 -08:00
Ayush Saxena 12b258115b HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena. 2020-01-11 10:34:37 +05:30
Chen Liang 5490061885 HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang. 
(cherry picked from commit 26a969ec73)
 2020-01-10 17:19:05 -08:00
Kihwal Lee 82fa8d88ef HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein. 
(cherry picked from commit 5fb901ac40)
 2020-01-10 12:47:49 -06:00
Surendra Singh Lilhore 09d3707875 HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina. 2020-01-09 12:18:58 +05:30
Rakesh Radhakrishnan b1eb47865c HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He. 
(cherry picked from commit 7030722e5d)
 2020-01-08 14:32:45 +05:30
Masatake Iwasaki c5580af1c4 HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. (#1797) 
(cherry picked from commit aba3f6c3e1)
 2020-01-08 17:06:49 +09:00
Masatake Iwasaki 4c8b01abcb
HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. (#1775) 
(cherry picked from commit a43c177f1d)
 2020-01-08 12:38:51 +09:00
Rakesh Radhakrishnan 8b748c0308 HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He. 
(cherry picked from commit d79cce20ab)
 2020-01-02 12:19:38 +05:30
Surendra Singh Lilhore 55510d32bf HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina. 2019-12-29 11:25:28 +05:30
Takanobu Asanuma 454a0c2913 HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena. 
(cherry picked from commit dc32f583af)
 2019-12-28 21:35:22 +09:00
Chandra Sanivarapu e33651ace7
HDFS-15073. Replace curator-shaded guava import with the standard one (#1782) 
(cherry picked from commit d8cd7098b4)
 2019-12-25 11:12:29 +09:00
Konstantin V Shvachko 1a22e4036c HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b98ac2a3af)
 2019-12-24 12:50:13 -08:00
Eric Badger ef00db4ce9 Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui." 
This reverts commit 27555620fa.
 2019-12-20 19:16:22 +00:00
Inigo Goiri 27555620fa HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui. 
(cherry picked from commit 52d7b745c6)
 2019-12-19 09:47:30 -08:00
Shashikant Banerjee 280cd6dd1e HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 
(cherry picked from commit fdd96e46d1)
 2019-12-18 23:11:18 +05:30
Chen Liang ea0eeb8f1a HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 
(cherry picked from commit 65c4660bcd)
(cherry picked from commit d4a6901c42)
 2019-12-12 15:58:09 -08:00
He Xiaoqiao 974706c5fd HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
(cherry picked from commit 8ba4e13dcc)
 2019-12-12 02:23:22 -08:00
Erik Krogen b06965ae32 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 
(cherry picked from c174d50b30)
(cherry picked from 3402c87353)
 2019-12-11 09:41:11 -08:00
Surendra Singh Lilhore 5eb21b83df HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-29 00:07:45 +05:30
Ayush Saxena 35f0478042 HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:47:59 +05:30
Surendra Singh Lilhore 8551c04726 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:45:55 +05:30
He Xiaoqiao d101b74814 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
 2019-11-19 12:22:54 -08:00
Siyao Meng df13f6d485 HADOOP-16703. Backport HADOOP-16152 to branch-3.1. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-19 12:14:27 -08:00
Ayush Saxena b4f0c63a6a HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun. 
(cherry picked from commit 9e287054a8)
(cherry picked from commit d16441460215155b25fa649a7ca733e579d97929)
 2019-11-18 10:23:48 -08:00
Erik Krogen 17779adb32 HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen. 
(cherry picked from b2cc8b6b4a)
(cherry picked from 60655bfe54)
 2019-11-15 11:14:49 -08:00
Erik Krogen 5cf36aa2b1 HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen. 
(cherry picked from 586defe711)
(cherry picked from dec765b329)
 2019-11-13 11:57:59 -08:00
Chao Sun 7e0f1eabd8 HDFS-14959: [SBNN read] access time should be turned off (#1706) Contributed by Chao Sun. 
(cherry picked from commit 97ec34e117)
(cherry picked from commit 0c12d633b9)
 2019-11-12 11:41:09 -08:00
Surendra Singh Lilhore 9179046368 HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina. 
(cherry picked from commit 320008bb7c)
 2019-11-09 22:36:48 +05:30
Ayush Saxena 9aa59c0629 HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan. 2019-11-07 22:42:07 +05:30
Chen Liang 76ca864406 HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen. 2019-11-06 16:21:41 -08:00
Chen Liang c6cbf67b82 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko. 2019-11-06 15:11:09 -08:00
Ayush Saxena 5841d0b2f9 HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui. 2019-11-06 22:17:11 +05:30
Surendra Singh Lilhore 4690142e86 HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit c36014165c)
 2019-11-06 19:48:16 +05:30
Wei-Chiu Chuang c9ff4b7a1b HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao. 
(cherry picked from commit eb73ba6ed5)
(cherry picked from commit 240b5ee380)
 2019-11-04 09:41:05 -08:00
Zhao Junwang cda8b2ac69 HDFS-14925. Rename operation should check nest snapshot (#1670) 
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.

Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
(cherry picked from commit de6b8b0c0b)
(cherry picked from commit c9fc118991)
 2019-11-01 16:39:29 -07:00
Ayush Saxena 9956d18129 HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong. 2019-11-01 22:52:12 +05:30
Ayush Saxena 98e3c6af2c HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui. 2019-10-31 23:53:37 +05:30
Duo Zhang 07f59c023c HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches 
Signed-off-by: stack <stack@apache.org>
 2019-10-30 21:30:34 -07:00
Jonathan Hung 4d274f60bc Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aa)
 2019-10-30 21:00:55 -07:00
Eric Badger 50c1f932c3 HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger 
(cherry picked from commit 9ef6ed9c1c)
 2019-10-28 16:54:05 +00:00