Wei-Chiu Chuang
61915fbd83
HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs.
...
Contributed by Wei-Chiu Chuang.
Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
(cherry picked from commit 096533c2dc
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
(cherry picked from commit 1d55e1e229
)
2020-03-18 09:59:12 -07:00
S O'Donnell
4e1021975b
HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree.
2020-03-13 15:47:57 +00:00
Surendra Singh Lilhore
64eb5b2709
HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena.
2020-02-28 18:47:54 +05:30
Masatake Iwasaki
abee7402e4
HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 037ec8cfb1
)
2020-02-27 16:37:23 -08:00
Konstantin V Shvachko
4af188587f
HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit ecbcb058b8
)
2020-02-27 16:02:30 -08:00
Ayush Saxena
7bfa123d58
HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li.
...
(cherry picked from commit b182dd809c
)
2020-02-27 14:55:50 -08:00
Kihwal Lee
b92477c638
HDFS-15147. LazyPersistTestCase wait logic is error-prone. Contributed
...
by Ahmed Hussein.
(cherry picked from commit 27cfda708e
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/LazyPersistTestCase.java
2020-02-27 09:45:12 -06:00
Stephen O'Donnell
8aaa8d1b71
HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 900430b990
)
(cherry picked from commit 2377649cdb
)
2020-02-25 13:29:15 -08:00
Wei-Chiu Chuang
268e35d3ae
HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun.
...
(cherry picked from commit cb4daeaad95470125c59ccb80b988261ae83d534)
(cherry picked from commit f9437b1817
)
2020-02-24 13:37:59 -08:00
Ayush Saxena
d318b48e12
HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li.
2020-02-24 20:17:52 +05:30
Konstantin V Shvachko
cd0525054d
HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit 6f84269bcd
)
2020-02-21 10:56:31 -08:00
Masatake Iwasaki
8095e3b149
HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. ( #1758 )
...
(cherry picked from commit 2338d25dc7
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
(cherry picked from commit 610805ec72
)
2020-02-21 13:51:26 +09:00
Kihwal Lee
c54051def5
HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
2020-02-17 16:07:03 -06:00
Ayush Saxena
2190fb1fcf
HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang
2020-02-16 10:09:40 +05:30
Ayush Saxena
cf5a7186e5
HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena.
2020-02-14 22:11:39 +05:30
Surendra Singh Lilhore
6a4297adc4
HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina.
2020-02-13 23:04:28 +05:30
Kihwal Lee
bf09bfa2db
HDFS-14758. Make lease hard limit configurable and reduce the default.
...
Contributed by hemanthboyina.
(cherry picked from commit 9b8a78d97b
)
(cherry picked from commit f1840669cf
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java
2020-02-11 14:57:49 -06:00
Ayush Saxena
547f1798aa
HDFS-15157. Fix compilation failure for branch-3.1. Contributed by Ayush Saxena
2020-02-09 08:14:33 +05:30
Chen Liang
466f77a5cf
HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang.
...
(cherry picked from commit 226cd3977b
)
2020-02-04 12:36:46 -08:00
Kihwal Lee
a55a0a1f6d
HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan.
...
(cherry picked from commit 10a60fbe20
)
2020-02-04 12:24:58 -06:00
Stephen O'Donnell
8ea4787f9f
HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1e3a0b0d93
)
(cherry picked from commit e89436b183
)
2020-01-31 16:13:53 -08:00
Inigo Goiri
ffaf0d3c82
HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein.
...
(cherry picked from commit 1839c467f6
)
2020-01-31 11:25:36 -08:00
Chen Liang
5e1d89b74d
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang
2020-01-29 11:15:35 -08:00
Kihwal Lee
b8d87cfc32
HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails
...
intermittently. Contributed by Ahmed Hussein.
(cherry picked from commit 799d4c1cf4
)
2020-01-29 11:01:51 -06:00
Ayush Saxena
1293199532
HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun.
2020-01-28 23:17:21 +05:30
Kihwal Lee
7dd43cec7f
HDFS-15119. Allow expiration of cached locations in DFSInputStream.
...
Contributed by Ahmed Hussein.
(cherry picked from commit d10f77e3c9
)
(cherry picked from commit 1bb9667137
)
2020-01-24 09:53:26 -06:00
Kihwal Lee
b4e9725955
HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein.
...
(cherry picked from commit bd03053ea2
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java
(cherry picked from commit 4842708320
)
2020-01-22 09:28:01 -06:00
Ayush Saxena
12b258115b
HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena.
2020-01-11 10:34:37 +05:30
Chen Liang
5490061885
HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang.
...
(cherry picked from commit 26a969ec73
)
2020-01-10 17:19:05 -08:00
Kihwal Lee
82fa8d88ef
HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein.
...
(cherry picked from commit 5fb901ac40
)
2020-01-10 12:47:49 -06:00
Surendra Singh Lilhore
09d3707875
HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina.
2020-01-09 12:18:58 +05:30
Rakesh Radhakrishnan
b1eb47865c
HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He.
...
(cherry picked from commit 7030722e5d
)
2020-01-08 14:32:45 +05:30
Masatake Iwasaki
c5580af1c4
HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. ( #1797 )
...
(cherry picked from commit aba3f6c3e1
)
2020-01-08 17:06:49 +09:00
Masatake Iwasaki
4c8b01abcb
HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. ( #1775 )
...
(cherry picked from commit a43c177f1d
)
2020-01-08 12:38:51 +09:00
Rakesh Radhakrishnan
8b748c0308
HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He.
...
(cherry picked from commit d79cce20ab
)
2020-01-02 12:19:38 +05:30
Surendra Singh Lilhore
55510d32bf
HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina.
2019-12-29 11:25:28 +05:30
Takanobu Asanuma
454a0c2913
HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena.
...
(cherry picked from commit dc32f583af
)
2019-12-28 21:35:22 +09:00
Konstantin V Shvachko
1a22e4036c
HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit b98ac2a3af
)
2019-12-24 12:50:13 -08:00
Eric Badger
ef00db4ce9
Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui."
...
This reverts commit 27555620fa
.
2019-12-20 19:16:22 +00:00
Inigo Goiri
27555620fa
HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.
...
(cherry picked from commit 52d7b745c6
)
2019-12-19 09:47:30 -08:00
Shashikant Banerjee
280cd6dd1e
HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.
...
(cherry picked from commit fdd96e46d1
)
2019-12-18 23:11:18 +05:30
Chen Liang
ea0eeb8f1a
HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.
...
(cherry picked from commit 65c4660bcd
)
(cherry picked from commit d4a6901c42
)
2019-12-12 15:58:09 -08:00
Erik Krogen
b06965ae32
HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.
...
(cherry picked from c174d50b30
)
(cherry picked from 3402c87353
)
2019-12-11 09:41:11 -08:00
Surendra Singh Lilhore
5eb21b83df
HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore.
2019-11-29 00:07:45 +05:30
Ayush Saxena
35f0478042
HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar.
2019-11-23 02:47:59 +05:30
Surendra Singh Lilhore
8551c04726
HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina.
2019-11-22 00:45:55 +05:30
He Xiaoqiao
d101b74814
HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd
)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
2019-11-19 12:22:54 -08:00
Siyao Meng
df13f6d485
HADOOP-16703. Backport HADOOP-16152 to branch-3.1. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-11-19 12:14:27 -08:00
Ayush Saxena
b4f0c63a6a
HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
...
(cherry picked from commit 9e287054a8
)
(cherry picked from commit d16441460215155b25fa649a7ca733e579d97929)
2019-11-18 10:23:48 -08:00
Erik Krogen
17779adb32
HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
...
(cherry picked from b2cc8b6b4a
)
(cherry picked from 60655bfe54
)
2019-11-15 11:14:49 -08:00