15935fa865
HDFS-16947. RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store ( #5470 )
2023-03-16 00:59:55 +08:00
eee2ea075d
HDFS-16942. Addendum. Send error to datanode if FBR is rejected due to bad lease ( #5478 ). Contributed by Stephen O'Donnell/
2023-03-15 10:03:00 +05:30
ca6f5afb6d
HDFS-16942. Send error to datanode if FBR is rejected due to bad lease ( #5460 )
2023-03-11 16:40:07 +00:00
927401886a
HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression ( #5434 )
...
Contributed by Shilun Fan
2023-03-06 15:26:53 +00:00
2cb0c35fc1
HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. ( #5450 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-06 20:10:31 +08:00
6bd2444815
HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist ( #5400 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-03-01 16:18:38 -08:00
162288bc0a
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… ( #5322 )
...
HDFS-16896 clear ignoredNodes list when we clear deadnode list on refetchLocations.
ignoredNodes list is only used on hedged read codepath
Co-authored-by: Tom McCormick <tmccormi@linkedin.com>
2023-03-01 19:47:04 +00:00
2ab7eb4caa
HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks ( #5432 )
...
Contributed by Viraj Jasani
2023-03-01 18:53:10 +00:00
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
0ca5686034
HDFS-16917 Add transfer rate quantile metrics for DataNode reads ( #5397 )
...
Co-authored-by: Ravindra Dingankar <rdingankar@linkedin.com>
2023-02-27 18:26:32 +00:00
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
a90238c0b8
HADOOP-18631. Migrate Async appenders to log4j properties ( #5418 )
2023-02-26 01:47:44 +08:00
8025a60ae7
HDFS-16901: Minor fix for unit test.
...
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 16:37:49 -08:00
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
88914cada0
HDFS-16925. Namenode audit log to only include IP address of client ( #5407 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-21 08:26:32 +08:00
a3b500d046
HDFS-16916. Improve the use of JUnit Test in DFSClient ( #5404 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:17:37 +08:00
723535b788
HDFS-16914. Add some logs for updateBlockForPipeline RPC. ( #5381 )
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-16 09:25:50 +08:00
4cbe19f3a2
HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down ( #5390 )
2023-02-15 16:16:04 +00:00
fe0541b58d
HDFS-16913. Fix flaky some unit tests since they offen timeout ( #5377 )
...
Co-authored-by: gf13871 <gf13871@ly.com>
Reviewed-by: Tao Li <tomscut@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-14 15:29:38 +08:00
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
f42c89dffb
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5372 )
...
This is a followup to the original patch, 08f58ecf07
2023-02-13 14:31:32 +00:00
f3c4277576
HDFS-16882. RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath ( #5276 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-11 08:00:43 +08:00
17c8cdf63c
HDFS-16907. ADDENDUM: Remove unused variables from testDataNodeMXBeanLastHeartbeats. ( #5373 )
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-02-10 00:33:20 +05:30
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders ( #5301 )
...
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
2023-02-09 13:28:34 +00:00
d5c046518e
HDFS-16910. Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN ( #5359 )
2023-02-09 10:47:57 +08:00
08f58ecf07
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5342 )
...
Contributed by Steve Vaughan Jr
2023-02-08 17:15:42 +00:00
3ba058a894
HDFS-16898. Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat ( #5330 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-02-08 11:19:07 +08:00
7e919212c4
Revert "HDFS-16898. Make write lock fine-grain in method processCommandFromActor ( #5330 ). Contributed by ZhangHB."
...
This reverts commit eb04ecd29d
2023-02-08 11:15:22 +08:00
eb04ecd29d
HDFS-16898. Make write lock fine-grain in method processCommandFromActor ( #5330 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-02-08 10:33:28 +08:00
d310642626
HDFS-16895. [RBF] NamenodeHeartbeatService should use credentials of logged in user
2023-02-07 18:45:05 +00:00
f02c452cf1
HDFS-16907. Add LastHeartbeatResponseTime for BP service actor ( #5349 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-07 09:54:40 +08:00
0ae075a2af
HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance ( #5147 )
2023-02-05 09:33:05 +08:00
bce388fd3f
HDFS-16902 Add Namenode status to BPServiceActor metrics and improve logging in offerservice ( #5334 )
...
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-03 09:11:02 +08:00
a70f84098f
HADOOP-18601. Fix build failure with docs profile. ( #5331 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2023-01-31 19:44:19 +09:00
88c8ac750d
HDFS-16888. BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout should be volatile ( #5296 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-01-31 17:46:38 +09:00
b677d40ab5
HADOOP-18602. Remove netty3 dependency
2023-01-27 16:32:50 +01:00
72b760130a
HDFS-16893. Standardize the usage of DFSClient debug log ( #5303 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-01-19 07:56:41 +08:00
04f3573f6a
HDFS-16891 Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel ( #5300 )
...
Reviewed-by: Stephen O'Donnell <sodonnell@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-18 13:13:41 -08:00
4de31123ce
Fix "the the" and friends typos ( #5267 )
...
Signed-off-by: Nikita Eshkeev <neshkeev@yandex.ru>
2023-01-17 03:33:59 +08:00
1263e024b9
HDFS-16887 Log start and end of phase/step in startup progress ( #5292 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-12 14:26:52 -08:00
f26d8bc9bd
HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. ( #5290 )
2023-01-11 19:46:51 +00:00
e3b09b7512
HDFS-16884. Fix TestFsDatasetImpl#testConcurrentWriteAndDeleteBlock failed ( #5280 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-01-11 09:12:26 +08:00
4cf304de45
HDFS-16872. Fix log throttling by declaring LogThrottlingHelper as static members ( #5246 )
...
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-01-10 10:03:25 -08:00
f3cff032e6
HDFS-16885. Fix TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed ( #5283 )
2023-01-10 09:02:56 +08:00
b56d483258
HDFS-16876: Changes cleanup of shared RouterStateIdContext to be driven by namenodeResolver data. ( #5282 )
2023-01-09 23:55:13 +00:00
cd19da1309
HDFS-16877: Enables state context for namenode in TestObserverWithRouter ( #5257 )
2023-01-07 00:18:35 +00:00
5022003e0f
HDFS-16881. Warn if AccessControlEnforcer runs for a long time to check permission. ( #5268 )
2023-01-05 09:31:52 +08:00
4511c360b9
HDFS-16883. Duplicate field name in hdfs-default.xml ( #5271 ). Contributed by YUBI LEE.
2023-01-05 04:58:43 +05:30
35ce60eadd
HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks ( #5264 )
2023-01-04 11:38:32 +08:00
Viraj Jasani
Stephen O'Donnell
slfan1989
zhangshuyan
ZanderXu
Tom
rdingankar
Simbarashe Dzinamarira
Owen O'Malley
hchaverr
zhtttylz
hfutatzhanghb
Zita Dombi
GuoPhilipse
Steve Vaughan
huhaiyang
He Xiaoqiao
hchaverri
sunhao
Masatake Iwasaki
Szilard Nemeth
Nikita Eshkeev
Chengbing Liu
Tsz-Wo Nicholas Sze
Yubi Lee