9c989515ba
HDFS-17037. Consider nonDfsUsed when running balancer. ( #5715 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-06-09 16:19:08 +08:00
0e6bd09ae3
HDFS-17003. Erasure Coding: invalidate wrong block after reporting bad blocks from datanode ( #5643 ). Contributed by hfutatzhanghb.
...
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-06-08 18:06:51 +08:00
ddae78b0ec
HDFS-17035. FsVolumeImpl#getActualNonDfsUsed may return negative value. ( #5708 ). Contributed by farmmamba.
...
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-08 14:28:01 +05:30
0c209961f8
HDFS-17019. Optimize the logic for reconfigure slow peer enable for Namenode" ( #5671 )
...
* HDFS-17019. Optimize the logic for reconfigure slow peer enable for Namenode
2023-06-08 10:05:49 +08:00
d9980ab40f
HDFS-17029. Support getECPolices API in WebHDFS ( #5698 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 17:33:37 +05:30
5d6ca13c5c
HDFS-16983. Fix concat operation doesn't honor dfs.permissions.enabled ( #5561 ). Contributed by caozhiqiang.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 16:42:59 +05:30
1d0c9ab433
Revert "HADOOP-18207. Introduce hadoop-logging module ( #5503 )"
...
This reverts commit 03a499821c
2023-06-05 09:34:40 +05:30
03a499821c
HADOOP-18207. Introduce hadoop-logging module ( #5503 )
...
Reviewed-by: Duo Zhang <zhangduo@apache.org>
2023-06-02 18:07:34 -07:00
f8b7ddf69c
HDFS-16996. Fix flaky testFsCloseAfterClusterShutdown in TestFileCreation ( #5697 ). Contributed by Nishtha Shah.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:23:48 +05:30
9acf462d26
HDFS-17000. Fix faulty loop condition in TestDFSStripedOutputStreamUpdatePipeline ( #5699 ). Contributed by Marcono1234.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-31 16:10:07 +05:30
e9740cb17a
HDFS-16908. Fix javadoc of field IncrementalBlockReportManager#readyToSend. ( #5351 ). Contributed by farmmamba.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-23 16:12:50 +05:30
5b22dc6ace
HDFS-16909. Improve ReplicaMap#mergeAll method. ( #5353 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-21 19:08:58 +08:00
af933f3a4f
HDFS-17017. Fix the issue of arguments number limit in report command in DFSAdmin ( #5667 ). Contributed by Haiyang Hu.
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-21 09:03:34 +05:30
0914b3e792
HDFS-16697. Add logs if resources are not available in NameNodeResourcePolicy. ( #5569 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 18:38:01 +05:30
339bc7b3a6
HDFS-16653. Improve error messages in ShortCircuitCache. ( #5568 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 07:43:18 +05:30
f6770dee47
HDFS-16979. RBF: Add proxyuser port in hdfsauditlog ( #5552 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:02:16 +05:30
78cc528739
HDFS-17012.Remove unused DFSConfigKeys#DFS_DATANODE_PMEM_CACHE_DIRS_DEFAULT. ( #5659 ). Contributed by JiangHua Zhu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 14:27:19 +05:30
350dcaf616
HDFS-17010. Add a subtree test to TestSnapshotDiffReport. ( #5656 )
2023-05-18 15:53:26 +08:00
03163f9de2
HDFS-17011. Fix the metric of "HttpPort" at DataNodeInfo ( #5657 ). Contributed by Zhaohui Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 12:12:11 +05:30
482897a0f6
Fix typos in HDFS documents. ( #5665 )
2023-05-17 09:28:01 -07:00
251439d769
HDFS-16985. Fix data missing issue when delete local block file. ( #5564 ). Contributed by Chengwei Wang.
...
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-14 21:33:38 +08:00
0c77629849
HDFS-17001. Support getStatus API in WebHDFS ( #5628 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 03:49:13 +05:30
a2dda0ce03
HADOOP-18359. Update commons-cli from 1.2 to 1.5. ( #5095 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 01:42:12 +05:30
03bf8f982a
HDFS-16999. Fix wrong use of processFirstBlockReport(). ( #5622 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-09 10:05:28 +08:00
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 )
...
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
2023-05-03 11:05:55 +01:00
1a2cd965a7
HDFS-16897. Fix abundant Broken pipe exception in BlockSender ( #5329 ). Contributed by fanluo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-29 17:24:56 +05:30
0e63152218
HDFS-16995. Remove unused parameters at NameNodeHttpServer#initWebHdfs ( #5601 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-28 05:25:01 +05:30
d9576bb9ee
HDFS-16972. Delete a snapshot may deleteCurrentFile. ( #5532 )
2023-04-27 09:17:47 -07:00
5af0845076
HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager ( #4598 ). Contributed by dzcxzl.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
2023-04-26 02:11:11 +05:30
c9e0af9961
HDFS-16981. Support getFileLinkStatus API in WebHDFS ( #5572 ). Contributed by Hualong Zhang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 03:30:56 +05:30
dc78849f27
HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage. ( #5546 )
2023-04-24 09:04:28 -07:00
6a23c376c9
HDFS-16986. EC: Fix locationBudget in getListing(). ( #5582 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-04-24 18:53:25 +08:00
51dcbd1d61
HDFS-16988. Improve NameServices info at JournalNode web UI ( #5584 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-24 01:13:02 +05:30
405ed1dde6
HADOOP-18470. Hadoop 3.3.5 release wrap-up ( #5558 )
...
Post-release updates of the branches
* Add jdiff xml files from 3.3.5 release.
* Declare 3.3.5 as the latest stable release.
* Copy release notes.
2023-04-18 10:12:07 +01:00
0185afafea
HDFS-16974. Consider volumes average load of each DataNode when choosing target. ( #5541 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-04-14 10:33:30 +08:00
3e2ae1da00
HDFS-16949 Introduce inverse quantiles for metrics where higher numer… ( #5495 )
2023-04-10 08:56:00 -07:00
523ff81624
HDFS-16952. Support getLinkTarget API in WebHDFS ( #5517 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-06 19:44:47 +08:00
811441d5bc
HDFS-16951. Add description of GETSERVERDEFAULTS to WebHDFS doc ( #5491 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-01 18:18:20 +08:00
700147b4ac
HDFS-16964. Improve processing of excess redundancy after failover. ( #5510 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-28 16:14:59 +08:00
5cf62d1498
HDFS-16948. Update log of BlockManager#chooseExcessRedundancyStriped when EC internal block is moved by balancer. ( #5474 ). Contributed by Kidd53685368.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-23 17:00:23 +08:00
e3cb9573e1
HADOOP-18662. ListFiles with recursive fails with FNF. ( #5477 ). Contributed by Ayush Saxena.
...
Reviewed-by: Steve Loughran <stevel@apache.org
2023-03-23 08:30:08 +05:30
0dbe1d3284
HADOOP-18668. Path capability probe for truncate is only honored by RawLocalFileSystem ( #5492 )
2023-03-21 10:23:16 +08:00
b6a9d7b442
HADOOP-18631. (ADDENDUM) Use LogCapturer to match audit log pattern and remove hdfs async audit log configs ( #5451 )
2023-03-18 06:33:50 +08:00
eee2ea075d
HDFS-16942. Addendum. Send error to datanode if FBR is rejected due to bad lease ( #5478 ). Contributed by Stephen O'Donnell/
2023-03-15 10:03:00 +05:30
ca6f5afb6d
HDFS-16942. Send error to datanode if FBR is rejected due to bad lease ( #5460 )
2023-03-11 16:40:07 +00:00
927401886a
HDFS-16934. TestDFSAdmin.testAllDatanodesReconfig regression ( #5434 )
...
Contributed by Shilun Fan
2023-03-06 15:26:53 +00:00
2cb0c35fc1
HDFS-16939. Fix the thread safety bug in LowRedundancyBlocks. ( #5450 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-03-06 20:10:31 +08:00
6bd2444815
HDFS-16923. [SBN read] getlisting RPC to observer will throw NPE if path does not exist ( #5400 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-03-01 16:18:38 -08:00
162288bc0a
HDFS-16896 clear ignoredNodes list when we clear deadnode list on ref… ( #5322 )
...
HDFS-16896 clear ignoredNodes list when we clear deadnode list on refetchLocations.
ignoredNodes list is only used on hedged read codepath
Co-authored-by: Tom McCormick <tmccormi@linkedin.com>
2023-03-01 19:47:04 +00:00
2ab7eb4caa
HDFS-16935. Fix TestFsDatasetImpl#testReportBadBlocks ( #5432 )
...
Contributed by Viraj Jasani
2023-03-01 18:53:10 +00:00
zhangshuyan
hfutatzhanghb
huhaiyang
zhtttylz
caozhiqiang
Ayush Saxena
Viraj Jasani
NishthaShah
Marcono1234
hfutatzhanghb
Keyao Li
LiuGuH
jianghuazhu
Tsz-Wo Nicholas Sze
wangzhaohui
liang3zy22
smarthan
slfan1989
Tak Lon (Stephen) Wu
fanluoo
cxzl25
Steve Loughran
rdingankar
Kidd5368
Stephen O'Donnell
ZanderXu
Tom