9c989515ba
HDFS-17037. Consider nonDfsUsed when running balancer. ( #5715 ). Contributed by Shuyan Zhang.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-06-09 16:19:08 +08:00
0e6bd09ae3
HDFS-17003. Erasure Coding: invalidate wrong block after reporting bad blocks from datanode ( #5643 ). Contributed by hfutatzhanghb.
...
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-06-08 18:06:51 +08:00
ddae78b0ec
HDFS-17035. FsVolumeImpl#getActualNonDfsUsed may return negative value. ( #5708 ). Contributed by farmmamba.
...
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-08 14:28:01 +05:30
0c209961f8
HDFS-17019. Optimize the logic for reconfigure slow peer enable for Namenode" ( #5671 )
...
* HDFS-17019. Optimize the logic for reconfigure slow peer enable for Namenode
2023-06-08 10:05:49 +08:00
d9980ab40f
HDFS-17029. Support getECPolices API in WebHDFS ( #5698 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 17:33:37 +05:30
5d6ca13c5c
HDFS-16983. Fix concat operation doesn't honor dfs.permissions.enabled ( #5561 ). Contributed by caozhiqiang.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 16:42:59 +05:30
2243cfd225
HDFS-17028. RBF: Optimize debug logs of class ConnectionPool and other related class. ( #5694 ). Contributed by farmmamba.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 14:34:46 +05:30
1d0c9ab433
Revert "HADOOP-18207. Introduce hadoop-logging module ( #5503 )"
...
This reverts commit 03a499821c
2023-06-05 09:34:40 +05:30
03a499821c
HADOOP-18207. Introduce hadoop-logging module ( #5503 )
...
Reviewed-by: Duo Zhang <zhangduo@apache.org>
2023-06-02 18:07:34 -07:00
9f1e23cc67
HDFS-17031. Reduce some repeated codes in RouterRpcServer. ( #5701 ). Contributed by Chengwei Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:32:38 +05:30
f8b7ddf69c
HDFS-16996. Fix flaky testFsCloseAfterClusterShutdown in TestFileCreation ( #5697 ). Contributed by Nishtha Shah.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:23:48 +05:30
d92a5815f4
HDFS-17027. RBF: Adds auto-msync support for clients connecting to routers. ( #5693 )
2023-05-31 10:20:19 -07:00
9acf462d26
HDFS-17000. Fix faulty loop condition in TestDFSStripedOutputStreamUpdatePipeline ( #5699 ). Contributed by Marcono1234.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-31 16:10:07 +05:30
124313d215
HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency. ( #5691 ). Contributed by hchaverri.
...
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-31 14:26:31 +08:00
e9740cb17a
HDFS-16908. Fix javadoc of field IncrementalBlockReportManager#readyToSend. ( #5351 ). Contributed by farmmamba.
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-23 16:12:50 +05:30
3b65b5d68f
HDFS-17020. RBF: mount table addAll should print failed records in std error ( #5674 )
2023-05-22 18:39:52 -07:00
5b22dc6ace
HDFS-16909. Improve ReplicaMap#mergeAll method. ( #5353 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-21 19:08:58 +08:00
af933f3a4f
HDFS-17017. Fix the issue of arguments number limit in report command in DFSAdmin ( #5667 ). Contributed by Haiyang Hu.
...
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-21 09:03:34 +05:30
9a524ede87
HDFS-17022. Fix the exception message to print the Identifier pattern ( #5678 ). Contributed by Nishtha Shah.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:40:23 +05:30
408dbf318e
HDFS-17014. HttpFS Add Support getStatus API ( #5660 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 14:37:38 +05:30
5272ed8670
HADOOP-17518. Update the regex to A-Z ( #5669 ). Contributed by Nishtha Shah.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-20 06:21:13 +05:30
0914b3e792
HDFS-16697. Add logs if resources are not available in NameNodeResourcePolicy. ( #5569 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 18:38:01 +05:30
339bc7b3a6
HDFS-16653. Improve error messages in ShortCircuitCache. ( #5568 ). Contributed by ECFuzz.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 07:43:18 +05:30
441fb23293
HDFS-17018. Improve dfsclient log format. ( #5668 ). Contributed by Xianming Lei.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 06:25:43 +05:30
f6770dee47
HDFS-16979. RBF: Add proxyuser port in hdfsauditlog ( #5552 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:02:16 +05:30
78cc528739
HDFS-17012.Remove unused DFSConfigKeys#DFS_DATANODE_PMEM_CACHE_DIRS_DEFAULT. ( #5659 ). Contributed by JiangHua Zhu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 14:27:19 +05:30
350dcaf616
HDFS-17010. Add a subtree test to TestSnapshotDiffReport. ( #5656 )
2023-05-18 15:53:26 +08:00
03163f9de2
HDFS-17011. Fix the metric of "HttpPort" at DataNodeInfo ( #5657 ). Contributed by Zhaohui Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-18 12:12:11 +05:30
8e17385141
HDFS-17009. RBF: state store putAll should also return failed records ( #5664 )
2023-05-17 09:33:34 -07:00
482897a0f6
Fix typos in HDFS documents. ( #5665 )
2023-05-17 09:28:01 -07:00
251439d769
HDFS-16985. Fix data missing issue when delete local block file. ( #5564 ). Contributed by Chengwei Wang.
...
Reviewed-by: Shuyan Zhang <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-14 21:33:38 +08:00
5d0cc455f5
HDFS-17008. Fix RBF JDK 11 javadoc warnings ( #5648 )
2023-05-12 17:27:13 -07:00
0c77629849
HDFS-17001. Support getStatus API in WebHDFS ( #5628 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 03:49:13 +05:30
fe61d8f073
HDFS-16978. RBF: Admin command to support bulk add of mount points ( #5554 ). Contributed by Viraj Jasani.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-11 08:45:34 +05:30
5084e881ef
HDFS-16990. HttpFS Add Support getFileLinkStatus API ( #5602 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 23:10:19 +05:30
a2dda0ce03
HADOOP-18359. Update commons-cli from 1.2 to 1.5. ( #5095 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-10 01:42:12 +05:30
03bf8f982a
HDFS-16999. Fix wrong use of processFirstBlockReport(). ( #5622 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-09 10:05:28 +08:00
4ee92efb73
HDFS-16865. The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. ( #5200 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-06 08:50:21 +05:30
ceb8878d4f
HDFS-16998. RBF: Add ops metrics for getSlowDatanodeReport in RouterClientActivity ( #5615 )
2023-05-04 09:45:40 -07:00
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 )
...
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
2023-05-03 11:05:55 +01:00
1079890ae3
HDFS-16707. RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus ( #4665 ). Contributed by Jiale Qi.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-01 21:22:26 +05:30
1a2cd965a7
HDFS-16897. Fix abundant Broken pipe exception in BlockSender ( #5329 ). Contributed by fanluo.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-29 17:24:56 +05:30
0e63152218
HDFS-16995. Remove unused parameters at NameNodeHttpServer#initWebHdfs ( #5601 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-28 05:25:01 +05:30
d9576bb9ee
HDFS-16972. Delete a snapshot may deleteCurrentFile. ( #5532 )
2023-04-27 09:17:47 -07:00
5af0845076
HDFS-16672. Fix lease interval comparison in BlockReportLeaseManager ( #4598 ). Contributed by dzcxzl.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org
2023-04-26 02:11:11 +05:30
6aac6cb212
HADOOP-18660. Filesystem Spelling Mistake ( #5475 ). Contributed by Sebastian Baunsgaard.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 21:44:04 +05:30
c9e0af9961
HDFS-16981. Support getFileLinkStatus API in WebHDFS ( #5572 ). Contributed by Hualong Zhang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 03:30:56 +05:30
dc78849f27
HDFS-16975. FileWithSnapshotFeature.isCurrentFileDeleted is not reloaded from FSImage. ( #5546 )
2023-04-24 09:04:28 -07:00
6a23c376c9
HDFS-16986. EC: Fix locationBudget in getListing(). ( #5582 ). Contributed by Shuyan Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-04-24 18:53:25 +08:00
51dcbd1d61
HDFS-16988. Improve NameServices info at JournalNode web UI ( #5584 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-24 01:13:02 +05:30
zhangshuyan
hfutatzhanghb
huhaiyang
zhtttylz
caozhiqiang
Ayush Saxena
Viraj Jasani
smarthan
NishthaShah
Simbarashe Dzinamarira
Marcono1234
hchaverri
hfutatzhanghb
Keyao Li
Xianming Lei
LiuGuH
jianghuazhu
Tsz-Wo Nicholas Sze
wangzhaohui
liang3zy22
slfan1989
ZanderXu
Tak Lon (Stephen) Wu
SevenAddSix
fanluoo
cxzl25
Sebastian Baunsgaard