2243cfd225
HDFS-17028. RBF: Optimize debug logs of class ConnectionPool and other related class. ( #5694 ). Contributed by farmmamba.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-05 14:34:46 +05:30
1d0c9ab433
Revert "HADOOP-18207. Introduce hadoop-logging module ( #5503 )"
...
This reverts commit 03a499821c
2023-06-05 09:34:40 +05:30
03a499821c
HADOOP-18207. Introduce hadoop-logging module ( #5503 )
...
Reviewed-by: Duo Zhang <zhangduo@apache.org>
2023-06-02 18:07:34 -07:00
9f1e23cc67
HDFS-17031. Reduce some repeated codes in RouterRpcServer. ( #5701 ). Contributed by Chengwei Wang.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-06-01 08:32:38 +05:30
d92a5815f4
HDFS-17027. RBF: Adds auto-msync support for clients connecting to routers. ( #5693 )
2023-05-31 10:20:19 -07:00
124313d215
HDFS-17026. RBF: NamenodeHeartbeatService should update JMX report with configurable frequency. ( #5691 ). Contributed by hchaverri.
...
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-05-31 14:26:31 +08:00
3b65b5d68f
HDFS-17020. RBF: mount table addAll should print failed records in std error ( #5674 )
2023-05-22 18:39:52 -07:00
f6770dee47
HDFS-16979. RBF: Add proxyuser port in hdfsauditlog ( #5552 ). Contributed by liuguanghua.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-19 05:02:16 +05:30
8e17385141
HDFS-17009. RBF: state store putAll should also return failed records ( #5664 )
2023-05-17 09:33:34 -07:00
5d0cc455f5
HDFS-17008. Fix RBF JDK 11 javadoc warnings ( #5648 )
2023-05-12 17:27:13 -07:00
0c77629849
HDFS-17001. Support getStatus API in WebHDFS ( #5628 ). Contributed by Hualong Zhang.
...
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-13 03:49:13 +05:30
fe61d8f073
HDFS-16978. RBF: Admin command to support bulk add of mount points ( #5554 ). Contributed by Viraj Jasani.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-11 08:45:34 +05:30
4ee92efb73
HDFS-16865. The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. ( #5200 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-06 08:50:21 +05:30
ceb8878d4f
HDFS-16998. RBF: Add ops metrics for getSlowDatanodeReport in RouterClientActivity ( #5615 )
2023-05-04 09:45:40 -07:00
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 )
...
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
2023-05-03 11:05:55 +01:00
1079890ae3
HDFS-16707. RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus ( #4665 ). Contributed by Jiale Qi.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-01 21:22:26 +05:30
0e63152218
HDFS-16995. Remove unused parameters at NameNodeHttpServer#initWebHdfs ( #5601 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-28 05:25:01 +05:30
c9e0af9961
HDFS-16981. Support getFileLinkStatus API in WebHDFS ( #5572 ). Contributed by Hualong Zhang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 03:30:56 +05:30
1ff7a65b9f
HDFS-16954. RBF: The operation of renaming a multi-subcluster directory to a single-cluster directory should throw ioexception. ( #5483 ). Contributed by Max Xie.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 15:19:18 +05:30
523ff81624
HDFS-16952. Support getLinkTarget API in WebHDFS ( #5517 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-06 19:44:47 +08:00
47c22e388e
HDFS-16943. RBF: Implements MySQL based StateStoreDriver. ( #5469 )
2023-04-05 16:44:29 -07:00
422bf3b24c
HDFS-16973. RBF: MountTableResolver cache size lookup should take read lock ( #5533 )
2023-04-05 14:06:38 -07:00
937caf7de9
HDFS-16967. RBF: File based state stores should allow concurrent access to the records ( #5523 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-04-04 22:39:53 +09:00
b4bcbb9515
HDFS-16959. RBF: State store cache loading metrics ( #5497 )
2023-03-29 10:43:13 -07:00
f8d0949f7d
HDFS-16953. RBF: Mount table store APIs should update cache only if state store record is successfully updated ( #5482 )
2023-03-18 14:43:25 -07:00
15935fa865
HDFS-16947. RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store ( #5470 )
2023-03-16 00:59:55 +08:00
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
8025a60ae7
HDFS-16901: Minor fix for unit test.
...
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 16:37:49 -08:00
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
4cbe19f3a2
HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down ( #5390 )
2023-02-15 16:16:04 +00:00
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
f3c4277576
HDFS-16882. RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath ( #5276 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-11 08:00:43 +08:00
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders ( #5301 )
...
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
2023-02-09 13:28:34 +00:00
d310642626
HDFS-16895. [RBF] NamenodeHeartbeatService should use credentials of logged in user
2023-02-07 18:45:05 +00:00
0ae075a2af
HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance ( #5147 )
2023-02-05 09:33:05 +08:00
f26d8bc9bd
HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. ( #5290 )
2023-01-11 19:46:51 +00:00
b56d483258
HDFS-16876: Changes cleanup of shared RouterStateIdContext to be driven by namenodeResolver data. ( #5282 )
2023-01-09 23:55:13 +00:00
cd19da1309
HDFS-16877: Enables state context for namenode in TestObserverWithRouter ( #5257 )
2023-01-07 00:18:35 +00:00
d25c1be517
HDFS-16861. RBF. Truncate API always fails when dirs use AllResolver oder on Router ( #5184 )
...
Co-authored-by: xiezhineng <xiezhineng@corp.netease.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-12-26 21:19:33 +08:00
df093ef9af
HDFS-16831. [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time ( #5098 )
2022-12-23 14:16:42 +08:00
8a9bdb1edc
HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces ( #5123 )
2022-12-05 16:15:47 +08:00
03471a736c
HDFS-16851: RBF: Add a utility to dump the StateStore. ( #5155 )
2022-11-29 22:12:35 +00:00
909aeca86c
HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider. ( #5142 )
2022-11-29 00:49:10 +00:00
ec2856d79c
HDFS-16847: RBF: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException. ( #5145 )
2022-11-29 00:47:01 +00:00
e0974298ce
HDFS-16826. [RBF SBN] ConnectionManager should advance the client stateId for each request ( #5086 )
2022-11-25 09:23:33 +08:00
c71a68ca80
HDFS-16844: Adds resilancy when StateStore gets exceptions. ( #5138 )
...
Allows the StateStore to stay up when there are errors reading the data.
2022-11-18 17:24:08 +00:00
552ee44eba
HDFS-16834: Removes request stateID consistency constraint between clients in different connection pools. ( #5121 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-11-11 15:26:31 +08:00
44b8bb7224
HDFS-16821: Fixes regression in HDFS-13522 that enables observer reads by default ( #5078 )
2022-11-07 08:43:04 -08:00
hfutatzhanghb
Ayush Saxena
Viraj Jasani
smarthan
Simbarashe Dzinamarira
hchaverri
LiuGuH
zhtttylz
ZanderXu
Tak Lon (Stephen) Wu
SevenAddSix
wangzhaohui
Neil
Owen O'Malley
hchaverr
Zita Dombi
hfutatzhanghb
huhaiyang
sunhao