4ee92efb73
HDFS-16865. The source path is always / after RBF proxied the complete, addBlock and getAdditionalDatanode RPC. ( #5200 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-06 08:50:21 +05:30
ceb8878d4f
HDFS-16998. RBF: Add ops metrics for getSlowDatanodeReport in RouterClientActivity ( #5615 )
2023-05-04 09:45:40 -07:00
0e46388474
HADOOP-18671. Add recoverLease(), setSafeMode(), isFileClosed() as interfaces to hadoop-common ( #5553 )
...
The HDFS lease APIs have been replicated as interfaces in hadoop-common so other filesystems can
also implement them. Applications which use the leasing APIs should migrate to the new
interface where possible.
Contributed by Stephen Wu
2023-05-03 11:05:55 +01:00
1079890ae3
HDFS-16707. RBF: Expose RouterRpcFairnessPolicyController related request record metrics for each nameservice to Prometheus ( #4665 ). Contributed by Jiale Qi.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-05-01 21:22:26 +05:30
0e63152218
HDFS-16995. Remove unused parameters at NameNodeHttpServer#initWebHdfs ( #5601 ). Contributed by Zhaohui Wang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-28 05:25:01 +05:30
c9e0af9961
HDFS-16981. Support getFileLinkStatus API in WebHDFS ( #5572 ). Contributed by Hualong Zhang.
...
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-25 03:30:56 +05:30
1ff7a65b9f
HDFS-16954. RBF: The operation of renaming a multi-subcluster directory to a single-cluster directory should throw ioexception. ( #5483 ). Contributed by Max Xie.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2023-04-20 15:19:18 +05:30
523ff81624
HDFS-16952. Support getLinkTarget API in WebHDFS ( #5517 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-04-06 19:44:47 +08:00
47c22e388e
HDFS-16943. RBF: Implements MySQL based StateStoreDriver. ( #5469 )
2023-04-05 16:44:29 -07:00
422bf3b24c
HDFS-16973. RBF: MountTableResolver cache size lookup should take read lock ( #5533 )
2023-04-05 14:06:38 -07:00
937caf7de9
HDFS-16967. RBF: File based state stores should allow concurrent access to the records ( #5523 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-04-04 22:39:53 +09:00
b4bcbb9515
HDFS-16959. RBF: State store cache loading metrics ( #5497 )
2023-03-29 10:43:13 -07:00
f8d0949f7d
HDFS-16953. RBF: Mount table store APIs should update cache only if state store record is successfully updated ( #5482 )
2023-03-18 14:43:25 -07:00
15935fa865
HDFS-16947. RBF NamenodeHeartbeatService to report error for not being able to register namenode in state store ( #5470 )
2023-03-16 00:59:55 +08:00
28d2753d2f
HADOOP-18645. Provide keytab file key name with ServiceStateException ( #5433 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2023-03-01 09:34:12 +08:00
61f369c43e
HDFS-16890: RBF: Ensures router periodically refreshes its record of a namespace's state. ( #5298 )
2023-02-27 17:56:24 +00:00
8025a60ae7
HDFS-16901: Minor fix for unit test.
...
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 16:37:49 -08:00
4cc33e5e37
HDFS-16901: RBF: Propagates real user's username via the caller context, when a proxy user is being used. ( #5346 )
2023-02-22 21:58:44 +00:00
fb31393b65
HADOOP-18535. Implement token storage solution based on MySQL
...
Fixes #1240
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2023-02-22 10:38:50 -08:00
4cbe19f3a2
HDFS-16761. Namenode UI for Datanodes page not loading if any data node is down ( #5390 )
2023-02-15 16:16:04 +00:00
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
f3c4277576
HDFS-16882. RBF: Add cache hit rate metric in MountTableResolver#getDestinationForPath ( #5276 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-11 08:00:43 +08:00
113a9e40cb
HADOOP-18625. Fix method name of RPC.Builder#setnumReaders ( #5301 )
...
Changes method name of RPC.Builder#setnumReaders to setNumReaders()
The original method is still there, just marked deprecated.
It is the one which should be used when working with older branches.
Contributed by Haiyang Hu
2023-02-09 13:28:34 +00:00
d310642626
HDFS-16895. [RBF] NamenodeHeartbeatService should use credentials of logged in user
2023-02-07 18:45:05 +00:00
0ae075a2af
HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance ( #5147 )
2023-02-05 09:33:05 +08:00
f26d8bc9bd
HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. ( #5290 )
2023-01-11 19:46:51 +00:00
b56d483258
HDFS-16876: Changes cleanup of shared RouterStateIdContext to be driven by namenodeResolver data. ( #5282 )
2023-01-09 23:55:13 +00:00
cd19da1309
HDFS-16877: Enables state context for namenode in TestObserverWithRouter ( #5257 )
2023-01-07 00:18:35 +00:00
d25c1be517
HDFS-16861. RBF. Truncate API always fails when dirs use AllResolver oder on Router ( #5184 )
...
Co-authored-by: xiezhineng <xiezhineng@corp.netease.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-12-26 21:19:33 +08:00
df093ef9af
HDFS-16831. [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time ( #5098 )
2022-12-23 14:16:42 +08:00
8a9bdb1edc
HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces ( #5123 )
2022-12-05 16:15:47 +08:00
03471a736c
HDFS-16851: RBF: Add a utility to dump the StateStore. ( #5155 )
2022-11-29 22:12:35 +00:00
909aeca86c
HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider. ( #5142 )
2022-11-29 00:49:10 +00:00
ec2856d79c
HDFS-16847: RBF: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException. ( #5145 )
2022-11-29 00:47:01 +00:00
e0974298ce
HDFS-16826. [RBF SBN] ConnectionManager should advance the client stateId for each request ( #5086 )
2022-11-25 09:23:33 +08:00
c71a68ca80
HDFS-16844: Adds resilancy when StateStore gets exceptions. ( #5138 )
...
Allows the StateStore to stay up when there are errors reading the data.
2022-11-18 17:24:08 +00:00
552ee44eba
HDFS-16834: Removes request stateID consistency constraint between clients in different connection pools. ( #5121 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-11-11 15:26:31 +08:00
44b8bb7224
HDFS-16821: Fixes regression in HDFS-13522 that enables observer reads by default ( #5078 )
2022-11-07 08:43:04 -08:00
c5c00f3d2c
HDFS-16803.Improve some annotations in hdfs module. ( #5031 )
2022-10-20 10:58:23 +08:00
6422eaf301
HDFS-16767: RBF: Support observer node in Router-Based Federation.
...
Fixes #4127
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-14 16:00:14 -07:00
8ce71c8882
[Documentation] RBF: Duplicate statement to be removed for better readabilty ( #4881 )
2022-09-13 09:24:05 -07:00
e77d54d1ee
HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients.
...
Fixes #4311
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-09 15:43:28 -07:00
4a01fadb94
HDFS-16756. RBF proxies the client's user by the login user to enable CacheEntry ( #4853 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-09 12:47:36 +05:30
be4c638e4c
HDFS-16748. RBF: DFSClient should uniquely identify writing files by namespace id and iNodeId via RBF ( #4813 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-05 17:46:15 +05:30
5567154f71
HDFS-16734. RBF: fix some bugs when handling getContentSummary RPC ( #4763 )
2022-08-26 16:04:33 -07:00
8d4f51c432
HDFS-16728. RBF throw IndexOutOfBoundsException with disableNameServices ( #4734 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-24 20:27:15 +05:30
c37f01d95b
HDFS-16724. RBF should support get the information about ancestor mount points ( #4719 )
2022-08-23 13:25:42 -07:00
a3b1bafa34
HDFS-16669: Enhance client protocol to propagate last seen state IDs for multiple nameservices.
...
Fixes #4584
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-08-23 11:12:50 -07:00
7f176d080c
HDFS-16729. RBF: fix some unreasonably annotated docs. ( #4745 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-21 07:29:31 +09:00
622ca0d51f
HDFS-16705. RBF supports healthMonitor timeout configurable and caching NN and client proxy in NamenodeHeartbeatService ( #4662 )
2022-08-15 13:55:16 -07:00
ZanderXu
Viraj Jasani
Tak Lon (Stephen) Wu
SevenAddSix
wangzhaohui
zhtttylz
Neil
Simbarashe Dzinamarira
Owen O'Malley
hchaverr
Zita Dombi
hfutatzhanghb
huhaiyang
hchaverri
sunhao
jianghuazhu
Renukaprasad C
ZanderXu