a70f84098f
HADOOP-18601. Fix build failure with docs profile. ( #5331 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2023-01-31 19:44:19 +09:00
88c8ac750d
HDFS-16888. BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout should be volatile ( #5296 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-01-31 17:46:38 +09:00
b677d40ab5
HADOOP-18602. Remove netty3 dependency
2023-01-27 16:32:50 +01:00
72b760130a
HDFS-16893. Standardize the usage of DFSClient debug log ( #5303 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-01-19 07:56:41 +08:00
04f3573f6a
HDFS-16891 Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel ( #5300 )
...
Reviewed-by: Stephen O'Donnell <sodonnell@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-18 13:13:41 -08:00
4de31123ce
Fix "the the" and friends typos ( #5267 )
...
Signed-off-by: Nikita Eshkeev <neshkeev@yandex.ru>
2023-01-17 03:33:59 +08:00
1263e024b9
HDFS-16887 Log start and end of phase/step in startup progress ( #5292 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-12 14:26:52 -08:00
f26d8bc9bd
HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. ( #5290 )
2023-01-11 19:46:51 +00:00
e3b09b7512
HDFS-16884. Fix TestFsDatasetImpl#testConcurrentWriteAndDeleteBlock failed ( #5280 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-01-11 09:12:26 +08:00
4cf304de45
HDFS-16872. Fix log throttling by declaring LogThrottlingHelper as static members ( #5246 )
...
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-01-10 10:03:25 -08:00
f3cff032e6
HDFS-16885. Fix TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed ( #5283 )
2023-01-10 09:02:56 +08:00
b56d483258
HDFS-16876: Changes cleanup of shared RouterStateIdContext to be driven by namenodeResolver data. ( #5282 )
2023-01-09 23:55:13 +00:00
cd19da1309
HDFS-16877: Enables state context for namenode in TestObserverWithRouter ( #5257 )
2023-01-07 00:18:35 +00:00
5022003e0f
HDFS-16881. Warn if AccessControlEnforcer runs for a long time to check permission. ( #5268 )
2023-01-05 09:31:52 +08:00
4511c360b9
HDFS-16883. Duplicate field name in hdfs-default.xml ( #5271 ). Contributed by YUBI LEE.
2023-01-05 04:58:43 +05:30
35ce60eadd
HDFS-16879. EC: Fsck -blockId shows number of redundant internal block replicas for EC Blocks ( #5264 )
2023-01-04 11:38:32 +08:00
d25c1be517
HDFS-16861. RBF. Truncate API always fails when dirs use AllResolver oder on Router ( #5184 )
...
Co-authored-by: xiezhineng <xiezhineng@corp.netease.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-12-26 21:19:33 +08:00
df093ef9af
HDFS-16831. [RBF SBN] GetNamenodesForNameserviceId should shuffle Observer NameNodes every time ( #5098 )
2022-12-23 14:16:42 +08:00
15b52fb6a4
HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer ( #4744 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-12-21 10:06:01 -08:00
8d221255f2
HDFS-16764. [SBN Read] ObserverNamenode should throw ObserverRetryOnActiveException instead of FileNotFoundException during processing of addBlock rpc ( #4872 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-12-20 15:50:58 -08:00
7ff326129d
HDFS-16871 DiskBalancer process may throw IllegalArgumentException when the target DataNode has capital letter in hostname ( #5240 )
2022-12-21 07:12:02 +08:00
f7bdf6c667
HDFS-16852. Skip KeyProviderCache shutdown hook registration if already shutting down ( #5160 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-12-16 08:46:14 -08:00
c5b42d59d2
HDFS-16866. Fix a typo in Dispatcher ( #5202 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2022-12-16 11:07:41 +08:00
fdcbc8b072
HDFS-16868. Fix audit log duplicate issue when an ACE occurs in FSNamesystem. ( #5206 ). Contributed by Beibei Zhao.
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-12-13 12:24:51 +08:00
e76616f690
HDFS-16860 Upgrade moment.min.js to 2.29.4 ( #5194 )
2022-12-09 11:18:44 +05:30
2fa540dca1
HDFS-16858. Dynamically adjust max slow disks to exclude. ( #5180 )
...
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-12-09 08:10:04 +08:00
8a9bdb1edc
HDFS-16837. [RBF SBN] ClientGSIContext should merge RouterFederatedStates to get the max state id for each namespaces ( #5123 )
2022-12-05 16:15:47 +08:00
02afb9ebe1
HDFS-16809. EC striped block is not sufficient when doing in maintenance. ( #5050 )
2022-12-05 16:34:51 +09:00
2067fcb646
HDFS-16550. Allow JN edit cache size to be set as a fraction of heap memory ( #4209 )
2022-11-30 07:44:21 -08:00
72749a4ff8
HDFS-16839 It should consider EC reconstruction work when we determine if a node is busy ( #5128 )
...
Co-authored-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Tao Li <tomscut@apache.org>
2022-11-30 09:43:15 +08:00
03471a736c
HDFS-16851: RBF: Add a utility to dump the StateStore. ( #5155 )
2022-11-29 22:12:35 +00:00
35c65005d0
HDFS-16846. EC: Only EC blocks should be effected by max-streams-hard-limit configuration ( #5143 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-11-29 10:51:21 +09:00
909aeca86c
HDFS-16845: Adds configuration flag to allow clients to use router observer reads without using the ObserverReadProxyProvider. ( #5142 )
2022-11-29 00:49:10 +00:00
ec2856d79c
HDFS-16847: RBF: Prevents StateStoreFileSystemImpl from committing tmp file after encountering an IOException. ( #5145 )
2022-11-29 00:47:01 +00:00
87429f443a
HDFS-16779. Add ErasureCodingPolicy information to the response description for GETFILESTATUS in WebHDFS.md ( #4922 )
2022-11-25 09:26:28 +08:00
e0974298ce
HDFS-16826. [RBF SBN] ConnectionManager should advance the client stateId for each request ( #5086 )
2022-11-25 09:23:33 +08:00
ef84d21867
HDFS-16841. Enhance the function of DebugAdmin#VerifyECCommand ( #5137 )
2022-11-24 09:17:27 +08:00
bcc3d2a20e
HDFS-16838. Fix NPE in testAddRplicaProcessorForAddingReplicaInMap ( #5125 )
...
Reviewed-by: Xing Lin <xinglin@linkedin.com>
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-11-24 09:07:27 +08:00
dfa9edacce
HDFS-16840. Enhance the usage description about oiv in HDFSCommands.md and OfflineImageViewerPB ( #5129 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: ZanderXu <zanderxu@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2022-11-24 09:00:27 +08:00
ac958777af
HDFS-16813. Remove parameter validation logic such as dfs.namenode.decommission.blocks.per.interval in DatanodeAdminManager#activate ( #5063 )
2022-11-23 09:50:57 +08:00
8f971b0e54
HDFS-16547. [SBN read] Namenode in safe mode should not be transfer to observer state ( #4201 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Zengqiang Xu <xuzq_zander@163.com>
2022-11-21 10:14:07 -08:00
dc2fba45fe
HDFS-16832. [SBN READ] Follow-on to HDFS-16732. Fix NPE when check the block location of empty directory ( #5099 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Zengqiang Xu <xuzq_zander@163.com>
2022-11-21 08:26:16 -08:00
c71a68ca80
HDFS-16844: Adds resilancy when StateStore gets exceptions. ( #5138 )
...
Allows the StateStore to stay up when there are errors reading the data.
2022-11-18 17:24:08 +00:00
cd929457c9
HDFS-16836: StandbyCheckpointer shouldn't trigger rollback fs image after RU is finalized ( #5135 )
...
Co-authored-by: Lei Yang <leyang@linkedin.com>
2022-11-15 23:06:37 +00:00
d3c1c453f0
HDFS-16785. Avoid to hold write lock to improve performance when add volume. ( #4945 ). Contributed by ZanderXu.
...
Signed-off-by: Tao Li <tomscut@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-11-13 22:01:35 +08:00
d340c4a7a1
HADOOP-18496. Upgrade okhttp3 and dependencies due to kotlin CVEs ( #5035 )
...
Updates okhttp3 and okio so their transitive dependency on Kotlin
stdlib is free from recent CVEs.
okhttp3:okhttp => 4.10.0
okio:okio => 3.2.0
kotlin stdlib => 1.6.20
kotlin CVEs fixed:
CVE-2022-24329
CVE-2020-29582
Contributed by PJ Fanning.
2022-11-12 14:14:19 +00:00
552ee44eba
HDFS-16834: Removes request stateID consistency constraint between clients in different connection pools. ( #5121 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Zander Xu <zanderxu@apache.org>
2022-11-11 15:26:31 +08:00
44b8bb7224
HDFS-16821: Fixes regression in HDFS-13522 that enables observer reads by default ( #5078 )
2022-11-07 08:43:04 -08:00
660530205e
HDFS-16833. NameNode should log internal EC blocks instead of the EC block group when it receives block reports. ( #5106 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
2022-11-07 10:49:48 +09:00
e9319e696c
HDFS-16811. Support DecommissionBackoffMonitor Parameters reconfigurablereconfigurable ( #5068 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
2022-11-04 14:18:59 +08:00
Masatake Iwasaki
huhaiyang
Szilard Nemeth
zhtttylz
Viraj Jasani
Nikita Eshkeev
Simbarashe Dzinamarira
Chengbing Liu
Tsz-Wo Nicholas Sze
Yubi Lee
Neil
ZanderXu
Daniel-009497
Xing Lin
Happy-shi
curie71
Anurag P
dingshun3016
litao
Kidd5368
Owen O'Malley
caozhiqiang
zhengchenyu
Lei Yang
ZanderXu
PJ Fanning
Takanobu Asanuma