c0f32f3cf8
HDFS-16221. RBF: Add usage of refreshCallQueue for Router ( #3421 )
...
(cherry picked from commit 7f6553af75
2021-09-13 10:48:09 +08:00
8affaa6312
HDFS-16210. RBF: Add the option of refreshCallQueue to RouterAdmin ( #3379 )
...
(cherry picked from commit c0890e6d04
2021-09-10 10:01:27 +08:00
1f61944e3b
HDFS-16207. Remove NN logs stack trace for non-existent xattr query ( #3375 )
...
Change-Id: Ibde523b20a6b8ac92991da52583e625a018d2ee6
2021-09-09 05:27:13 +00:00
580b6c400b
HDFS-16192: ViewDistributedFileSystem#rename wrongly using src in the place of dst. ( #3353 )
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
(cherry picked from commit 164608b546
2021-08-31 12:27:43 +08:00
0967483b7c
HDFS-16184. De-flake TestBlockScanner#testSkipRecentAccessFile ( #3329 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 1b9927afe1
2021-08-25 17:42:56 +09:00
5a1ed37893
HDFS-16177. Bug fix for Util#receiveFile ( #3310 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 07627ef19e
2021-08-19 12:31:52 +08:00
00c382d118
Fix potential heap buffer overflow in hdfs.c. Contributed by Igor Chervatyuk.
...
(cherry picked from commit 4972e7a246
2021-08-07 11:27:52 +09:00
7068e18842
HDFS-16154. TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely ( #3270 )
...
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit e85c44657c
2021-08-06 18:25:00 +08:00
e9ba4f4591
HDFS-16153. Avoid evaluation of LOG.debug statement in QuorumJournalManager ( #3269 ). Contributed by wangzhaohui.
...
(cherry picked from commit a73b64f86b
2021-08-06 09:28:39 +01:00
72508e6430
HDFS-16055. Quota is not preserved in snapshot INode ( #3078 )
...
(cherry picked from commit ebee2aed00
2021-08-03 20:24:18 +01:00
797c595ba5
HDFS-15175. Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog. Contributed by Wan Chang.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
(cherry picked from commit 683feaa1d4
2021-07-29 19:04:08 +08:00
43f15578fc
HDFS-15936.Solve SocketTimeoutException#sendPacket() does not record SocketTimeout exception. ( #2836 )
...
Co-authored-by: zhujianghua <zhujianghua@zhujianghuadeMacBook-Pro.local>
Reviewed-by: cxorm <lianp964@gmail.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6f730fd25c
2021-07-29 15:44:02 +08:00
d661afc06f
HDFS-16144. Revert HDFS-15372 (Files in snapshots no longer see attribute provider permissions). Contributed by Stephen O'Donnell
...
(cherry picked from commit 4eae284827
2021-07-28 15:02:16 +01:00
9e72b697a2
HDFS-16140. TestBootstrapAliasmap fails by BindException. ( #3229 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit d710ec8d85
2021-07-26 16:47:26 +09:00
3f573b6751
HDFS-12920. HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop. ( #3227 )
...
Revert "HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin"
This reverts commit b6d839a60c2f2f822488
2021-07-26 01:54:32 +09:00
95a70c0fdf
HDFS-16127. Improper pipeline close recovery causes a permanent write failure or data loss. Contributed by Kihwal Lee.
...
(cherry picked from commit 47002719f2
2021-07-16 14:23:36 -05:00
3faae2e06d
HDFS-15650. Make the socket timeout for computing checksum of striped blocks configurable ( #2414 )
...
(cherry picked from commit 4bb25c810b
2021-07-16 11:07:27 +08:00
0f2b89b791
HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein.
2021-07-15 20:47:02 +00:00
450dae7383
HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das ( #2260 )
...
(cherry picked from commit 1dd03cc4b5
2021-07-13 18:23:27 -07:00
c18ad9a007
HADOOP-17793. Better token validation ( #3189 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada
2021-07-10 13:43:31 +09:00
705884b87a
HDFS-15796. ConcurrentModificationException error happens on NameNode occasionally. Contributed by Daniel Ma
...
(cherry picked from commit b6c06c4b76
2021-07-09 10:19:47 +01:00
ab939814a3
HDFS-16121. Iterative snapshot diff report can generate duplicate records for creates, deletes and Renames. ( #3188 )
...
(cherry picked from commit 6450c1ea9a
2021-07-09 09:17:10 +01:00
2d56335a3d
HDFS-16108. Fix incorrect log placeholders used in JournalNodeSyncer ( #3169 )
...
(cherry picked from commit da575a53cf
2021-07-05 13:42:51 +09:00
aec8c9188a
HDFS-16109. Fix flaky some unit tests since they offen timeout ( #3172 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b87bac13e4
2021-07-05 08:15:10 +09:00
bf6ed21104
HDFS-16090. Fine grained lock for datanodeNetworkCounts ( #3148 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack Chuang <stack@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-30 12:59:25 -07:00
4ee7bd345f
HDFS-16089. Add metric EcReconstructionValidateTimeMillis for StripedBlockReconstructor ( #3146 )
...
(cherry picked from commit 95454d821c
2021-06-29 18:15:56 +08:00
8538881710
HDFS-16092. Avoid creating LayoutFlags redundant objects ( #3150 )
...
(cherry picked from commit 93a1685073
2021-06-29 18:13:00 +08:00
4ca49fcd44
HDFS-16085. Move the getPermissionChecker out of the read lock ( #3134 ). Contributed by tomscut.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-24 16:00:47 +05:30
5fc61d2e56
HDFS-16076. Avoid using slow DataNodes for reading by sorting locations ( #3117 )
...
(cherry picked from commit fdef2b4cca
2021-06-24 13:27:33 +09:00
68604b4cd0
HDFS-16082. Atomic operations on exceptionsSinceLastBalance and failedTimesSinceLastSuccessfulBalance in Balancer ( #3127 )
...
(cherry picked from commit d9fbb3c508
2021-06-23 11:24:00 +08:00
b8a98e4f82
HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects ( #3115 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit c488abbc79
2021-06-21 10:28:05 +09:00
7d202e8365
HDFS-16079. Improve the block state change log ( #3120 ). Contributed by tomscut.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:24:03 +05:30
ef3a2a3b3b
HDFS-16080. RBF: Invoking method in all locations should break the loop after successful result ( #3121 ). Contributed by Viraj Jasani.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:23:52 +05:30
92ade1f6f9
HDFS-16061. DFTestUtil.waitReplication can produce false positives ( #3095 ). Contributed by Ahmed Hussein.
...
Reviewed-by: Jim Brennan <jbrennan@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:26:49 +05:30
8d241482bd
HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeline() ( #3119 ). Contributed by tomscut.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:26:38 +05:30
e55d76e26c
HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet ( #3113 )
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
2021-06-18 14:41:02 +08:00
577b96ef42
HDFS-16074. Remove an expensive debug string concatenation ( #3107 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Reviewed-by: litao <tomleescut@gmail.com>
(cherry picked from commit 1ad674e548
2021-06-17 11:39:34 +08:00
3e03ecd680
HDFS-16073. Remove redundant RPC requests for getFileLinkInfo in ClientNamenodeProtocolTranslatorPB. Contributed by lei w.
2021-06-16 15:54:36 +05:30
546c66b812
HDFS-16068. WebHdfsFileSystem has a possible connection leak in connection with HttpFS ( #3104 )
...
(cherry picked from commit 839fcf7682
2021-06-15 17:22:34 +09:00
840612b3b1
HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false ( #2443 ). Contributed by Ahmed Hussein.
...
(cherry picked from commit 747883af4d
2021-06-14 14:37:27 +00:00
52d055a7eb
HDFS-16057. Make sure the order for location in ENTERING_MAINTENANCE state ( #3084 )
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 51ebf7b2a0
2021-06-11 17:33:06 +09:00
83d119fe94
HDFS-15916. DistCp: Backward compatibility: Distcp fails from Hadoop 3 to Hadoop 2 for snapshotdiff. ( #2863 ). Contributed by Ayush Saxena. ( #3054 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c6539e3289
2021-06-08 19:32:34 -07:00
02249171b1
HDFS-16048. RBF: Print network topology on the router web ( #3062 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c748fce17a
2021-06-08 16:59:59 +09:00
46d4b51bff
HDFS-15814. Make some parameters configurable for DataNodeDiskMetrics for branch-3.3 ( #3021 )
2021-06-08 00:10:37 -07:00
ede03cc35c
HDFS-15998. Fix NullPointException In listOpenFiles ( #3036 )
...
Co-authored-by: huhaiyang <huhaiyang@didichuxing.com>
(cherry picked from commit b38b00e528
2021-06-01 15:30:50 +08:00
c5535caf6e
HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira.
...
(cherry picked from commit 8ce30f51f9
2021-05-27 18:53:46 -07:00
5308d44be4
HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit 1abd03d68f
2021-05-26 15:09:47 -07:00
41025cd585
HDFS-16031. Possible Resource Leak in org.apache.hadoop.hdfs.server.aliasmap#InMemoryAliasMap ( #3027 )
...
(cherry picked from commit 3fdeb7435a
2021-05-26 14:45:20 +09:00
86c28f0639
Revert "HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )"
...
This reverts commit 4ffe5eb1dd
2021-05-24 17:37:18 +08:00
ca52d9946b
HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu.
...
(cherry picked from commit da1ea2530f
2021-05-24 10:48:48 +08:00
Symious
Ahmed Hussein
Uma Maheswara Rao G
Viraj Jasani
litao
Akira Ajisaka
wangzhaohui
Siyao Meng
He Xiaoqiao
jianghuazhu
S O'Donnell
Kihwal Lee
crossfire
Jim Brennan
Abhishek Das
Artem Smotrakov
bshashikant
Hui Fei
Wei-Chiu Chuang
Ayush Saxena
Takanobu Asanuma
huhaiyang
Konstantin V Shvachko
Narges Shadab
Yiqun Lin