3f573b6751
HDFS-12920. HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop. ( #3227 )
...
Revert "HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin"
This reverts commit b6d839a60c2f2f822488
2021-07-26 01:54:32 +09:00
3faae2e06d
HDFS-15650. Make the socket timeout for computing checksum of striped blocks configurable ( #2414 )
...
(cherry picked from commit 4bb25c810b
2021-07-16 11:07:27 +08:00
0f2b89b791
HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein.
2021-07-15 20:47:02 +00:00
450dae7383
HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das ( #2260 )
...
(cherry picked from commit 1dd03cc4b5
2021-07-13 18:23:27 -07:00
c18ad9a007
HADOOP-17793. Better token validation ( #3189 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada
2021-07-10 13:43:31 +09:00
705884b87a
HDFS-15796. ConcurrentModificationException error happens on NameNode occasionally. Contributed by Daniel Ma
...
(cherry picked from commit b6c06c4b76
2021-07-09 10:19:47 +01:00
ab939814a3
HDFS-16121. Iterative snapshot diff report can generate duplicate records for creates, deletes and Renames. ( #3188 )
...
(cherry picked from commit 6450c1ea9a
2021-07-09 09:17:10 +01:00
2d56335a3d
HDFS-16108. Fix incorrect log placeholders used in JournalNodeSyncer ( #3169 )
...
(cherry picked from commit da575a53cf
2021-07-05 13:42:51 +09:00
aec8c9188a
HDFS-16109. Fix flaky some unit tests since they offen timeout ( #3172 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b87bac13e4
2021-07-05 08:15:10 +09:00
bf6ed21104
HDFS-16090. Fine grained lock for datanodeNetworkCounts ( #3148 )
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack Chuang <stack@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-30 12:59:25 -07:00
4ee7bd345f
HDFS-16089. Add metric EcReconstructionValidateTimeMillis for StripedBlockReconstructor ( #3146 )
...
(cherry picked from commit 95454d821c
2021-06-29 18:15:56 +08:00
8538881710
HDFS-16092. Avoid creating LayoutFlags redundant objects ( #3150 )
...
(cherry picked from commit 93a1685073
2021-06-29 18:13:00 +08:00
4ca49fcd44
HDFS-16085. Move the getPermissionChecker out of the read lock ( #3134 ). Contributed by tomscut.
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-24 16:00:47 +05:30
5fc61d2e56
HDFS-16076. Avoid using slow DataNodes for reading by sorting locations ( #3117 )
...
(cherry picked from commit fdef2b4cca
2021-06-24 13:27:33 +09:00
68604b4cd0
HDFS-16082. Atomic operations on exceptionsSinceLastBalance and failedTimesSinceLastSuccessfulBalance in Balancer ( #3127 )
...
(cherry picked from commit d9fbb3c508
2021-06-23 11:24:00 +08:00
b8a98e4f82
HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects ( #3115 )
...
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit c488abbc79
2021-06-21 10:28:05 +09:00
7d202e8365
HDFS-16079. Improve the block state change log ( #3120 ). Contributed by tomscut.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:24:03 +05:30
92ade1f6f9
HDFS-16061. DFTestUtil.waitReplication can produce false positives ( #3095 ). Contributed by Ahmed Hussein.
...
Reviewed-by: Jim Brennan <jbrennan@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:26:49 +05:30
8d241482bd
HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeline() ( #3119 ). Contributed by tomscut.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:26:38 +05:30
e55d76e26c
HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet ( #3113 )
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
2021-06-18 14:41:02 +08:00
840612b3b1
HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false ( #2443 ). Contributed by Ahmed Hussein.
...
(cherry picked from commit 747883af4d
2021-06-14 14:37:27 +00:00
52d055a7eb
HDFS-16057. Make sure the order for location in ENTERING_MAINTENANCE state ( #3084 )
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 51ebf7b2a0
2021-06-11 17:33:06 +09:00
02249171b1
HDFS-16048. RBF: Print network topology on the router web ( #3062 )
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c748fce17a
2021-06-08 16:59:59 +09:00
46d4b51bff
HDFS-15814. Make some parameters configurable for DataNodeDiskMetrics for branch-3.3 ( #3021 )
2021-06-08 00:10:37 -07:00
ede03cc35c
HDFS-15998. Fix NullPointException In listOpenFiles ( #3036 )
...
Co-authored-by: huhaiyang <huhaiyang@didichuxing.com>
(cherry picked from commit b38b00e528
2021-06-01 15:30:50 +08:00
c5535caf6e
HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira.
...
(cherry picked from commit 8ce30f51f9
2021-05-27 18:53:46 -07:00
5308d44be4
HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit 1abd03d68f
2021-05-26 15:09:47 -07:00
41025cd585
HDFS-16031. Possible Resource Leak in org.apache.hadoop.hdfs.server.aliasmap#InMemoryAliasMap ( #3027 )
...
(cherry picked from commit 3fdeb7435a
2021-05-26 14:45:20 +09:00
86c28f0639
Revert "HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )"
...
This reverts commit 4ffe5eb1dd
2021-05-24 17:37:18 +08:00
fa4915fdbb
Preparing for 3.3.2 development
2021-05-19 21:52:37 +08:00
63392376c9
HDFS-16027. Replace abstract methods with default methods in JournalNodeMXBean. ( #3018 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-05-18 19:49:00 -07:00
4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 ( #2959 )
...
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb
* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661 ). Contributed by Vinayakumar B.
(cherry picked from commit c4c0683dff
2021-05-17 18:57:46 -07:00
2f56e2b54d
HDFS-15873. Add namenode address in logs for block report ( #2743 )
...
(cherry picked from commit 6a922f9896
2021-05-17 12:29:11 +08:00
28079e9c30
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 ( #2330 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 486ddb73f9
2021-05-13 12:16:17 +08:00
41b39ff292
HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang.
...
(cherry picked from commit 960c9ebaea
2021-05-13 12:14:09 +08:00
3176ab028b
HDFS-16007. Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException ( #2982 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b944084b32
2021-05-11 12:39:42 +09:00
43db94c7a3
HDFS-16003. ProcessReport print invalidatedBlocks should judge debug level at first. Contributed by lei w.
2021-05-10 17:11:09 +05:30
1326cad66a
HDFS-16001. TestOfflineEditsViewer.testStored() fails reading negative value of FSEditsLogOpCodes (branch-3.3) ( #2989 ). Contributed by Akira Ajisaka.
2021-05-08 10:48:56 +08:00
670205c541
HADOOP-17653. Do not use guava's Files.createTempDir(). ( #2945 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f1e1809029
2021-05-02 11:12:37 +09:00
9aa6106689
HDFS-15652. Make block size from NNThroughputBenchmark configurable ( #2416 )
...
(cherry picked from commit 8a6d5b9151
2021-04-29 12:10:04 -07:00
0e9042fed5
HDFS-15991. Add location into datanode info for NameNodeMXBean ( #2933 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit b968fa0957
2021-04-28 18:52:05 +09:00
842e05ba53
HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 ( #2495 )
...
(cherry picked from commit 68442b48c8
2021-04-28 18:51:50 +09:00
f809b863fe
HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
...
(cherry picked from commit 6e416a83d1
2021-04-28 18:51:43 +09:00
187b4a63a4
HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.
...
(cherry picked from commit 8db38c98a6
2021-04-27 23:18:12 +09:00
f45365f201
HDFS-15975. Use LongAdder instead of AtomicLong for branch-3.3 ( #2940 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-27 22:39:25 +09:00
f6efb58b07
HDFS-15621. Datanode DirectoryScanner uses excessive memory ( #2849 ). Contributed by Stephen O'Donnell
...
(cherry picked from commit 605ed85c29
2021-04-26 11:23:56 +01:00
a5f038b3f6
HDFS-15967. Improve the log for Short Circuit Local Reads. Contributed by Bhavik Patel.
...
(cherry picked from commit 01bad0e92a
2021-04-26 13:31:51 +09:00
99dfd3b2d0
HDFS-15566. NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0. Contributed by Brahma Reddy Battula.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 2621d3f15b
2021-04-26 11:40:16 +08:00
96a1dfa313
HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime() ( #2942 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-23 12:57:23 +09:00
9f2db2c9fd
HDFS-15963. Unreleased volume references cause an infinite loop. ( #2941 ) Contributed by Shuyan Zhang.
...
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-04-21 18:32:56 +08:00
Akira Ajisaka
crossfire
Jim Brennan
Abhishek Das
Artem Smotrakov
S O'Donnell
bshashikant
Viraj Jasani
litao
Ahmed Hussein
Hui Fei
huhaiyang
Konstantin V Shvachko
Narges Shadab
Wei-Chiu Chuang
Aryan Gupta
Ayush Saxena
Stephen O'Donnell
Takanobu Asanuma
Brahma Reddy Battula
zhangshuyan0