Commit Graph

8431 Commits

Author SHA1 Message Date
wangzhaohui 7068e18842
HDFS-16154. TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely (#3270)
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit e85c44657c)
2021-08-06 18:25:00 +08:00
wangzhaohui e9ba4f4591 HDFS-16153. Avoid evaluation of LOG.debug statement in QuorumJournalManager (#3269). Contributed by wangzhaohui.
(cherry picked from commit a73b64f86b)
2021-08-06 09:28:39 +01:00
Siyao Meng 72508e6430 HDFS-16055. Quota is not preserved in snapshot INode (#3078)
(cherry picked from commit ebee2aed00)
2021-08-03 20:24:18 +01:00
He Xiaoqiao 797c595ba5
HDFS-15175. Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog. Contributed by Wan Chang.
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
(cherry picked from commit 683feaa1d4)
2021-07-29 19:04:08 +08:00
jianghuazhu 43f15578fc
HDFS-15936.Solve SocketTimeoutException#sendPacket() does not record SocketTimeout exception. (#2836)
Co-authored-by: zhujianghua <zhujianghua@zhujianghuadeMacBook-Pro.local>
Reviewed-by: cxorm <lianp964@gmail.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit 6f730fd25c)
2021-07-29 15:44:02 +08:00
S O'Donnell d661afc06f HDFS-16144. Revert HDFS-15372 (Files in snapshots no longer see attribute provider permissions). Contributed by Stephen O'Donnell
(cherry picked from commit 4eae284827)
2021-07-28 15:02:16 +01:00
Akira Ajisaka 9e72b697a2
HDFS-16140. TestBootstrapAliasmap fails by BindException. (#3229)
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit d710ec8d85)
2021-07-26 16:47:26 +09:00
Akira Ajisaka 3f573b6751 HDFS-12920. HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop. (#3227)
Revert "HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin"

This reverts commit b6d839a60c.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 2f2f822488)
2021-07-26 01:54:32 +09:00
crossfire 3faae2e06d
HDFS-15650. Make the socket timeout for computing checksum of striped blocks configurable (#2414)
(cherry picked from commit 4bb25c810b)
2021-07-16 11:07:27 +08:00
Jim Brennan 0f2b89b791 HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein. 2021-07-15 20:47:02 +00:00
Abhishek Das 450dae7383 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260)
(cherry picked from commit 1dd03cc4b5)
2021-07-13 18:23:27 -07:00
Artem Smotrakov c18ad9a007 HADOOP-17793. Better token validation (#3189)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada)
2021-07-10 13:43:31 +09:00
S O'Donnell 705884b87a HDFS-15796. ConcurrentModificationException error happens on NameNode occasionally. Contributed by Daniel Ma
(cherry picked from commit b6c06c4b76)
2021-07-09 10:19:47 +01:00
bshashikant ab939814a3 HDFS-16121. Iterative snapshot diff report can generate duplicate records for creates, deletes and Renames. (#3188)
(cherry picked from commit 6450c1ea9a)
2021-07-09 09:17:10 +01:00
Viraj Jasani 2d56335a3d
HDFS-16108. Fix incorrect log placeholders used in JournalNodeSyncer (#3169)
(cherry picked from commit da575a53cf)
2021-07-05 13:42:51 +09:00
litao aec8c9188a
HDFS-16109. Fix flaky some unit tests since they offen timeout (#3172)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b87bac13e4)
2021-07-05 08:15:10 +09:00
Viraj Jasani bf6ed21104 HDFS-16090. Fine grained lock for datanodeNetworkCounts (#3148)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
    Signed-off-by: stack Chuang <stack@apache.org>
    Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-06-30 12:59:25 -07:00
litao 4ee7bd345f
HDFS-16089. Add metric EcReconstructionValidateTimeMillis for StripedBlockReconstructor (#3146)
(cherry picked from commit 95454d821c)
2021-06-29 18:15:56 +08:00
Viraj Jasani 8538881710
HDFS-16092. Avoid creating LayoutFlags redundant objects (#3150)
(cherry picked from commit 93a1685073)
2021-06-29 18:13:00 +08:00
litao 4ca49fcd44 HDFS-16085. Move the getPermissionChecker out of the read lock (#3134). Contributed by tomscut.
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-24 16:00:47 +05:30
litao 5fc61d2e56 HDFS-16076. Avoid using slow DataNodes for reading by sorting locations (#3117)
(cherry picked from commit fdef2b4cca)
2021-06-24 13:27:33 +09:00
Viraj Jasani 68604b4cd0
HDFS-16082. Atomic operations on exceptionsSinceLastBalance and failedTimesSinceLastSuccessfulBalance in Balancer (#3127)
(cherry picked from commit d9fbb3c508)
2021-06-23 11:24:00 +08:00
Viraj Jasani b8a98e4f82 HDFS-16075. Use empty array constants present in StorageType and DatanodeInfo to avoid creating redundant objects (#3115)
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit c488abbc79)
2021-06-21 10:28:05 +09:00
litao 7d202e8365 HDFS-16079. Improve the block state change log (#3120). Contributed by tomscut.
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 17:24:03 +05:30
Ahmed Hussein 92ade1f6f9 HDFS-16061. DFTestUtil.waitReplication can produce false positives (#3095). Contributed by Ahmed Hussein.
Reviewed-by: Jim Brennan <jbrennan@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:26:49 +05:30
litao 8d241482bd HDFS-16078. Remove unused parameters for DatanodeManager.handleLifeline() (#3119). Contributed by tomscut.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-06-20 10:26:38 +05:30
Hui Fei e55d76e26c
HDFS-13671. Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet (#3113)
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
2021-06-18 14:41:02 +08:00
Ahmed Hussein 840612b3b1 HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false (#2443). Contributed by Ahmed Hussein.
(cherry picked from commit 747883af4d)
2021-06-14 14:37:27 +00:00
litao 52d055a7eb HDFS-16057. Make sure the order for location in ENTERING_MAINTENANCE state (#3084)
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 51ebf7b2a0)
2021-06-11 17:33:06 +09:00
litao 02249171b1 HDFS-16048. RBF: Print network topology on the router web (#3062)
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c748fce17a)
2021-06-08 16:59:59 +09:00
litao 46d4b51bff
HDFS-15814. Make some parameters configurable for DataNodeDiskMetrics for branch-3.3 (#3021) 2021-06-08 00:10:37 -07:00
huhaiyang ede03cc35c
HDFS-15998. Fix NullPointException In listOpenFiles (#3036)
Co-authored-by: huhaiyang <huhaiyang@didichuxing.com>
(cherry picked from commit b38b00e528)
2021-06-01 15:30:50 +08:00
Konstantin V Shvachko c5535caf6e HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira.
(cherry picked from commit 8ce30f51f9)
2021-05-27 18:53:46 -07:00
Konstantin V Shvachko 5308d44be4 HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko.
(cherry picked from commit 1abd03d68f)
2021-05-26 15:09:47 -07:00
Narges Shadab 41025cd585
HDFS-16031. Possible Resource Leak in org.apache.hadoop.hdfs.server.aliasmap#InMemoryAliasMap (#3027)
(cherry picked from commit 3fdeb7435a)
2021-05-26 14:45:20 +09:00
Wei-Chiu Chuang 86c28f0639
Revert "HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 (#2959)"
This reverts commit 4ffe5eb1dd.
2021-05-24 17:37:18 +08:00
Wei-Chiu Chuang fa4915fdbb
Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
Wei-Chiu Chuang 63392376c9
HDFS-16027. Replace abstract methods with default methods in JournalNodeMXBean. (#3018)
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-05-18 19:49:00 -07:00
Wei-Chiu Chuang 4ffe5eb1dd
HADOOP-17669. Backport HADOOP-17079, HADOOP-17505 to branch-3.3 (#2959)
* HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet.

Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Change-Id: I0f31409923ece24a82dfba4c4610d8a38c52d9fb

* HADOOP-17505. public interface GroupMappingServiceProvider needs default impl for getGroupsSet() (#2661). Contributed by Vinayakumar B.

(cherry picked from commit c4c0683dff)

Co-authored-by: Xiaoyu Yao <xyao@apache.org>
Co-authored-by: Vinayakumar B <vinayakumarb@apache.org>
2021-05-17 18:57:46 -07:00
litao 2f56e2b54d
HDFS-15873. Add namenode address in logs for block report (#2743)
(cherry picked from commit 6a922f9896)
2021-05-17 12:29:11 +08:00
Aryan Gupta 28079e9c30
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 486ddb73f9)
2021-05-13 12:16:17 +08:00
Ayush Saxena 41b39ff292
HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang.
(cherry picked from commit 960c9ebaea)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNode.java
2021-05-13 12:14:09 +08:00
Viraj Jasani 3176ab028b
HDFS-16007. Deserialization of ReplicaState should avoid throwing ArrayIndexOutOfBoundsException (#2982)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b944084b32)
2021-05-11 12:39:42 +09:00
Ayush Saxena 43db94c7a3 HDFS-16003. ProcessReport print invalidatedBlocks should judge debug level at first. Contributed by lei w. 2021-05-10 17:11:09 +05:30
Akira Ajisaka 1326cad66a
HDFS-16001. TestOfflineEditsViewer.testStored() fails reading negative value of FSEditsLogOpCodes (branch-3.3) (#2989). Contributed by Akira Ajisaka. 2021-05-08 10:48:56 +08:00
Wei-Chiu Chuang 670205c541
HADOOP-17653. Do not use guava's Files.createTempDir(). (#2945)
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f1e1809029)
2021-05-02 11:12:37 +09:00
Hui Fei 9aa6106689 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416)
(cherry picked from commit 8a6d5b9151)
2021-04-29 12:10:04 -07:00
litao 0e9042fed5 HDFS-15991. Add location into datanode info for NameNodeMXBean (#2933)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit b968fa0957)
2021-04-28 18:52:05 +09:00
Hui Fei 842e05ba53 HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 (#2495)
(cherry picked from commit 68442b48c8)
2021-04-28 18:51:50 +09:00
Ayush Saxena f809b863fe HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
(cherry picked from commit 6e416a83d1)
2021-04-28 18:51:43 +09:00
Ayush Saxena 187b4a63a4 HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.
(cherry picked from commit 8db38c98a6)
2021-04-27 23:18:12 +09:00
litao f45365f201
HDFS-15975. Use LongAdder instead of AtomicLong for branch-3.3 (#2940)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-27 22:39:25 +09:00
Stephen O'Donnell f6efb58b07 HDFS-15621. Datanode DirectoryScanner uses excessive memory (#2849). Contributed by Stephen O'Donnell
(cherry picked from commit 605ed85c29)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
2021-04-26 11:23:56 +01:00
Takanobu Asanuma a5f038b3f6 HDFS-15967. Improve the log for Short Circuit Local Reads. Contributed by Bhavik Patel.
(cherry picked from commit 01bad0e92a)
2021-04-26 13:31:51 +09:00
Brahma Reddy Battula 99dfd3b2d0 HDFS-15566. NN restart fails after RollingUpgrade from 3.1.3/3.2.1 to 3.3.0. Contributed by Brahma Reddy Battula.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 2621d3f15b)
2021-04-26 11:40:16 +08:00
Viraj Jasani 96a1dfa313
HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime() (#2942)
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-23 12:57:23 +09:00
zhangshuyan0 9f2db2c9fd
HDFS-15963. Unreleased volume references cause an infinite loop. (#2941) Contributed by Shuyan Zhang.
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-04-21 18:32:56 +08:00
litao 29414871bd
HDFS-15879. Exclude slow nodes when choose targets for blocks (#2928) 2021-04-21 10:41:13 +09:00
hemanthboyina 90f63cb12c HDFS-15569. Speed up the Storage#doRecover during datanode rolling upgrade. Contributed by Hemanth Boyina.
(cherry picked from commit 16a20503ca)
2021-04-20 16:21:57 +01:00
Ayush Saxena c93a2f6789 HDFS-14999. Avoid Potential Infinite Loop in DFSNetworkTopology. Contributed by Ayush Saxena.
(cherry picked from commit c84e6beada)
2021-04-20 12:13:39 +01:00
litao dc830cf277 HDFS-15970. Print network topology on the web (#2896)
(cherry picked from commit 85a3532849)
2021-04-19 21:22:47 +09:00
Hui Fei 4c0b833bab HDFS-15898. Test case TestOfflineImageViewer fails (#2779)
(cherry picked from commit 0d42104119)
2021-04-16 14:07:15 +09:00
Wei-Chiu Chuang bfba6f1f3c
HDFS-15815. if required storageType are unavailable, log the failed reason during choosing Datanode. Contributed by Yang Yun. (#2882)
(cherry picked from commit e391844e8e)

Co-authored-by: Ayush Saxena <ayushsaxena@apache.org>
2021-04-12 23:55:32 -07:00
Stephen O'Donnell 0e6cd352b5
HDFS-15160. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock. Contributed by Stephen O'Donnell. (#2813)
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 2a67e2b1a0)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DirectoryScanner.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
2021-04-09 10:25:14 +01:00
Viraj Jasani 4994b73eeb HDFS-15940. Fix TestBlockRecovery2#testRaceBetweenReplicaRecoveryAndFinalizeBlock (ADDENDUM) (#2874)
(cherry picked from commit 56bd968fb4)
2021-04-09 11:13:39 +09:00
He Xiaoqiao df99ac0399 HDFS-15942. Increase Quota initialization threads. Contributed by Stephen O'Donnell.
(cherry picked from commit 459df41997)
2021-04-08 13:15:37 +01:00
Stephen O'Donnell ef95f7a963 HDFS-15937. Reduce memory used during datanode layout upgrade. Contributed by Stephen O'Donnell (#2838)
(cherry picked from commit 4c567fcff7)
2021-04-08 12:15:31 +01:00
touchida dca2bf9dd5 HDFS-15759. EC: Verify EC reconstruction correctness on DataNode (#2585)
(cherry picked from commit 95e6892675)
2021-04-08 17:20:08 +08:00
Viraj Jasani 8b4b3d6fe6 HADOOP-17622. Avoid usage of deprecated IOUtils#cleanup API. (#2862)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 3f2682b92b)
2021-04-06 14:18:31 +09:00
Viraj Jasani 2b5fd341b9 HDFS-15940. Fixing and refactoring tests specific to Block recovery.(#2844). Contributed by Viraj Jasani
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2021-04-06 00:20:56 +05:30
Inigo Goiri d973f37054 HDFS-15362. FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks. Contributed by hemanthboyina.
(cherry picked from commit 2148a8fe64)
2021-04-02 17:14:34 +08:00
Inigo Goiri 887948d127 HDFS-15351. Blocks scheduled count was wrong on truncate. Contributed by hemanthboyina.
(cherry picked from commit 719b53a79d)
2021-04-02 17:14:24 +08:00
Surendra Singh Lilhore 02d37c7d22 HDFS-15316. Deletion failure should not remove directory from snapshottables. Contributed by hemanthboyina
(cherry picked from commit 743c2e9071)
2021-04-02 17:14:14 +08:00
Inigo Goiri 57d2fff84b HDFS-15246. ArrayIndexOfboundsException in BlockManager CreateLocatedBlock. Contributed by Hemanth Boyina.
(cherry picked from commit 8b7695bb26)
2021-04-02 14:06:52 +08:00
Inigo Goiri 1148d3895f HDFS-15332. Quota Space consumed was wrong in truncate with Snapshots. Contributed by hemanthboyina.
(cherry picked from commit 130f89e068)
2021-04-02 14:06:52 +08:00
Ayush Saxena 5187bd37ae HDFS-15266. Add missing DFSOps Statistics in WebHDFS. Contributed by Ayush Saxena.
(cherry picked from commit 37d6582223)
2021-04-02 14:06:52 +08:00
maobaolong 2b207ea402 HDFS-15667. Audit log record the unexpected allowed result when delete (#2437)
(cherry picked from commit 95c96605b3)
2021-04-02 14:06:52 +08:00
litao 62937d15bb HDFS-15892. Add metric for editPendingQ in FSEditLogAsync (#2770)
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 4bd04126d6)
2021-04-02 10:57:05 +09:00
Viraj Jasani f707d8407a
HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2021-04-01 16:35:44 -07:00
He Xiaoqiao e5d7b22b8b
HDFS-15921. Improve the log for the Storage Policy Operations. Contributed by Bhavik Patel. 2021-04-01 11:59:53 +08:00
Brahma Reddy Battula 13878fc06b HDFS-15494. TestReplicaCachingGetSpaceUsed#testReplicaCachingGetSpaceUsedByRBWReplica Fails on Windows. Contributed by Ravuri Sushma sree.
(cherry picked from commit 0665ce9930)
2021-04-01 09:22:18 +05:30
He Xiaoqiao ab451b8e42 HDFS-15651. Client could not obtain block when DN CommandProcessingThread exit. Contributed by Aiphago.
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Yiqun Lin <yqlin@apache.org>
(cherry picked from commit 3067a25fa1)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
2021-03-30 22:29:49 +08:00
Akira Ajisaka c4f7b8578b HDFS-15506. [JDK 11] Fix javadoc errors in hadoop-hdfs module. Contributed by Xieming Li.
(cherry picked from commit 7938ebfb9d)
2021-03-30 22:29:49 +08:00
Karthik Palanisamy 769df1e964 HDFS-15253 Default checkpoint transfer speed, 50mb per second (#2366)
(cherry picked from commit df4006eb81)
2021-03-30 22:29:49 +08:00
Ayush Saxena 7f20fad419 HDFS-14383. Compute datanode load based on StoragePolicy. Contributed by Ayush Saxena.
(cherry picked from commit 2e8cafac3b)
2021-03-30 22:29:48 +08:00
Stephen O'Donnell 56ef16468a
HADOOP-17222. Create socket address leveraging URI cache (#2817)
Contributed by fanrui.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-03-30 11:59:44 +01:00
Viraj Jasani d6aaf516d9 HDFS-15926 : Removed duplicate dependency of hadoop-annotations (#2823). Contributed by Viraj Jasani.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-03-29 23:25:14 +05:30
leosunli b3e968d21e HDFS-13639. SlotReleaser is not fast enough (#1885)
(cherry picked from commit be374faf42)
2021-03-25 13:25:17 +00:00
He Xiaoqiao bf018e7237
HDFS-15919. BlockPoolManager should log stack trace if unable to get Namenode addresses. Contributed by Stephen O'Donnell. 2021-03-25 12:25:22 +08:00
LeonGao91 4db5643ff9 HDFS-15456. TestExternalStoragePolicySatisfier fails intermittently (#2299)
(cherry picked from commit de055ec8aa)
2021-03-25 05:09:11 +08:00
Inigo Goiri 813aafcee2 HDFS-12288. Fix DataNode's xceiver count calculation. Contributed by Lisheng Sun.
(cherry picked from commit 6e04b00df1)
2021-03-24 17:44:29 +08:00
Ayush Saxena 580634882e HDFS-14546. Document block placement policies. Contributed by Amithsha.
(cherry picked from commit 17ffcab5f6)
2021-03-24 17:44:29 +08:00
Karthik Palanisamy c96821ece8 HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6 (#2365)
(cherry picked from commit 735e85a1d2)
2021-03-24 17:44:29 +08:00
sunlisheng a8c0083b1b HDFS-15809. DeadNodeDetector does not remove live nodes from dead node set. Contributed by Jinglun.
(cherry picked from commit 7025f39944)
2021-03-24 14:04:47 +08:00
Ayush Saxena 94766fdb13 HDFS-15806. DeadNodeDetector should close all the threads when it is closed. Contributed by Jinglun.
(cherry picked from commit ff84a57483)
2021-03-24 14:04:47 +08:00
sunlisheng 3ecd02ac78 HDFS-15661. The DeadNodeDetector should not be shared by different DFSClients. Contributed by Jinglun.
(cherry picked from commit f8769e0f4b)
2021-03-24 14:04:47 +08:00
Narges Shadab f07bde90c9 HDFS-15791. Possible Resource Leak in FSImageFormatProtobuf. (#2652)
(cherry picked from commit 115623a6ee)
2021-03-24 14:04:47 +08:00
Narges Shadab 3a355282c9 HDFS-15868. Possible Resource Leak in EditLogFileOutputStream (#2736). Contributed by Narges Shadab.
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit f5f3fc6fe9)
2021-03-24 14:04:47 +08:00
Narges Shadab 4abe6e6ac1 HDFS-15908. Possible Resource Leak in org.apache.hadoop.hdfs.qjournal.server.Journal (#2790). Contributed by Narges Shadab.
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit d77c7ab4e0)
2021-03-24 14:04:47 +08:00
Toshihiro Suzuki e9b06d942c HDFS-15249 ThrottledAsyncChecker is not thread-safe. (#1922)
(cherry picked from commit c12ddbd1de)
2021-03-24 14:04:47 +08:00