Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,445 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

5023 Commits

Author SHA1 Message Date
Abhishek Das 99b952fcb4 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260, #3218) 
(cherry picked from commit 1dd03cc4b5)
 2021-07-20 18:26:24 -07:00
Jim Brennan 39c8297350 HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein. 2021-07-15 21:13:25 +00:00
Artem Smotrakov 9241470dfd HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 15:06:37 +09:00
Jim Brennan 7d231f4b98 HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false (#2443). Contributed by Ahmed Hussein. 2021-06-25 18:06:37 +00:00
S O'Donnell caa59d7a58 HDFS-15150. Introduce read write lock to Datanode. Contributed by Stephen O'Donnell and Ahmed Hussein 2021-06-22 16:24:19 +01:00
Kihwal Lee ab6b5681e8 HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. 2021-06-16 11:38:30 -05:00
Kihwal Lee e7c7fb2896 HDFS-15963. Unreleased volume references cause an infinite loop. (#2941) Contributed by Shuyan Zhang. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 9f2db2c9fd)
 2021-06-15 19:57:21 -05:00
Konstantin V Shvachko 149325484f HDFS-16040. RpcQueueTime metric counts requeued calls as unique events. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit 8ce30f51f9)
 2021-05-27 20:19:44 -07:00
Konstantin V Shvachko 8715a18f95 HDFS-15915. Race condition with async edits logging due to updating txId outside of the namesystem log. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 1abd03d68f)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java
 2021-05-26 16:00:18 -07:00
Hui Fei 77d9c6d0f7 HDFS-15652. Make block size from NNThroughputBenchmark configurable (#2416) 
(cherry picked from commit 8a6d5b9151)
 2021-04-29 12:56:57 -07:00
Viraj Jasani 3c1c1b40d8
HDFS-15931 : Fix non-static inner classes for better memory management (#2830). Contributed by Viraj Jasani 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2021-04-01 17:48:27 -07:00
Akira Ajisaka f207d9e620
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin. Contributed by Ahmed Hussein. 
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
 2021-03-29 09:44:28 +09:00
touchida 7183e269ac
HDFS-13975. TestBalancer#testMaxIterationTime fails sporadically (#2726) 
Co-authored-by: toshihiko.uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 25390babc7)
 2021-03-17 16:34:52 +09:00
Viraj Jasani 3d0a0ed92b HDFS-15895 : Remove redundant String#format in DFSAdmin#printOpenFiles (#2772) 
(cherry picked from commit 9c43b60348)
 2021-03-17 15:16:18 +09:00
tomscut 68cccba893 HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time. (#2668) Contributed by tomscut. 
(cherry picked from commit 9cb51bf106)
 2021-03-06 14:22:51 -08:00
Arpit Agarwal 9f713bb825 HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq and Stephen O'Donnell. 
(cherry picked from commit 102c8fca10f3c626ab8bc47f818c8391a5c35289)
(cherry picked from commit 4a1d51dea2)
 2021-03-02 20:31:33 +00:00
Konstantin V Shvachko cdd3982db4 HDFS-15849. ExpiredHeartbeats metric should be of Type.COUNTER. Contributed by Qi Zhu. 
(cherry picked from commit 9501c698f4)
 2021-03-01 17:30:40 -08:00
S O'Donnell 628ccf81ef HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee 2021-02-24 14:47:36 +00:00
He Xiaoqiao ef14e16d36
HDFS-15792. ClasscastException while loading FSImage. Contributed by Renukaprasad C. 2021-02-07 15:50:10 +08:00
Symious 5dd6db1856
HDFS-15801. Backport HDFS-14582 to branch-2.10 (Failed to start DN with ArithmeticException when NULL checksum used) (#2659) 
Co-authored-by: Symious <yiyang0203@gmail.com>
 2021-02-01 15:43:44 -08:00
He Xiaoqiao b30e9198bb
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Felix N, Haibin Huang. 2021-01-30 12:11:44 +08:00
Jim Brennan c1a3e8158d [HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan. 
(cherry picked from commit 6184ab07f6)
 2021-01-28 17:46:06 +00:00
Wei-Chiu Chuang e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db)
 2021-01-21 17:48:14 +00:00
Akira Ajisaka db5b804238
HDFS-15717. Improve fsck logging. Contributed by Kihwal Lee and Ahmed Hussein (#2620) 
(cherry picked from commit be35fa186c)
 2021-01-15 10:28:11 +09:00
S O'Donnell 35c99be3da HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 2020-12-14 12:37:26 +00:00
Jim Brennan 8af247df85 HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) Contributed by Ahmed Hussein 2020-12-08 20:32:31 +00:00
Konstantin V Shvachko a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4)
 2020-11-03 14:39:38 -08:00
Wei-Chiu Chuang e7e165a927 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-28 09:22:47 -07:00
Wei-Chiu Chuang 90ebbaa393 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837a)
(cherry picked from commit 975d4b3d60)
 2020-10-24 13:38:58 -07:00
Konstantin V Shvachko 104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3c)
 2020-10-12 18:02:05 -07:00
Masatake Iwasaki 7b0dbcb657 Publishing the bits for release 2.10.1 (addendum) 2020-09-22 11:51:53 +09:00
Masatake Iwasaki f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Jitendra Pandey 43c6c54b8c HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 
(cherry picked from commit f29e55bf5e)
 2020-09-08 22:35:01 +09:00
Chen Liang 2b6b5476ba HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit f734455e5d)
 2020-08-28 17:48:38 -07:00
S O'Donnell a1fe3eed9b HDFS-15313. Ensure inodes in active filesytem are not deleted during snapshot delete. Contributed by Shashikant Banerjee. 2020-07-30 15:48:15 +01:00
Chen Liang 3044a13080 HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang. 
(cherry picked from commit 3833c616e0)
(cherry picked from commit 370efc6d95)
(cherry picked from commit 1e33696bc8)
 2020-07-20 15:26:44 -07:00
Akira Ajisaka e53186954d HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li. 
(cherry picked from commit 1defe3a65a)
(cherry picked from commit 0d949d375e)
(cherry picked from commit 635067d03d)
 2020-07-20 13:39:00 -07:00
He Xiaoqiao 685b81f447 HDFS-14498. LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell. 2020-07-17 00:17:17 +08:00
He Xiaoqiao b2c44db79e Revert "HDFS-14498 LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell." 
This reverts commit 3ebd783dc7.
 2020-07-13 23:57:11 +08:00
He Xiaoqiao 3ebd783dc7 HDFS-14498 LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell. 2020-07-13 14:45:09 +08:00
Yiqun Lin c63068be5f HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by Yiqun Lin. 
(cherry picked from commit 2e43c28e01)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2020-07-10 15:16:51 +09:00
Akira Ajisaka fb694e87a6
HDFS-15421. IBR leak causes standby NN to be stuck in safe mode. 
(cherry picked from commit c71ce7ac33)
 2020-06-28 17:23:27 +09:00
Kihwal Lee 17109758dd HDFS-12453. TestDataNodeHotSwapVolumes fails in trunk Jenkins runs. Contributed by Jim Brennan and Lei Xu. 2020-06-12 16:19:36 -05:00
Toshihiro Suzuki d1f4c8f10f
HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories (#2054) 
Contributed by Toshihiro Suzuki.
 2020-06-09 21:44:27 +01:00
Inigo Goiri b411fcee86 HDFS-14054. TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky. Contributed by Zsolt Venczel. 
(cherry picked from commit 993c2140cc)
 2020-06-03 14:16:38 -07:00
Kihwal Lee 54c7d23e58 HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger 
(cherry picked from commit 9ef6ed9c1c)
 2020-06-02 09:52:06 -05:00
cnauroth 73ae3cee85 HDFS-9376. TestSeveralNameNodes fails occasionally. Contributed by Masatake Iwasaki. 
(cherry picked from commit 84a8147791)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2020-05-20 08:22:47 +09:00
Chen Liang d0504cf74d HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang 2020-05-18 12:10:05 -07:00
Yiqun Lin e6374f031a HDFS-15264. Backport Datanode detection to branch-2.10. Contributed by Lisheng Sun. 2020-05-17 11:59:10 +08:00
Inigo Goiri b5eb08a0d9 HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina. 2020-05-14 11:26:31 -07:00