6a49bf9bff
HDFS-14852. Removing from LowRedundancyBlocks does not remove the block from all queues. Contributed by Fei Hui.
2020-08-25 15:18:36 +01:00
a7830423c5
HDFS-15448. Remove duplicate BlockPoolManager starting when run DataNode. Contriubted by jianghua zhu.
2020-08-24 21:16:00 +08:00
83fd15b412
HDFS-14504. Rename with Snapshots does not honor quota limit.
2020-08-23 16:10:47 +05:30
f734455e5d
HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira.
2020-08-20 15:25:10 -07:00
dd013f2fdf
HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. ( #2229 ). Contributed by Uma Maheswara Rao G.
2020-08-19 09:30:41 -07:00
b65e43fe38
HDFS-15525. Make trash root inside each snapshottable directory for WebHDFS ( #2220 )
2020-08-18 19:28:19 +09:00
092bfe7c8e
HDFS-15483. Ordered snapshot deletion: Disallow rename between two snapshottable directories. ( #2172 )
...
* HDFS-15483. Ordered snapshot deletion: Disallow rename between two snapshottable directories.
* Addressed review comments.
* Rebased to latest trunk and added snapshotTrashRoot config check.
* Addressed review comments.
* Removede empty line added in SnapshotManager.Java.
* Addressed whitespace issues.
2020-08-17 12:56:13 +05:30
e3d1966f58
HDFS-15439. Setting dfs.mover.retry.max.attempts to negative value will retry forever. Contributed by AMC-team.
2020-08-15 15:07:58 +05:30
b93dd7c281
HDFS-15519. Check inaccessible INodes in FsImageValidation. ( #2224 )
2020-08-14 10:10:01 -07:00
15a76e8d65
HDFS-15524. Add edit log entry for Snapshot deletion GC thread snapshot deletion. ( #2219 )
2020-08-14 20:46:48 +05:30
cb50e3fcf7
HDFS-15496. Add UI for deleted snapshots ( #2212 )
2020-08-13 22:36:15 +05:30
e592ec5f8b
HDFS-15518. Wrong operation name in FsNamesystem for listSnapshots. ( #2217 )
...
* HDFS-15518. Fixed String operationName = ListSnapshot.
* HDFS-15518. Changed ListSnapshot to operationName.
Co-authored-by: Aryan Gupta <aryan.gupta@cloudera.com>
2020-08-13 00:09:38 +05:30
10716040a8
HDFS-15493. Update block map and name cache in parallel while loading fsimage. Contributed by Chengwei Wang
2020-08-12 09:02:47 +01:00
11cec9ab94
HDFS-15523. Fix findbugs warnings from HDFS-15520. ( #2218 )
2020-08-11 21:41:28 -07:00
3fd3aeb621
HDFS-15492. Make trash root inside each snapshottable directory ( #2176 )
2020-08-11 08:52:16 -07:00
8955a6ceb7
HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false ( #2205 )
...
* HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false
* Used LambdaTestUtils#intercept in test
2020-08-11 00:01:58 -07:00
592127bdc2
HDFS-15520 Use visitor pattern to visit namespace tree ( #2203 )
2020-08-11 12:30:30 +05:30
7938ebfb9d
HDFS-15506. [JDK 11] Fix javadoc errors in hadoop-hdfs module. Contributed by Xieming Li.
2020-08-11 11:14:58 +09:00
c2a17659d1
HDFS-15443. Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. Contributed by AMC-team.
2020-08-08 14:33:53 +05:30
975b6024dd
HDFS-15514. Remove useless dfs.webhdfs.enabled. Contributed by Fei Hui.
2020-08-07 22:19:17 +05:30
e072d33327
HDFS-15497. Make snapshot limit on global as well per snapshot root directory configurable ( #2175 )
2020-08-04 14:10:29 +05:30
82f3ffcd64
HDFS-15503. File and directory permissions are not able to be modified from WebUI.
2020-08-03 23:21:01 +05:30
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors ( #2098 )
2020-08-03 10:46:51 +09:00
d8a2df25ad
HDFS-15498. Show snapshots deletion status in snapList cmd. ( #2181 )
2020-08-02 21:59:56 +05:30
528a799a78
HDFS-15229. Truncate info should be logged at INFO level. Contributed by Ravuri Sushma sree.
2020-08-02 12:02:37 +05:30
05b3337a46
HDFS-15481. Ordered snapshot deletion: garbage collect deleted snapshots ( #2165 )
2020-07-30 23:06:51 +05:30
68287371cc
HDFS-15488. Add a command to list all snapshots for a snaphottable root with snapshot Ids. ( #2166 )
2020-07-29 21:33:25 +05:30
026dce5334
HDFS-15465. Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode. ( #2135 )
2020-07-27 09:55:11 -07:00
2d12496643
HDFS-15480. Ordered snapshot deletion: record snapshot deletion in XAttr ( #2163 )
2020-07-22 16:16:27 -07:00
ac9a07b51a
HDFS-15478: When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target fs. ( #2160 ). Contributed by Uma Maheswara Rao G.
2020-07-21 23:29:10 -07:00
1b29c9bfee
HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. ( #2155 )
2020-07-22 13:40:20 +09:00
8b7695bb26
HDFS-15246. ArrayIndexOfboundsException in BlockManager CreateLocatedBlock. Contributed by Hemanth Boyina.
2020-07-21 09:00:07 -07:00
d57462f2da
HDFS-15479. Ordered snapshot deletion: make it a configurable feature ( #2156 )
2020-07-21 11:36:24 +05:30
d9441f95c3
HDFS-15470. Added more unit tests to validate rename behaviour across snapshots. Contributed by Shashikant Banerjee.
2020-07-21 11:13:05 +05:30
3833c616e0
HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang.
2020-07-20 12:49:58 -07:00
6cbd8854ee
HDFS-15381. Fix typos corrputBlocksFiles to corruptBlocksFiles. Contributed by bianqi.
2020-07-20 21:49:05 +05:30
2cec50cf16
HDFS-15463. Add a tool to validate FsImage. ( #2140 )
2020-07-20 11:44:30 +05:30
6bcb24d269
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-18 14:33:43 +05:30
1f71c4ae71
HADOOP-17099. Replace Guava Predicate with Java8+ Predicate
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 11:39:06 -05:00
98fcffe93f
HADOOP-17101. Replace Guava Function with Java8+ Function
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 09:53:18 -05:00
85d4718ed7
HDFS-15319. Fix INode#isInLatestSnapshot() API. Contributed by Shashikant Banerjee.
2020-07-15 10:32:25 +05:30
bdce75d737
HDFS-15371. Nonstandard characters exist in NameNode.java ( #2032 )
...
Contributed by zhaoyim
2020-07-14 14:42:12 +01:00
b97fea65e7
HDFS-14498 LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell.
2020-07-13 14:12:48 +08:00
3e70006639
HDFS-15464: ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links ( #2132 ). Contributed by Uma Maheswara Rao G.
2020-07-11 23:50:04 -07:00
f91a8ad88b
HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. ( #2085 )
2020-07-09 11:33:37 -07:00
f77bbc2123
HDFS-15312. Apply umask when creating directory by WebHDFS ( #2096 )
2020-07-06 19:01:46 -07:00
dc0626b5f2
HDFS-15449. Optionally ignore port number in mount-table name when picking from initialized uri. Contributed by Uma Maheswara Rao G.
2020-07-06 18:50:03 -07:00
834372f404
HDFS-15451. Do not discard non-initial block report for provided storage. ( #2119 ). Contributed by Shanyu Zhao.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-07-06 23:43:34 +08:00
55a2ae80dc
HDFS-15450. Fix NN trash emptier to work if ViewFSOveroadScheme enabled. Contributed by Uma Maheswara Rao G.
2020-07-04 13:45:49 -07:00
f86f15cf20
HDFS-15446. CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with error java.io.FileNotFoundException: Directory does not exist: /.reserved/raw/path. Contributed by Stephen O'Donnell.
2020-07-04 13:00:43 +05:30
1f2a80b5e5
HDFS-15430. create should work when parent dir is internalDir and fallback configured. Contributed by Uma Maheswara Rao G.
2020-07-04 00:12:10 -07:00
9ac498e300
HDFS-15416. Improve DataStorage#addStorageLocations() for empty locations. Contibuted by jianghua zhu.
2020-07-01 12:38:07 +08:00
2a67e2b1a0
HDFS-15160. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-06-30 07:10:17 -07:00
c71ce7ac33
HDFS-15421. IBR leak causes standby NN to be stuck in safe mode.
2020-06-28 16:02:47 +09:00
8db38c98a6
HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.
2020-06-27 19:13:01 +05:30
bed0a3a374
HDFS-15436. Default mount table name used by ViewFileSystem should be configurable ( #2100 )
...
* HDFS-15436. Default mount table name used by ViewFileSystem should be configurable
* Replace Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE use in tests
* Address Uma's comments on PR#2100
* Sort lists in test to match without concern to order
* Address comments, fix checkstyle and fix failing tests
* Fix checkstyle
2020-06-26 13:19:16 -07:00
d5e1bb6155
HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured. Contributed by Uma Maheswara Rao G.
2020-06-26 01:29:38 -07:00
7c02d1889b
HDFS-15427. Merged ListStatus with Fallback target filesystem and InternalDirViewFS. Contributed by Uma Maheswara Rao G.
2020-06-23 01:42:25 -07:00
17ffcab5f6
HDFS-14546. Document block placement policies. Contributed by Amithsha.
2020-06-21 17:27:28 +05:30
b27810aa60
HDFS-15418. ViewFileSystemOverloadScheme should represent mount links as non symlinks. Contributed by Uma Maheswara Rao G.
2020-06-20 00:32:02 -07:00
d50e93ce7b
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-06-18 06:45:15 -07:00
edf716a5c3
Revert "HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell."
...
This reverts commit 730a39d138
2020-06-18 06:45:15 -07:00
123777823e
HDFS-15406. Improve the speed of Datanode Block Scan. Contributed by hemanthboyina
2020-06-18 12:26:22 +01:00
c8ed33cd2a
HADOOP-9851. dfs -chown does not like "+" plus sign in user name. Contributed by Andras Bokor.
2020-06-17 13:55:40 +05:30
730a39d138
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-06-16 15:59:13 -07:00
f41a144077
HDFS-15403. NPE in FileIoProvider#transferToSocketFully. Contributed by hemanthboyina.
2020-06-15 09:15:53 +09:00
719b53a79d
HDFS-15351. Blocks scheduled count was wrong on truncate. Contributed by hemanthboyina.
2020-06-13 09:35:05 -07:00
785b1def95
HDFS-15387. FSUsage#DF should consider ViewFSOverloadScheme in processPath. Contributed by Uma Maheswara Rao G.
2020-06-12 14:32:19 -07:00
e154084770
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:16:33 +05:30
b735a77717
HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang.
2020-06-10 12:06:16 +05:30
8525874561
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
...
*Added missed test file.
2020-06-09 18:53:33 +05:30
3ca15292c5
HDFS-15394. Add all available fs.viewfs.overload.scheme.target.<scheme>.impl classes in core-default.xml bydefault. Contributed by Uma Maheswara Rao G.
2020-06-06 08:11:57 -07:00
cc671b16f7
HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena
2020-06-05 22:19:38 -07:00
76fa0222f0
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G.
2020-06-05 10:58:21 -07:00
2326123705
HDFS-15359. EC: Allow closing a file with committed blocks. Contributed by Ayush Saxena.
2020-06-05 19:15:05 +05:30
545a0a147c
HDFS-15386 ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories ( #2052 )
...
Contributed by Toshihiro Suzuki.
2020-06-05 11:11:49 +01:00
e8cb2ae409
HDFS-11041. Unable to unregister FsDatasetState MBean if DataNode is shutdown twice. Contributed by Wei-Chiu Chuang.
2020-06-03 12:47:15 +05:30
f6453244ab
HDFS-14960. TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology. Contributed by Jim Brennan.
2020-06-03 12:16:36 +05:30
ed83c865dd
HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G.
2020-06-02 11:09:26 -07:00
ae13a5ccbe
HDFS-10792. RedundantEditLogInputStream should log caught exceptions. Contributed by Wei-Chiu Chuang.
2020-05-31 17:10:59 +05:30
d9838f2d42
HADOOP-17055. Remove residual code of Ozone ( #2039 )
2020-05-29 16:49:18 +09:00
a838d871a7
HDFS-15368. TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally. Contributed by Xiaoqiao He.
2020-05-28 20:52:58 +05:30
9b38be43c6
HDFS-13183. Addendum: Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
2020-05-28 13:00:04 +05:30
2148a8fe64
HDFS-15362. FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks. Contributed by hemanthboyina.
2020-05-27 11:06:13 -07:00
6c9f75cf16
HDFS-15373. Fix number of threads in IPCLoggerChannel#createParallelExecutor. Contributed by Ayush Saxena.
2020-05-26 16:26:57 +05:30
f43a152b97
HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun.
2020-05-24 18:03:41 +05:30
f4901d0778
HDFS-15355. Make the default block storage policy ID configurable. Contributed by Yang Yun.
2020-05-24 17:59:16 +05:30
6e04b00df1
HDFS-12288. Fix DataNode's xceiver count calculation. Contributed by Lisheng Sun.
2020-05-23 09:58:19 -07:00
e0ae232f66
HDFS-15093. RENAME.TO_TRASH is ignored When RENAME.OVERWRITE is specified. Contributed by Ayush Saxena.
2020-05-23 18:38:32 +05:30
f99fcb26ab
HDFS-15288. Add Available Space Rack Fault Tolerant BPP. Contributed by Ayush Saxena.
2020-05-23 18:29:31 +05:30
4d22d1c58f
HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina.
2020-05-23 17:28:38 +09:00
4734c77b4b
HDFS-15322. Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. Contributed by Uma Maheswara Rao G.
2020-05-21 21:34:58 -07:00
be374faf42
HDFS-13639. SlotReleaser is not fast enough ( #1885 )
2020-05-21 13:21:17 -07:00
ce4ec74453
HADOOP-17024. ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). Contributed by Abhishek Das.
2020-05-18 22:27:12 -07:00
7bb902bc0d
HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang
2020-05-18 10:58:52 -07:00
c84e6beada
HDFS-14999. Avoid Potential Infinite Loop in DFSNetworkTopology. Contributed by Ayush Saxena.
2020-05-18 22:24:34 +05:30
2abcf7762a
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
2020-05-18 09:23:09 -07:00
4525292d41
Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )"
...
This reverts commit 86e6aa8eec
2020-05-18 09:22:05 -07:00
50caba1a92
HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.
2020-05-18 08:40:38 -07:00
a3f44dacc1
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-05-18 07:09:43 -07:00
S O'Donnell
He Xiaoqiao
hemanthboyina
Chen Liang
Uma Maheswara Rao G
Siyao Meng
bshashikant
Ayush Saxena
Tsz-Wo Nicholas Sze
Vivek Ratnavel Subramanian
Aryan Gupta
Akira Ajisaka
touchida
Masatake Iwasaki
Inigo Goiri
Ahmed Hussein
zhaoyim
Xiaoyu Yao
Ye Ni
Shanyu Zhao
Stephen O'Donnell
Virajith Jalaparti
Wei-Chiu Chuang
Takanobu Asanuma
Vinayakumar B
Toshihiro Suzuki
Wanqiang Ji
leosunli
Abhishek Das