7bb902bc0d
HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang
2020-05-18 10:58:52 -07:00
c84e6beada
HDFS-14999. Avoid Potential Infinite Loop in DFSNetworkTopology. Contributed by Ayush Saxena.
2020-05-18 22:24:34 +05:30
2abcf7762a
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
2020-05-18 09:23:09 -07:00
4525292d41
Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )"
...
This reverts commit 86e6aa8eec
2020-05-18 09:22:05 -07:00
50caba1a92
HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.
2020-05-18 08:40:38 -07:00
a3f44dacc1
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-05-18 07:09:43 -07:00
86e6aa8eec
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.
2020-05-18 07:04:04 -07:00
6e416a83d1
HDFS-15358. RBF: Unify router datanode UI with namenode datanode UI. Contributed by Ayush Saxena.
2020-05-17 03:06:27 +05:30
178336f8a8
HDFS-15356. Unify configuration `dfs.ha.allow.stale.reads` to DFSConfigKeys. Contributed by Xiaoqiao He.
2020-05-16 16:35:06 +05:30
ac4a2e11d9
HDFS-15306. Make mount-table to read from central place ( Let's say from HDFS). Contributed by Uma Maheswara Rao G.
2020-05-14 17:29:35 -07:00
1958cb7c2b
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:09:34 +05:30
0918433b4d
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:36:20 +05:30
3cacf1ce56
HDFS-15344. DataNode#checkSuperuserPrivilege should use UGI#getGroups after HADOOP-13442. ( #2004 )
2020-05-13 11:47:19 -07:00
743c2e9071
HDFS-15316. Deletion failure should not remove directory from snapshottables. Contributed by hemanthboyina
2020-05-13 15:01:07 +05:30
a3f945fb84
HADOOP-17035. fixed typos (timeout, interruped) ( #2007 )
...
Co-authored-by: Sungpeo Kook <elixir.kook@kakaocorp.com>
2020-05-12 10:50:04 -05:00
29dddb8a14
HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.
2020-05-12 15:07:51 +01:00
928b81a533
HDFS-15350. Set dfs.client.failover.random.order to true as default. ( #2008 )
2020-05-12 09:04:03 -05:00
8dad38c0be
HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei.
2020-05-12 18:34:26 +05:30
0fe49036e5
HDFS-15243. Add an option to prevent sub-directories of protected directories from deletion. Contributed by liuyanyu.
2020-05-12 13:11:31 +05:30
4c53fb9ce1
HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.
2020-05-11 16:48:34 +05:30
cb64e993c2
HDFS-15347. Replace the deprecated method shaHex. Contributed by bianqi.
2020-05-10 00:11:46 +05:30
d59de27c01
HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.
2020-05-08 00:53:49 +05:30
130f89e068
HDFS-15332. Quota Space consumed was wrong in truncate with Snapshots. Contributed by hemanthboyina.
2020-05-06 13:18:32 -07:00
7fddf4855e
HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.
2020-05-06 16:55:04 +05:30
adecdb8b53
HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. ( #1998 )
...
Reviewed-by: Arpit Agarwal <arp@apache.org>
2020-05-05 16:04:54 -07:00
9c8236d04d
HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.
2020-05-04 17:55:40 -07:00
ebb878bab9
HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
2020-05-04 10:29:50 -07:00
8dace8ff3a
HDFS-15328. Use DFSConfigKeys MONITOR_CLASS_DEFAULT constant. Contributed by bianqi.
2020-05-03 23:59:17 +05:30
d6fc482a54
Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
2020-05-03 19:09:14 +05:30
44de193bec
HDFS-15325. TestRefreshCallQueue is failing due to changed CallQueue constructor. Contributed by Fengnan Li
2020-05-02 20:56:29 -07:00
f40dacdc2e
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter ( #1992 )
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:02:27 -07:00
82343790ee
HDFS-15313. Ensure inodes in active filesytem are not deleted during snapshot delete. Contributed by Shashikant Banerjee.
2020-05-01 12:16:02 +05:30
263c76b678
HADOOP-17011. Tolerate leading and trailing spaces in fs.defaultFS. Contributed by Ctest
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2020-04-30 14:15:28 -07:00
9ca6298a9a
HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
2020-04-29 15:56:06 +01:00
5e0eda5d5f
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
2020-04-28 06:17:14 +09:00
62c26b91fd
HDFS-15298 Fix the findbugs warnings introduced in HDFS-15217 ( #1979 )
2020-04-27 17:20:47 +09:00
74d4a18735
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena
2020-04-25 15:43:02 -07:00
453c9c9a67
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address ( #1964 )
...
Contributed by Dhiraj Hegde.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
2020-04-25 13:04:32 -07:00
453771f170
HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun.
2020-04-24 11:53:24 -07:00
5958af4dbc
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina
2020-04-22 13:31:02 -07:00
1824aee9da
HDFS-15217 Add more information to longest write/read lock held log
2020-04-18 13:52:07 -07:00
37d6582223
HDFS-15266. Add missing DFSOps Statistics in WebHDFS. Contributed by Ayush Saxena.
2020-04-17 13:32:02 +05:30
3481895f8a
HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-04-16 20:18:25 -07:00
cc5c1da7c1
HDFS-15277. Parent directory in the explorer does not support all path formats. Contributed by Jianfei Jiang.
2020-04-15 23:01:58 +05:30
061afcdf30
HDFS-15269. NameNode should check the authorization API version only … ( #1945 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-04-09 09:19:35 -07:00
1189af4746
HDFS-15263. Fix the logic of scope and excluded scope in Network Topology. Contributed by Ayush Saxena.
2020-04-08 15:55:14 +05:30
c12ddbd1de
HDFS-15249 ThrottledAsyncChecker is not thread-safe. ( #1922 )
2020-04-07 14:38:09 +09:00
0b855b9f35
HDFS-15256. Fix typo in DataXceiverServer#run(). Contributed by Lisheng Sun.
2020-04-06 13:05:33 -07:00
d3b5951572
HDFS-15242. Add metrics for operations hold lock times of FsDatasetImpl. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2020-04-01 16:36:46 -07:00
960c9ebaea
HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang.
2020-03-30 02:40:25 +05:30
8914cf9167
Preparing for 3.4.0 development
2020-03-29 23:24:25 +05:30
f7a94ec0a4
HDFS-15239. Add button to go to the parent directory in the explorer. Contributed by hemanthboyina.
2020-03-29 08:17:02 -07:00
696a663cd0
HDFS-15193. Improving the error message for missing `dfs.namenode.rpc-address.$NAMESERVICE`. Contributed by Ctest.
2020-03-29 00:01:22 +05:30
f531a4a487
HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand.
2020-03-27 12:55:38 -07:00
0fa7bf47df
HDFS-15234. Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API. ( #1909 )
2020-03-25 16:03:26 -07:00
cdcb77a2c5
HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He.
2020-03-25 11:30:54 -07:00
a700803a18
HDFS-15154. Allow only hdfs superusers the ability to assign HDFS storage policies. Contributed by Siddharth Wagle.
...
Change-Id: I32d6dd2837945b8fc026a759aa367c55daefe348
2020-03-25 10:28:30 -07:00
d353b30baf
HDFS-15215. The Timestamp for longest write/read lock held log is wrong
2020-03-24 14:50:15 -07:00
ea87d60493
HDFS-13377. The owner of folder can set quota for his sub folder. Contributed by Yang Yun.
2020-03-24 22:56:09 +05:30
5250cd6db3
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit ( #1870 )
2020-03-24 15:15:46 +05:30
af64ce2f4a
HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-03-23 12:43:51 -07:00
62c8015683
HDFS-15227. NPE if the last block changes from COMMITTED to COMPLETE during FSCK. Contributed by Ayush Saxena.
2020-03-22 11:44:18 +05:30
b74e47e31a
HDFS-15214. WebHDFS: Add snapshot counts to Content Summary. Contributed by hemanthboyina.
2020-03-22 01:44:55 +09:00
f9bb2a8cc5
HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena.
2020-03-19 22:26:34 +05:30
bb41ddaf1e
HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena.
2020-03-19 21:23:13 +05:30
654db35fa2
HDFS-14919. Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
2020-03-18 21:00:45 +05:30
096533c2dc
HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs.
...
Contributed by Wei-Chiu Chuang.
Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
2020-03-18 13:27:13 +00:00
8a7c54995a
HDFS-15159. Prevent adding same DN multiple times in PendingReconstructionBlocks. Contributed by hemanthboyina.
2020-03-15 16:16:27 +05:30
7b637a3341
HDFS-14820. The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big. Contributed by Lisheng Sun.
2020-03-13 19:01:23 -07:00
e9955bb8ff
HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2020-03-13 18:55:06 -07:00
4b95c242ec
HDFS-14743. Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... ( #1829 )
...
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
2020-03-13 11:29:44 -07:00
fb1d7635ae
HDFS-15155. writeIoRate of DataNodeVolumeMetrics is never used. Contributed by Haibin Huang.
2020-03-13 21:10:03 +05:30
25497799cb
HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree.
2020-03-13 15:24:19 +00:00
20903f72b4
HDFS-15039. Cache meta file length of FinalizedReplica to reduce call File.length(). Contributed by Yang Yun.
2020-03-12 19:23:12 -07:00
0695f7a538
HDFS-14612. SlowDiskReport won't update when SlowDisks is always empty in heartbeat. Contributed by Haibin Huang.
2020-03-12 17:22:11 -07:00
f736408a83
HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree.
2020-03-12 18:59:17 +05:30
c734d69a55
HADOOP-16898. Batch listing of multiple directories via an (unstable) interface
...
Contributed by Steve Loughran.
This moves the new API of HDFS-13616 into a interface which is implemented by
HDFS RPC filesystem client (not WebHDFS or any other connector)
This new interface, BatchListingOperations, is in hadoop-common,
so applications do not need to be compiled with HDFS on the classpath.
They must cast the FS into the interface.
instanceof can probe the client for having the new interface -the patch
also adds a new path capability to probe for this.
The FileSystem implementation is cut; tests updated as appropriate.
All new interfaces/classes/constants are marked as @unstable.
Change-Id: I5623c51f2c75804f58f915dd7e60cb2cffdac681
2020-03-09 14:51:16 +00:00
4062217189
HDFS-15212. TestEncryptionZones.testVersionAndSuiteNegotiation fails in trunk. Contributed by Ayush Saxena.
2020-03-06 10:14:53 -08:00
3afd4cbe89
HDFS-14977. Quota Usage and Content summary are not same in Truncate with Snapshot. Contributed by hemanthboyina.
2020-03-04 10:13:23 -08:00
1d4d0fcbe1
HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein.
2020-03-04 10:02:54 -08:00
bbd704bb82
HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. ( #1871 )
2020-03-04 11:07:31 +09:00
1a636da041
HDFS-15033. Support to save replica cached files to other place and make expired time configurable. Contributed by Yang Yun.
2020-02-29 09:55:29 +05:30
97b797c314
HDFS-15149. TestDeadNodeDetection test cases time-out. Contributed by Lisheng Sun.
2020-02-28 18:47:22 -08:00
fccfb02672
HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena.
2020-02-28 18:33:06 +05:30
429da635ec
HDFS-15186. Erasure Coding: Decommission may generate the parity block's content with all 0 in some case. Contributed by Yao Guangdong.
2020-02-28 00:31:55 +05:30
cd2c6b1aac
HDFS-15124. Crashing bugs in NameNode when using a valid configuration for . Contributed by Ctest.
2020-02-27 22:19:35 +05:30
2059f255d3
HDFS-15167. Block Report Interval shouldn't be reset apart from first Block Report. Contributed by Ayush Saxena.
2020-02-27 14:08:42 +05:30
033a3d7ff9
HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li.
2020-02-27 04:02:26 +05:30
209630472a
HDFS-15120. Refresh BlockPlacementPolicy at runtime. Contributed by Jinglun.
2020-02-27 02:22:24 +05:30
352a4ec16d
HDFS-15147. LazyPersistTestCase wait logic is flawed. Contributed by Ahmed Hussein.
2020-02-26 09:33:29 -06:00
900430b990
HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-02-25 13:28:50 -08:00
1c5d2f1fdc
HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun.
2020-02-24 13:39:16 -08:00
7f8685f476
HDFS-15187. CORRUPT replica mismatch between namenodes after failover. Contributed by Ayush Saxena.
2020-02-24 20:38:04 +05:30
93b8f453b9
HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li.
2020-02-24 20:17:01 +05:30
b5698e0c33
HDFS-15176. Enable GcTimePercentage Metric in NameNode's JvmMetrics. Contributed by Jinglun.
2020-02-24 00:07:18 +05:30
9eb7a8bdf8
HDFS-15041. Make MAX_LOCK_HOLD_MS and full queue size configurable. Contributed by zhuqi.
2020-02-23 14:25:39 +05:30
ba9025c7cd
HDFS-15182. TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk. Contributed by Lisheng Sun.
2020-02-22 14:27:26 +05:30
ecbcb058b8
HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko.
2020-02-21 17:53:37 -08:00
6f84269bcd
HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko.
2020-02-21 10:51:14 -08:00
2338d25dc7
HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. ( #1758 )
2020-02-21 11:56:07 +09:00
ec7507162c
HDFS-15165. In Du missed calling getAttributesProvider. Contributed by Bharat Viswanadham.
2020-02-19 11:33:58 -08:00
ac4b556e2d
HDFS-13739. Add option to disable rack local write preference. Contributed by Ayush Saxena.
2020-02-19 08:20:59 +05:30
84f7638840
HADOOP-13666. Supporting rack exclusion in countNumOfAvailableNodes in NetworkTopology. Contributed by Inigo Goiri.
2020-02-18 00:43:33 +05:30
810783d443
HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree.
2020-02-16 12:15:03 +05:30
c75756fe13
HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena.
2020-02-14 22:07:24 +05:30
a98352ced1
HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina.
2020-02-13 16:57:41 +05:30
0ddb5f0881
HDFS-13989. RBF: Add FSCK to the Router ( #1832 )
...
Co-authored-by: Inigo Goiri <inigoiri@apache.org>
2020-02-13 10:06:07 +09:00
9b8a78d97b
HDFS-14758. Make lease hard limit configurable and reduce the default.
...
Contributed by hemanthboyina.
2020-02-11 12:40:00 -06:00
d7c136b9ed
HDFS-15150. Introduce read write lock to Datanode. Contributed Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-02-11 08:00:15 -08:00
6191d4b4a0
HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun.
2020-02-09 23:32:22 +05:30
d23317b102
HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang
2020-02-08 10:33:57 +05:30
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
ce7b8b5634
HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang.
2020-02-04 12:12:35 -08:00
10a60fbe20
HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan.
2020-02-04 12:22:35 -06:00
1e3a0b0d93
HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
2020-01-31 16:13:02 -08:00
799d4c1cf4
HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails
...
intermittently. Contributed by Ahmed Hussein.
2020-01-29 11:00:27 -06:00
483397c7f7
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang
2020-01-28 15:20:36 -08:00
3e86807802
Revert "[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang."
...
This reverts commit ff8ff0f7e5
2020-01-28 15:19:47 -08:00
ff8ff0f7e5
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang.
2020-01-28 15:14:58 -08:00
1839c467f6
HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein.
2020-01-28 10:10:35 -08:00
87c198468b
HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun.
2020-01-28 23:09:38 +05:30
f876dc228b
HDFS-15143. LocatedStripedBlock returns wrong block type. Contributed by Ayush Saxena.
2020-01-28 19:34:21 +05:30
6d008c0d39
HDFS-15128. Unit test failing to clean testing data and crashed future Maven test run due to failure in TestDataNodeVolumeFailureToleration. Contributed by Ctest
2020-01-25 11:47:29 +05:30
d10f77e3c9
HDFS-15119. Allow expiration of cached locations in DFSInputStream.
...
Contributed by Ahmed Hussein.
2020-01-24 09:15:27 -06:00
92c58901d7
HDFS-15117. EC: Add getECTopologyResultForPolicies to DistributedFileSystem. Contributed by Ayush Saxena
2020-01-23 18:18:34 +05:30
bd03053ea2
HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein.
2020-01-22 09:14:38 -06:00
8cfc3673dc
HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui.
2020-01-21 13:29:20 -08:00
b657822b98
HDFS-15126. TestDatanodeRegistration#testForcedRegistration fails intermittently. Contributed by Ahmed Hussein.
2020-01-21 13:22:53 -08:00
1defe3a65a
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.
2020-01-21 10:58:32 +09:00
d7c4f8ab21
HDFS-13616. Batch listing of multiple directories ( #1725 )
2020-01-15 17:22:47 -08:00
52b360a928
HADOOP-16797. Add Dockerfile for ARM builds. Contributed by Vinayakumar B. ( #1801 )
2020-01-13 10:40:29 +05:30
cebce0a348
HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena.
2020-01-11 10:29:11 +05:30
26a969ec73
HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang.
2020-01-10 17:06:59 -08:00
5fb901ac40
HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein.
2020-01-10 12:43:21 -06:00
b32757c616
HDFS-15107. dfs.client.server-defaults.validity.period.ms to support time units. Contributed by Ayush Saxena.
2020-01-10 08:14:56 +05:30
bf45f3b80a
HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina.
2020-01-09 12:04:05 +05:30
7030722e5d
HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He.
2020-01-08 14:25:17 +05:30
aba3f6c3e1
HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. ( #1797 )
2020-01-08 16:45:39 +09:00
a43c177f1d
HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. ( #1775 )
2020-01-08 11:28:34 +09:00
f8644fbe9f
HDFS-15091. Cache Admin and Quota Commands Should Check SuperUser Before Taking Lock. Contributed by Ayush Saxena.
2020-01-04 19:02:59 +05:30
037ec8cfb1
HDFS-15068. DataNode could meet deadlock if invoke refreshVolumes when register. Contributed by Aiphago.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2020-01-04 01:55:36 +09:00
d79cce20ab
HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He.
2020-01-02 11:44:00 +05:30
074050ca59
HDFS-15063. HttpFS: getFileStatus doesn't return ecPolicy. Contributed by hemanthboyina.
2020-01-01 11:26:38 +09:00
ee51eadda0
HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina.
2019-12-29 11:15:54 +05:30
dc32f583af
HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena.
2019-12-28 21:32:15 +09:00
8730a7bf60
HDFS-15003. RBF: Make Router support storage type quota. Contributed by Jinglun.
2019-12-27 09:22:58 +05:30
80f91d14ab
HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He.
2019-12-27 09:12:44 +05:30
300505c562
HDFS-15054. Delete Snapshot not updating new modification time. Contributed by hemanthboyina.
2019-12-25 12:42:50 +05:30
b98ac2a3af
HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.
2019-12-24 12:08:34 -08:00
52d7b745c6
HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.
2019-12-19 09:42:17 -08:00
b86895485d
HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He.
2019-12-19 09:34:43 -08:00
fdd96e46d1
HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.
2019-12-18 22:50:46 +05:30
24080666e5
HDFS-14908. LeaseManager should check parent-child relationship when filter open files. Contributed by Jinglun.
2019-12-16 18:41:45 -08:00
dc6cf17b34
HDFS-15048. Fix findbug in DirectoryScanner.
2019-12-16 19:00:57 +09:00
7a87007545
HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina.
2019-12-15 00:33:32 +05:30
65c4660bcd
HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.
2019-12-12 10:22:05 -08:00
bdd00f10b4
HDFS-15047. Document the new decommission monitor (HDFS-14854). ( #1755 )
2019-12-13 00:35:41 +09:00
93bb368094
HDFS-14983. RBF: Add dfsrouteradmin -refreshSuperUserGroupsConfiguration command option. Contributed by Xieming Li
2019-12-12 10:56:19 +09:00
c174d50b30
HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.
2019-12-11 09:39:37 -08:00
c93cb6790e
HDFS-14854. Create improved decommission monitor implementation. Contributed by Stephen O'Donnell.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-12-10 17:16:43 -08:00
11cd5b6e39
HDFS-15028. Keep the capacity of volume and reduce a system call. Contributed by Yang Yun.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-12-07 08:46:45 +09:00
ecd461f940
HDFS-14751. Synchronize on diffs in DirectoryScanner. Contributed by Lisheng Sun.
2019-12-06 15:10:40 -08:00
313b76f8e9
HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
2019-12-06 15:10:40 -08:00
705b172b95
HDFS-14998. [SBN read] Update Observer Namenode doc for ZKFC after HDFS-14130. Contributed by Fei Hui.
2019-12-06 23:04:09 +05:30
83a14559e5
HDFS-15023. [SBN read] ZKFC should check the state before joining the election. Contributed by Fei Hui.
2019-12-05 18:52:01 +05:30
0c217feed8
HDFS-15026. TestPendingReconstruction#testPendingReconstruction() fail in trunk. Contributed by Lisheng Sun.
2019-12-03 09:18:09 +05:30
6b2d6d4aaf
HDFS-15009. FSCK -list-corruptfileblocks return Invalid Entries. Contributed by hemanthboyina.
2019-11-29 23:55:37 +05:30
44f7b9159d
HDFS-15013. Reduce NameNode overview tab response time. Contributed by HuangTao.
2019-11-29 00:42:31 +05:30
0384687811
HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore.
2019-11-28 23:49:35 +05:30
46166bd8d1
HDFS-14961. [SBN read] Prevent ZKFC changing Observer Namenode state. Contributed by Ayush Saxena.
2019-11-28 18:28:20 +05:30
c3659f8f94
HDFS-15019. Refactor the unit test of TestDeadNodeDetection. Contributed by Lisheng Sun.
2019-11-28 14:41:49 +08:00
2b452b4e60
HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago.
2019-11-28 10:43:35 +08:00
c8bef4d6a6
HDFS-14649. Add suspect probe for DeadNodeDetector. Contributed by Lisheng Sun.
2019-11-27 10:57:20 +08:00
049940e77b
HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar.
2019-11-23 02:30:02 +05:30
b25e94ce29
HDFS-14924. RenameSnapshot not updating new modification time. Contributed by hemanthboyina
2019-11-22 13:48:02 +09:00
9b6906fe91
HDFS-14651. DeadNodeDetector checks dead node periodically. Contributed by Lisheng Sun.
2019-11-22 10:53:55 +08:00
26270196a2
HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina.
2019-11-22 00:31:25 +05:30
98d249dcda
HDFS-14996. RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations. Contributed by Ayush Saxena.
2019-11-21 12:08:42 +05:30
fd264b8265
HDFS-14995. Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime(). Contributed by Lisheng Sun.
2019-11-21 00:00:42 +05:30
0b50aa29fd
HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
2019-11-19 12:16:57 -08:00
c870514740
HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina.
2019-11-20 00:07:43 +05:30
34cb595d2e
HDFS-14967. TestWebHDFS fails in Windows. Contributed by Renukaprasad C.
2019-11-18 18:49:58 +05:30
b3119b9ab6
HDFS-14648. Implement DeadNodeDetector basic model. Contributed by Lisheng Sun.
2019-11-16 11:32:41 +08:00
67f2c491fe
HDFS-14802. The feature of protect directories should be used in RenameOp ( #1669 )
2019-11-15 12:35:37 -08:00
c892a879dd
HDFS-14882. Consider DataNode load when #getBlockLocation. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Istvan Fajth <pifta@cloudera.com>
2019-11-15 12:16:31 -08:00
b2cc8b6b4a
HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
2019-11-15 10:10:37 -08:00
586defe711
HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.
2019-11-13 11:46:35 -08:00
97ec34e117
HDFS-14959: [SBNN read] access time should be turned off ( #1706 ) Contributed by Chao Sun.
2019-11-12 11:35:50 -08:00
40150da1e1
HDFS-14922. Prevent snapshot modification time got change on startup. Contributed by hemanthboyina.
2019-11-12 10:53:54 -08:00
77934bc07b
HDFS-14975. Add CR for SetECPolicyCommand usage. Contributed by Fei Hui.
2019-11-11 11:05:34 +05:30
6663d6a5c2
HDFS-14928. UI: unifying the WebUI across different components. Contributed by Xieming Li
2019-11-11 14:04:57 +09:00
320008bb7c
HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.
2019-11-09 22:14:35 +05:30
247584eb63
HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.
2019-11-07 22:49:01 +05:30
dd900259c4
HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.
2019-11-06 09:56:19 -08:00
9d0d580031
HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.
2019-11-06 09:30:04 -08:00
9e287054a8
HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
2019-11-06 22:02:03 +05:30
dcf55838ae
HDFS-14880. Correct the sequence of statistics & exit message in balencer. Contributed by Renukaprasad C.
2019-11-06 21:45:03 +05:30
c36014165c
HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
2019-11-06 19:28:55 +05:30
bfb8f28cc9
HDFS-14775. Add Timestamp for longest FSN write/read lock held log. Contributed by Chen Zhang.
2019-11-05 09:19:52 -08:00
b643a1cbe8
HDFS-14938. Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Lisheng Sun.
2019-11-05 09:23:20 +05:30
2ffec347eb
HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.
2019-11-05 01:37:15 +05:30
eb73ba6ed5
HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.
2019-11-04 09:40:42 -08:00
de6b8b0c0b
HDFS-14925. Rename operation should check nest snapshot ( #1670 )
...
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.
Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
2019-11-01 16:37:53 -07:00
7d7acb004a
HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.
2019-11-01 22:50:10 +05:30
02009c3bb7
HDFS-14768. EC : Busy DN replica should be consider in live replica check. Contributed by guojh.
2019-11-01 22:15:05 +05:30
9d25ae7669
HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.
2019-10-31 23:49:37 +05:30
d9fbedc4ae
HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun.
2019-10-31 23:39:27 +05:30
e6137d0e50
Make upstream aware of 2.10.0 release
2019-10-30 20:58:32 -07:00
fa4904cdca
HDFS-14935. Refactor DFSNetworkTopology#isNodeInScope. Contributed by Lisheng Sun.
2019-10-29 12:30:24 +05:30
30ed24a421
HDFS-14730. Removed unused configuration dfs.web.authentication.filter.
...
Contributed by Chen Zhang
2019-10-28 19:27:32 -04:00
9ef6ed9c1c
HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger
2019-10-28 16:37:04 +00:00
30db895b59
HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() ( #1667 )
...
Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-25 13:09:14 -07:00
862526530a
HDFS-14933. Fixing a typo in documentation of Observer NameNode. Contributed by Xieming Li
2019-10-25 15:23:05 +09:00
0db0f1e399
HDFS-14917. Change the ICON of "Decommissioned & dead" datanode on "dfshealth.html". Contributed by Xieming Li
2019-10-25 10:27:38 +09:00
a1b4eebcc9
HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. ( #1672 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-10-24 13:09:52 -07:00
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
ee699dc26c
HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena.
2019-10-24 12:14:09 +05:30
a901405ad8
HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-22 13:31:15 -07:00
6020505943
HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena.
2019-10-22 18:44:22 +05:30
19f35cfd57
HDFS-14918. Remove useless getRedundancyThread from BlockManagerTestUtil. Contributed by Fei Hui.
2019-10-22 18:34:02 +05:30
72003b19bf
HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.
2019-10-21 17:31:43 -07:00
c4e27ef773
HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.
2019-10-21 16:33:15 -05:00
74c2329fc3
HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena.
2019-10-21 18:35:12 +05:30
447f46d962
HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui.
2019-10-20 06:10:59 +05:30
3d41f33018
HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng.
...
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-18 08:25:02 -07:00
54dc6b7d72
HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore.
2019-10-17 22:28:47 +05:30
5527d79adb
HDFS-14810. Review FSNameSystem editlog sync. Contributed by Xiaoqiao He.
2019-10-17 21:56:30 +05:30
336abbd873
HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina.
2019-10-14 23:31:43 +05:30
fabd41fa48
HDFS-14856. Fetch file ACLs while mounting external store. ( #1478 )
2019-10-14 09:44:56 -07:00
5f4641a120
HDFS-14238. A log in NNThroughputBenchmark should change log level to INFO instead of ERROR. Contributed by Shen Yinjie.
2019-10-12 21:06:23 +05:30
d76e2655ac
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore.
2019-10-09 23:20:28 +05:30
72ae371e7a
HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.
2019-10-08 11:56:52 -07:00
91320b4461
HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.
2019-10-08 14:14:14 +05:30
382967be51
HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
2019-10-08 00:14:30 +05:30
aa24add8f0
HDFS-14890. Fixed namenode and journalnode startup on Windows.
...
Contributed by Siddharth Wagle
2019-10-04 13:13:10 -04:00
b23bdaf085
HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina
2019-10-04 16:47:06 +09:00
c99a12167f
HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-03 22:13:50 -07:00
844b766da5
HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. ( #1573 )"
2019-10-03 21:32:48 -07:00
a3fe404244
HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C.
2019-10-03 22:34:55 +05:30
1303255aee
HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang.
2019-10-02 16:38:20 -07:00
3df733c250
HDFS-14885. UI: Fix a typo on WebUI of DataNode. Contributed by Xieming Li.
2019-10-02 11:03:16 +09:00
6ef6594c7e
HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. ( #1370 )
...
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.
Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37
* Fix checkstyle
Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
2019-10-01 21:16:46 +05:30
b3275ab1f2
HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko.
2019-09-30 16:48:10 -07:00
760b523e58
Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao."
...
This reverts commit 0feba4396f
2019-09-29 13:11:31 -07:00
2200871607
HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
2019-09-28 22:03:53 +05:30
c4c8d5fd0e
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-28 21:44:44 +05:30
0d5d0b914a
Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
...
This reverts commit ce58c05f1d
2019-09-28 21:32:15 +05:30
13b427fc05
HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable ( #963 ) Contributed by Sahil Takiar.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
ce58c05f1d
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-27 18:58:19 +05:30
2870668cfa
Make upstream aware of 3.1.3 release.
2019-09-27 16:44:07 +08:00
a3f68933d7
HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
2019-09-26 19:54:07 +05:30
66400c1cbb
HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.
2019-09-25 01:31:15 +05:30
eb96a3093e
HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.
2019-09-25 01:16:30 +05:30
afa1006a53
HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.
2019-09-24 09:58:42 -07:00
c30e495557
HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.
2019-09-23 21:22:50 +05:30
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes ( #1494 ). Contributed by Vinayakumar B.
2019-09-23 12:37:47 +05:30
4c0a7a9e13
Make upstream aware of 3.2.1 release.
2019-09-23 06:20:54 +00:00
659c88801d
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
2019-09-22 22:02:54 +05:30
3f223bebfa
HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
2019-09-20 11:45:01 -07:00
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 ( #1432 )
...
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
2019-09-20 16:08:30 +05:30
4ed0aefe9f
HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
2019-09-18 16:47:46 -07:00
5dd859a8a0
HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-09-18 09:34:01 -07:00
f580a87079
HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
2019-09-17 14:55:34 -07:00
e04b8a46c3
HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
2019-09-14 10:39:20 +05:30
06ad6540b3
HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-13 10:36:45 -07:00
4852a90e4b
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
2019-09-13 00:17:13 +05:30
2ff2a7f612
HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
2019-09-12 19:43:18 +05:30
d1c303a497
HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
2019-09-12 19:11:50 +05:30
68612a0410
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
2019-09-12 12:41:57 +09:00
c92a3e94d8
HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.
2019-09-06 10:22:28 -07:00
d98c54816d
HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
2019-09-06 18:12:42 +05:30
fa7f03fc56
HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
2019-09-06 17:55:30 +05:30
494d75eb2b
HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.
2019-09-05 21:15:17 -07:00
72d8b92ba5
HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
2019-09-05 11:44:02 -07:00
3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
2019-09-03 09:29:58 -07:00
91b01a1db7
HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
2019-09-03 18:08:42 +05:30
5ff76cb8bc
HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
2019-09-03 12:37:09 +05:30
915cbc91c0
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-02 09:47:04 -07:00
d207aba026
Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell."
...
This reverts commit 7bebad61d9
2019-09-02 09:43:44 -07:00
c3abfcefdd
HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena.
2019-09-02 07:24:04 +05:30
96f7dc1992
HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.
2019-08-30 14:31:05 +05:30
e220dac15c
HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.
2019-08-30 08:40:13 +05:30
915271245b
Make DataNodePeerMetrics#minOutlierDetectionSamples configurable ( #1314 ). Contributed by Lisheng Sun.
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-08-29 18:35:26 -07:00
7bebad61d9
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 17:38:07 -07:00
fcb7884bfc
HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 16:56:14 -07:00
3b22fcd377
HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.
2019-08-29 14:21:09 -07:00
ffca734c62
HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.
2019-08-29 14:18:35 -07:00
f600fbb6c4
HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Kuhu Shukla <kshukla@apache.org>
2019-08-29 10:10:52 -07:00
29bd6f3fc3
HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.
2019-08-28 23:58:23 +05:30
6e37d65b03
HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 16:20:25 -07:00
dde9399b37
HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 15:26:59 -07:00
ce8eb1283a
Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
...
This reverts commit 62445021d5
2019-08-27 23:23:44 +05:30
8ab7020e64
HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns
2019-08-27 10:30:31 -07:00
07e3cf952e
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.
2019-08-26 15:57:55 -07:00
ebef99dcf4
HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.
2019-08-23 12:18:43 -07:00
d699022fce
HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.
2019-08-23 11:33:21 -07:00
bd7baea5a5
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.
2019-08-23 15:28:27 +09:00
abc8fde4ca
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.
2019-08-23 14:44:34 +09:00
b67812ea21
HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index ( #1028 ). Contributed by Stephen O'Donnell.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-08-22 17:09:57 -07:00
93daf69f90
HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
2019-08-22 15:21:33 -07:00
Chen Liang
Ayush Saxena
Wei-Chiu Chuang
He Xiaoqiao
pustota2009
Uma Maheswara Rao G
Surendra Singh Lilhore
Xiaoyu Yao
Elixir Kook
S O'Donnell
Takanobu Asanuma
Inigo Goiri
Konstantin V Shvachko
Mingliang Liu
Akira Ajisaka
Shashikant Banerjee
Toshihiro Suzuki
Dhiraj
Stephen O'Donnell
Brahma Reddy Battula
Chen Liang
Arpit Agarwal
Karthik Palanisamy
Steve Loughran
Akshay Nehe
Kihwal Lee
Masatake Iwasaki
Vinayakumar B
Chao Sun
Rakesh Radhakrishnan
Erik Krogen
Yiqun Lin
Hui Fei
Zhao Junwang
Jonathan Hung
Eric Yang
Eric Badger
zhaoyim
Mukul Kumar Singh
Yuming Wang
Ashvin
Virajith Jalaparti
Sahil Takiar
Zhankun Tang
Rohith Sharma K S
leosunli
Stephen O'Donnell
Istvan Fajth
CR Hota
Jonathan Hung
Arpit Agarwal