bee2846bee
HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena
...
(cherry picked from commit cc671b16f7
2020-06-16 16:54:40 -07:00
418580446b
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 76fa0222f0
2020-06-16 16:54:01 -07:00
0b5e202614
HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit ed83c865dd
2020-06-16 16:53:38 -07:00
8e71e85af7
HDFS-15322. Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 4734c77b4b
2020-06-16 16:53:10 -07:00
5b248de42d
HADOOP-17024. ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). Contributed by Abhishek Das.
...
(cherry picked from commit ce4ec74453
2020-06-16 16:52:29 -07:00
544996c857
HDFS-15306. Make mount-table to read from central place ( Let's say from HDFS). Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit ac4a2e11d9
2020-06-16 16:50:57 -07:00
0b9e5ea592
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 730a39d138
2020-06-16 16:02:07 -07:00
d73cdb1c86
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 81d8a887b0
2020-06-15 11:12:08 +09:00
3b8418250f
HDFS-15403. NPE in FileIoProvider#transferToSocketFully. Contributed by hemanthboyina.
...
(cherry picked from commit f41a144077
2020-06-15 09:17:40 +09:00
534b15caf9
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:20:10 +05:30
043628dcf1
HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang.
2020-06-10 12:09:34 +05:30
2d4faa39e8
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
...
*Added missed test file.
2020-06-09 18:47:24 +05:30
ec8f3714e0
HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories ( #2052 )
...
Contributed by Toshihiro Suzuki.
(cherry picked from commit 545a0a147c
2020-06-05 11:17:13 +01:00
9b84a637b7
HADOOP-17055. Remove residual code of Ozone ( #2039 )
...
(cherry picked from commit d9838f2d42
2020-05-29 16:50:10 +09:00
6b7040a1cb
HDFS-13183. Addendum: Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
2020-05-28 13:04:22 +05:30
718c8a5868
HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun.
2020-05-24 18:06:19 +05:30
c44d8b8bdf
HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina.
...
(cherry picked from commit 4d22d1c58f
2020-05-23 17:33:39 +09:00
e452163a06
HDFS-15293. Relax the condition for accepting a fsimage when receiving a checkpoint. Contributed by Chen Liang
...
(cherry picked from commit 7bb902bc0d
2020-05-19 09:52:25 +09:00
eb045ea056
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a3f44dacc1
2020-05-18 09:59:12 -07:00
3915d1afc7
HDFS-15356. Unify configuration `dfs.ha.allow.stale.reads` to DFSConfigKeys. Contributed by Xiaoqiao He.
...
(cherry picked from commit 178336f8a8
2020-05-18 09:58:59 -07:00
240cba7e6d
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
(cherry picked from commit 2abcf7762a
2020-05-18 09:41:11 -07:00
289f8acc64
Revert "HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )"
...
This reverts commit ad9a6a0ee3
2020-05-18 09:41:03 -07:00
53d22fdb88
Revert "HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He."
...
This reverts commit acae31aa28
2020-05-18 09:39:57 -07:00
032ccba67c
HDFS-15207. VolumeScanner skip to scan blocks accessed during recent scan peroid. Contributed by Yang Yun.
...
(cherry picked from commit 50caba1a92
2020-05-18 08:42:20 -07:00
acae31aa28
HDFS-13183. Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a3f44dacc1
2020-05-18 07:10:27 -07:00
ad9a6a0ee3
HDFS-15202 Boost short circuit cache (rebase PR-1884) ( #2016 )
...
Added parameter dfs.client.short.circuit.num improving HDFS-client's massive reading performance by create few instances ShortCircuit caches instead of one. It helps avoid locks and lets CPU do job.
(cherry picked from commit 86e6aa8eec
2020-05-18 07:05:03 -07:00
7cef7109de
YARN-10265. Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue. Contributed by liusheng
2020-05-14 15:17:13 +05:30
74887ab2ad
YARN-9898. Dependency netty-all-4.1.27.Final doesn't support ARM platform. Contributed by liusheng.
2020-05-14 00:37:09 +05:30
ee58c112d0
Revert "SPNEGO TLS verification"
...
This reverts commit 0f27c04c23
2020-05-13 17:14:51 +09:00
433aaeefa4
HDFS-15255. Consider StorageType when DatanodeManager#sortLocatedBlock(). Contributed by Lisheng Sun.
2020-05-12 15:25:05 +01:00
ef41229f70
HDFS-14367. EC: Parameter maxPoolSize in striped reconstruct thread pool isn't affecting number of threads. Contributed by Guo Lei.
2020-05-12 18:43:37 +05:30
d1e5e393c3
HDFS-15270. Account for *env == NULL in hdfsThreadDestructor ( #1951 )
...
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.
The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.
Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
(cherry picked from commit 1996351b0b
2020-05-11 10:45:57 -07:00
61a4cd5539
HDFS-15338. listOpenFiles() should throw InvalidPathException in case of invalid paths. Contributed by Jinglun.
2020-05-11 16:52:23 +05:30
6ccb1cd5f1
HDFS-15250. Setting `dfs.client.use.datanode.hostname` to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest.
2020-05-10 11:44:02 +05:30
817814105a
HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole.
2020-05-08 00:57:06 +05:30
edf52d29f1
HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 9c8236d04d
2020-05-06 15:13:33 -07:00
4e0d99c4d6
HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun.
2020-05-06 17:02:40 +05:30
78c97907cb
HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. ( #1998 )
...
Reviewed-by: Arpit Agarwal <arp@apache.org>
(cherry picked from commit adecdb8b53
2020-05-05 21:25:56 -07:00
8809b43fb9
HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit ebb878bab9
2020-05-04 10:45:18 -07:00
df16146e7f
Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida.
2020-05-03 19:17:14 +05:30
e32e1384d9
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter ( #1992 )
...
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:59:58 -07:00
0f27c04c23
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454
2020-05-02 21:51:12 +09:00
dfa7f160a5
Preparing for 3.3.1 development
2020-04-30 13:33:42 +09:00
a34174fea3
HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun.
...
(cherry picked from commit 9ca6298a9a
2020-04-29 16:01:48 +01:00
08dbe32122
HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.( #1980 )
...
(cherry picked from commit 816042e62b
2020-04-29 11:16:56 +05:30
d504574d82
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina.
...
(cherry picked from commit 5e0eda5d5f
2020-04-28 06:18:25 +09:00
1db1992b31
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena
2020-04-25 15:44:01 -07:00
1c19107ce8
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address ( #1964 )
...
Contributed by Dhiraj Hegde.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
2020-04-25 13:06:08 -07:00
d6c48f8100
HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun.
...
(cherry picked from commit 453771f170
2020-04-24 11:54:11 -07:00
47b330dc34
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina
2020-04-22 13:41:04 -07:00
5459dd64fd
HDFS-14742. RBF: TestRouterFaultTolerant tests are flaky
...
(cherry picked from commit e069a06137
2020-04-21 10:27:27 +09:00
a6c718fd0f
HDFS-15218. RBF: MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 8e6227441a
2020-04-18 20:21:01 +05:30
aaad947c74
HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a
2020-04-16 20:19:00 -07:00
749a5b81da
HDFS-15269. NameNode should check the authorization API version only … ( #1945 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 061afcdf30
2020-04-09 09:45:05 -07:00
d501c6a43d
HDFS-15258. RBF: Mark Router FSCK unstable. ( #1934 )
...
(cherry picked from commit 1695d8d59c
2020-04-04 00:16:52 +09:00
f7a94ec0a4
HDFS-15239. Add button to go to the parent directory in the explorer. Contributed by hemanthboyina.
2020-03-29 08:17:02 -07:00
696a663cd0
HDFS-15193. Improving the error message for missing `dfs.namenode.rpc-address.$NAMESERVICE`. Contributed by Ctest.
2020-03-29 00:01:22 +05:30
f531a4a487
HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand.
2020-03-27 12:55:38 -07:00
cbe71eaddf
HDFS-15238. RBF: NamenodeHeartbeatService caused memory to grow rapidly. Contributed by xuzq.
2020-03-27 13:42:06 +05:30
679631b188
HDFS-13470. RBF: Add Browse the Filesystem button to the UI.
2020-03-26 08:23:18 -07:00
0fa7bf47df
HDFS-15234. Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API. ( #1909 )
2020-03-25 16:03:26 -07:00
cdcb77a2c5
HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He.
2020-03-25 11:30:54 -07:00
a700803a18
HDFS-15154. Allow only hdfs superusers the ability to assign HDFS storage policies. Contributed by Siddharth Wagle.
...
Change-Id: I32d6dd2837945b8fc026a759aa367c55daefe348
2020-03-25 10:28:30 -07:00
d353b30baf
HDFS-15215. The Timestamp for longest write/read lock held log is wrong
2020-03-24 14:50:15 -07:00
d9c4f1129c
HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error ( #1902 ). Contributed by zhengchenyu.
2020-03-24 23:17:22 +05:30
ea87d60493
HDFS-13377. The owner of folder can set quota for his sub folder. Contributed by Yang Yun.
2020-03-24 22:56:09 +05:30
5250cd6db3
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit ( #1870 )
2020-03-24 15:15:46 +05:30
5eddc82fb8
HDFS-15088. RBF: Correct annotation typo of RouterPermissionChecker#checkPermission. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-03-23 22:41:31 -07:00
f59f6891c8
HDFS-15232. Fix libhdfspp test failures with GCC 7. ( #1906 )
2020-03-24 11:36:14 +09:00
af64ce2f4a
HDFS-15113. Addendum: Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-03-23 12:43:51 -07:00
62c8015683
HDFS-15227. NPE if the last block changes from COMMITTED to COMPLETE during FSCK. Contributed by Ayush Saxena.
2020-03-22 11:44:18 +05:30
b74e47e31a
HDFS-15214. WebHDFS: Add snapshot counts to Content Summary. Contributed by hemanthboyina.
2020-03-22 01:44:55 +09:00
f9bb2a8cc5
HDFS-15200. Delete Corrupt Replica Immediately Irrespective of Replicas On Stale Storage. Contributed by Ayush Saxena.
2020-03-19 22:26:34 +05:30
bb41ddaf1e
HDFS-15223. FSCK fails if one namenode is not available. Contributed by Ayush Saxena.
2020-03-19 21:23:13 +05:30
654db35fa2
HDFS-14919. Provide Non DFS Used per DataNode in DataNode UI. Contributed by Lisheng Sun.
2020-03-18 21:00:45 +05:30
096533c2dc
HDFS-15208. Suppress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr in hdfs.
...
Contributed by Wei-Chiu Chuang.
Change-Id: I7abeaf4b65e2a4758356aa69ab8930f7e993077d
2020-03-18 13:27:13 +00:00
1d772dc542
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
2020-03-15 20:44:32 +05:30
74780c22eb
HDFS-15197. [SBN read] Change ObserverRetryOnActiveException log to debug. Contributed by Chen Liang.
2020-03-15 16:30:39 +05:30
8a7c54995a
HDFS-15159. Prevent adding same DN multiple times in PendingReconstructionBlocks. Contributed by hemanthboyina.
2020-03-15 16:16:27 +05:30
7b637a3341
HDFS-14820. The default 8KB buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream is too big. Contributed by Lisheng Sun.
2020-03-13 19:01:23 -07:00
e9955bb8ff
HDFS-15113. Missing IBR when NameNode restart if open processCommand async feature. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2020-03-13 18:55:06 -07:00
4b95c242ec
HDFS-14743. Enhance INodeAttributeProvider/ AccessControlEnforcer Interface in HDFS to support Authorization of mkdir, rm, rmdir, copy, move etc... ( #1829 )
...
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
2020-03-13 11:29:44 -07:00
fb1d7635ae
HDFS-15155. writeIoRate of DataNodeVolumeMetrics is never used. Contributed by Haibin Huang.
2020-03-13 21:10:03 +05:30
25497799cb
HDFS-15216. Wrong Use Case of -showprogress in fsck. Contributed by Ravuri Sushma sree.
2020-03-13 15:24:19 +00:00
20903f72b4
HDFS-15039. Cache meta file length of FinalizedReplica to reduce call File.length(). Contributed by Yang Yun.
2020-03-12 19:23:12 -07:00
0695f7a538
HDFS-14612. SlowDiskReport won't update when SlowDisks is always empty in heartbeat. Contributed by Haibin Huang.
2020-03-12 17:22:11 -07:00
f736408a83
HDFS-14442. Disagreement between HAUtil.getAddressOfActive and RpcInvocationHandler.getConnectionId. Contributed by Ravuri Sushma sree.
2020-03-12 18:59:17 +05:30
c734d69a55
HADOOP-16898. Batch listing of multiple directories via an (unstable) interface
...
Contributed by Steve Loughran.
This moves the new API of HDFS-13616 into a interface which is implemented by
HDFS RPC filesystem client (not WebHDFS or any other connector)
This new interface, BatchListingOperations, is in hadoop-common,
so applications do not need to be compiled with HDFS on the classpath.
They must cast the FS into the interface.
instanceof can probe the client for having the new interface -the patch
also adds a new path capability to probe for this.
The FileSystem implementation is cut; tests updated as appropriate.
All new interfaces/classes/constants are marked as @unstable.
Change-Id: I5623c51f2c75804f58f915dd7e60cb2cffdac681
2020-03-09 14:51:16 +00:00
d4d4c37810
HADOOP-14630 Contract Tests to verify create, mkdirs and rename under a file is forbidden
...
Contributed by Steve Loughran.
Not all stores do complete validation here; in particular the S3A
Connector does not: checking up the entire directory tree to see if a path matches
is a file significantly slows things down.
This check does take place in S3A mkdirs(), which walks backwards up the list of
parent paths until it finds a directory (success) or a file (failure).
In practice production applications invariably create destination directories
before writing 1+ file into them -restricting check purely to the mkdirs()
call deliver significant speed up while implicitly including the checks.
Change-Id: I2c9df748e92b5655232e7d888d896f1868806eb0
2020-03-09 14:44:28 +00:00
4062217189
HDFS-15212. TestEncryptionZones.testVersionAndSuiteNegotiation fails in trunk. Contributed by Ayush Saxena.
2020-03-06 10:14:53 -08:00
3afd4cbe89
HDFS-14977. Quota Usage and Content summary are not same in Truncate with Snapshot. Contributed by hemanthboyina.
2020-03-04 10:13:23 -08:00
1d4d0fcbe1
HDFS-15204. TestRetryCacheWithHA testRemoveCacheDescriptor fails intermittently. Contributed by Ahmed Hussein.
2020-03-04 10:02:54 -08:00
bbd704bb82
HADOOP-16899. Update HdfsDesign.md to reduce ambiguity. ( #1871 )
2020-03-04 11:07:31 +09:00
0dd8956f2e
HADOOP-16885. Encryption zone file copy failure leaks a temp file
...
Contributed by Xiaoyu Yao.
Contains HDFS-14892. Close the output stream if createWrappedOutputStream() fails
Copying file through the FsShell command into an HDFS encryption zone where
the caller lacks permissions is leaks a temp ._COPYING file
and potentially a wrapped stream unclosed.
This is a convergence of a fix for S3 meeting an issue in HDFS.
S3: a HEAD against a file can cache a 404,
-you must not do any existence checks, including deleteOnExit(),
until the file is written.
Hence: HADOOP-16490, only register files for deletion the create worked
and the upload is not direct.
HDFS-14892. HDFS doesn't close wrapped streams when IOEs are raised on
create() failures. Which means that an entry is retained on the NN.
-you need to register a file with deleteOnExit() even if the file wasn't
created.
This patch:
* Moves the deleteOnExit to ensure the created file get deleted cleanly.
* Fixes HDFS to close the wrapped stream on failures.
2020-03-02 13:22:00 +00:00
1a636da041
HDFS-15033. Support to save replica cached files to other place and make expired time configurable. Contributed by Yang Yun.
2020-02-29 09:55:29 +05:30
97b797c314
HDFS-15149. TestDeadNodeDetection test cases time-out. Contributed by Lisheng Sun.
2020-02-28 18:47:22 -08:00
9a3a28018a
HDFS-15190. HttpFS: Add Support for Storage Policy Satisfier. Contributed by hemanthboyina.
2020-02-29 10:30:26 +09:00
fccfb02672
HDFS-15199. NPE in BlockSender. Contributed by Ayush Saxena.
2020-02-28 18:33:06 +05:30
429da635ec
HDFS-15186. Erasure Coding: Decommission may generate the parity block's content with all 0 in some case. Contributed by Yao Guangdong.
2020-02-28 00:31:55 +05:30
cd2c6b1aac
HDFS-15124. Crashing bugs in NameNode when using a valid configuration for . Contributed by Ctest.
2020-02-27 22:19:35 +05:30
57aa048516
HDFS-14668 Support Fuse with Users from multiple Security Realms ( #1739 )
2020-02-27 07:48:14 -08:00
2059f255d3
HDFS-15167. Block Report Interval shouldn't be reset apart from first Block Report. Contributed by Ayush Saxena.
2020-02-27 14:08:42 +05:30
033a3d7ff9
HDFS-15111. stopStandbyServices() should log which service state it is transitioning from. Contributed by Xieming Li.
2020-02-27 04:02:26 +05:30
209630472a
HDFS-15120. Refresh BlockPlacementPolicy at runtime. Contributed by Jinglun.
2020-02-27 02:22:24 +05:30
352a4ec16d
HDFS-15147. LazyPersistTestCase wait logic is flawed. Contributed by Ahmed Hussein.
2020-02-26 09:33:29 -06:00
900430b990
HDFS-14861. Reset LowRedundancyBlocks Iterator periodically. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-02-25 13:28:50 -08:00
1c5d2f1fdc
HDFS-15174. Optimize ReplicaCachingGetSpaceUsed by reducing unnecessary io operations. Contributed by Lisheng Sun.
2020-02-24 13:39:16 -08:00
7f8685f476
HDFS-15187. CORRUPT replica mismatch between namenodes after failover. Contributed by Ayush Saxena.
2020-02-24 20:38:04 +05:30
93b8f453b9
HDFS-15166. Remove redundant field fStream in ByteStringLog. Contributed by Xieming Li.
2020-02-24 20:17:01 +05:30
b5698e0c33
HDFS-15176. Enable GcTimePercentage Metric in NameNode's JvmMetrics. Contributed by Jinglun.
2020-02-24 00:07:18 +05:30
9eb7a8bdf8
HDFS-15041. Make MAX_LOCK_HOLD_MS and full queue size configurable. Contributed by zhuqi.
2020-02-23 14:25:39 +05:30
ba9025c7cd
HDFS-15182. TestBlockManager#testOneOfTwoRacksDecommissioned() fail in trunk. Contributed by Lisheng Sun.
2020-02-22 14:27:26 +05:30
ecbcb058b8
HDFS-14731. [FGL] Remove redundant locking on NameNode. Contributed by Konstantin V Shvachko.
2020-02-21 17:53:37 -08:00
ed70c115a8
HDFS-15172. Remove unnecessary deadNodeDetectInterval in DeadNodeDetector#checkDeadNodes(). Contributed by Lisheng Sun.
2020-02-21 16:36:30 -08:00
6f84269bcd
HDFS-15185. StartupProgress reports edits segments until the entire startup completes. Contributed by Konstantin V Shvachko.
2020-02-21 10:51:14 -08:00
2338d25dc7
HDFS-15052. WebHDFS getTrashRoot leads to OOM due to FileSystem object creation. ( #1758 )
2020-02-21 11:56:07 +09:00
ec7507162c
HDFS-15165. In Du missed calling getAttributesProvider. Contributed by Bharat Viswanadham.
2020-02-19 11:33:58 -08:00
ac4b556e2d
HDFS-13739. Add option to disable rack local write preference. Contributed by Ayush Saxena.
2020-02-19 08:20:59 +05:30
84f7638840
HADOOP-13666. Supporting rack exclusion in countNumOfAvailableNodes in NetworkTopology. Contributed by Inigo Goiri.
2020-02-18 00:43:33 +05:30
439d935e1d
HDFS-15173. RBF: Delete repeated configuration 'dfs.federation.router.metrics.enable' ( #1849 )
2020-02-17 15:55:10 +09:00
810783d443
HDFS-15135. EC : ArrayIndexOutOfBoundsException in BlockRecoveryWorker#RecoveryTaskStriped. Contributed by Ravuri Sushma sree.
2020-02-16 12:15:03 +05:30
c75756fe13
HDFS-15164. Fix TestDelegationTokensWithHA. Contributed by Ayush Saxena.
2020-02-14 22:07:24 +05:30
a98352ced1
HDFS-15086. Block scheduled counter never get decremet if the block got deleted before replication. Contributed by hemanthboyina.
2020-02-13 16:57:41 +05:30
0ddb5f0881
HDFS-13989. RBF: Add FSCK to the Router ( #1832 )
...
Co-authored-by: Inigo Goiri <inigoiri@apache.org>
2020-02-13 10:06:07 +09:00
f09710bbb8
HDFS-15161. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException in ShortCircuitCache#close(). Contributed by Lisheng Sun
2020-02-12 20:29:35 +05:30
3df0adaaea
HDFS-15127. RBF: Do not allow writes when a subcluster is unavailable for HASH_ALL mount points. Contributed by Inigo Goiri
2020-02-12 19:41:04 +05:30
9b8a78d97b
HDFS-14758. Make lease hard limit configurable and reduce the default.
...
Contributed by hemanthboyina.
2020-02-11 12:40:00 -06:00
d7c136b9ed
HDFS-15150. Introduce read write lock to Datanode. Contributed Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-02-11 08:00:15 -08:00
6191d4b4a0
HDFS-15158. The number of failed volumes mismatch with volumeFailures of Datanode metrics. Contributed by Yang Yun.
2020-02-09 23:32:22 +05:30
d23317b102
HDFS-15115. Namenode crash caused by NPE in BlockPlacementPolicyDefault when dynamically change logger to debug. Contributed by wangzhixiang
2020-02-08 10:33:57 +05:30
7dac7e1d13
HADOOP-16596. [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency ( #1635 ). Contributed by Vinayakumar B.
2020-02-07 14:51:24 +05:30
ce7b8b5634
HDFS-15148. dfs.namenode.send.qop.enabled should not apply to primary NN port. Contributed by Chen Liang.
2020-02-04 12:12:35 -08:00
10a60fbe20
HDFS-12491. Support wildcard in CLASSPATH for libhdfs. Contributed by Muhammad Samir Khan.
2020-02-04 12:22:35 -06:00
1e3a0b0d93
HDFS-7175. Client-side SocketTimeoutException during Fsck. Contributed by Stephen O'Donnell, Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
2020-01-31 16:13:02 -08:00
799d4c1cf4
HDFS-15146. TestBalancerRPCDelay.testBalancerRPCDelay fails
...
intermittently. Contributed by Ahmed Hussein.
2020-01-29 11:00:27 -06:00
483397c7f7
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Conntributed by Chen Liang
2020-01-28 15:20:36 -08:00
3e86807802
Revert "[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang."
...
This reverts commit ff8ff0f7e5
2020-01-28 15:19:47 -08:00
ff8ff0f7e5
[SBN Read] Slow clients when Observer reads are enabled but there are no Observers on the cluster. Contributed by Chen Liang.
2020-01-28 15:14:58 -08:00
1839c467f6
HDFS-13179. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. Contributed by Ahmed Hussein.
2020-01-28 10:10:35 -08:00
061421fc6d
HDFS-15145. HttpFS: getAclStatus() returns permission as null. Contributed by hemanthboyina.
2020-01-28 10:04:38 -08:00
87c198468b
HDFS-14993. checkDiskError doesn't work during datanode startup. Contributed by Yang Yun.
2020-01-28 23:09:38 +05:30
f876dc228b
HDFS-15143. LocatedStripedBlock returns wrong block type. Contributed by Ayush Saxena.
2020-01-28 19:34:21 +05:30
6d008c0d39
HDFS-15128. Unit test failing to clean testing data and crashed future Maven test run due to failure in TestDataNodeVolumeFailureToleration. Contributed by Ctest
2020-01-25 11:47:29 +05:30
d10f77e3c9
HDFS-15119. Allow expiration of cached locations in DFSInputStream.
...
Contributed by Ahmed Hussein.
2020-01-24 09:15:27 -06:00
92c58901d7
HDFS-15117. EC: Add getECTopologyResultForPolicies to DistributedFileSystem. Contributed by Ayush Saxena
2020-01-23 18:18:34 +05:30
bd03053ea2
HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein.
2020-01-22 09:14:38 -06:00
8cfc3673dc
HDFS-15092. TestRedudantBlocks#testProcessOverReplicatedAndRedudantBlock sometimes fails. Contributed by Fei Hui.
2020-01-21 13:29:20 -08:00
b657822b98
HDFS-15126. TestDatanodeRegistration#testForcedRegistration fails intermittently. Contributed by Ahmed Hussein.
2020-01-21 13:22:53 -08:00
f6d20daf40
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li.
2020-01-21 18:03:24 +09:00
1defe3a65a
HADOOP-16753. Refactor HAAdmin. Contributed by Xieming Li.
2020-01-21 10:58:32 +09:00
Ayush Saxena
Uma Maheswara Rao G
Abhishek Das
Stephen O'Donnell
Eric Yang
Takanobu Asanuma
Vinayakumar B
Toshihiro Suzuki
Wanqiang Ji
Chen Liang
He Xiaoqiao
Wei-Chiu Chuang
pustota2009
Surendra Singh Lilhore
Akira Ajisaka
S O'Donnell
Babneet Singh
Konstantin V Shvachko
Aryan Gupta
Mingliang Liu
Dhiraj
Inigo Goiri
Chen Liang
Arpit Agarwal
Isa Hekmatizadeh
Karthik Palanisamy
Steve Loughran
Akshay Nehe
Xiaoyu Yao
Istvan Fajth
Kihwal Lee
Masatake Iwasaki
xiaoxiaopan118