9b0aace1e6
HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He.
2019-05-08 17:20:21 +05:30
69b903bbd8
HDFS-14372. NPE while DN is shutting down. Contributed by lujie.
2019-05-05 16:33:52 +05:30
e424392a62
HDFS-14438. Fix typo in OfflineEditsVisitorFactory. Contributed by bianqi.
2019-05-05 13:38:09 +05:30
1d59cc490c
HADOOP-16292. Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner. Contributed by Sherwood Zheng.
2019-05-03 10:14:17 -07:00
f1875b205e
HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena.
2019-05-03 11:22:14 +05:30
d6b7609c96
HDFS-14453. Improve Bad Sequence Number Error Message. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-02 14:25:02 -07:00
7a3188d054
HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.
2019-05-02 12:58:42 -07:00
865c328930
HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota.
2019-05-02 10:09:21 -07:00
7cb46f035a
HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-01 16:48:44 -07:00
4877f0aa51
HDFS-3246: pRead equivalent for direct read path ( #597 )
...
HDFS-3246: pRead equivalent for direct read path
Contributed by Sahil Takiar
2019-04-30 14:52:16 -07:00
4b4200f1f8
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq.
2019-04-29 14:49:35 -07:00
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
174b7d3126
HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen.
2019-04-22 13:29:56 -07:00
5321235fe8
HDFS-14445. TestTrySendErrorReportWhenNNThrowsIOException fails in trunk. Contributed by Ayush Saxena.
2019-04-22 13:23:42 -07:00
685cb83e4c
HDFS-14433. Remove the extra empty space in the DataStreamer logging. Contributed by Yishuang Lu. ( #747 )
2019-04-17 10:38:48 -07:00
13907d8479
HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-04-17 21:39:06 +09:00
be6c8014e6
HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena.
2019-04-16 10:34:31 -07:00
2382f63fc0
HADOOP-14747. S3AInputStream to implement CanUnbuffer.
...
Author: Sahil Takiar <stakiar@cloudera.com>
2019-04-12 18:12:02 -07:00
626fec652b
HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang.
2019-04-12 17:37:51 -07:00
586826fe99
HDFS-12245. Fix INodeId javadoc
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-11 10:01:47 +09:00
e9b859f749
HDFS-14371. Improve Logging in FSNamesystem by adding parameterized logging. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-05 16:09:00 -07:00
260d843b25
HDFS-14416. Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN. Contributed by Fengnan Li.
2019-04-05 09:26:06 -07:00
67020f0950
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji.
2019-04-05 12:29:11 +09:00
d2637cb176
HDFS-14389. getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-04 04:07:18 -07:00
7b5b783f66
HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li.
2019-04-03 16:11:13 -07:00
d7979079ea
HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
2019-04-03 12:59:39 -06:00
be488b6070
HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang.
2019-04-03 11:00:59 -07:00
cf268114c9
HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-02 12:24:55 -07:00
53a86e2b8e
HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project
...
This closes #665
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-01 13:49:37 +09:00
35ff31dd94
HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He.
2019-03-30 23:33:25 -07:00
f3f51284d5
HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R
2019-03-29 00:18:15 +05:30
49b02d4a9b
HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor.
2019-03-28 11:16:01 -07:00
15d38b1bf9
HDFS-14295. Add Threadpool for DataTransfers. Contributed by David Mollitor.
2019-03-28 03:37:33 -07:00
18c57cf046
HDFS-14304: High lock contention on hdfsHashMutex in libhdfs
...
This closes #595
Signed-off-by: Todd Lipcon <todd@apache.org>
2019-03-26 20:32:51 -07:00
fe29b3901b
HDFS-14348: Fix JNI exception handling issues in libhdfs
...
This closes #600
Signed-off-by: Todd Lipcon <todd@apache.org>
2019-03-26 16:33:34 -07:00
55fb3c32fb
HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory.
2019-03-27 03:27:02 +09:00
3f6d6d2811
HDFS-14359. Inherited ACL permissions masked when parent directory does not exist (mkdir -p)
...
(Contributed by Stephen O'Donnell via Daniel Templeton)
Change-Id: Ia83f799a8f56aa8057a967b234f184683395fa41
2019-03-25 16:27:04 -07:00
399563fec6
HDFS-14176. Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.
2019-03-20 11:20:45 -07:00
55b3a718e9
HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen.
2019-03-19 08:14:49 -07:00
ff06ef0631
HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun.
2019-03-15 11:06:42 -07:00
ba50a36a3e
HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He.
2019-03-14 22:21:08 -07:00
66357574ae
HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.
2019-03-13 13:15:56 -07:00
34b14061b3
HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-12 10:17:25 -07:00
1bc282e0b3
HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam.
2019-03-07 08:41:23 -08:00
618e009ac0
HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 15:04:06 -08:00
6192c1fe3b
Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
...
This reverts commit f5a4b43a49
2019-03-06 15:02:18 -08:00
f5a4b43a49
HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 14:58:45 -08:00
945b504c25
HDFS-14326. Add CorruptFilesCount to JMX. Contributed by Danny Becker.
2019-03-05 17:39:52 -08:00
f940ab242d
HDFS-7663. Erasure Coding: Append on striped file. Contributed by Ayush Saxena.
2019-03-05 19:26:42 +05:30
4b7313e640
HDFS-14336. Fix checkstyle for NameNodeMXBean. Contributed by Danny Becker.
2019-03-04 23:03:32 -08:00
cb0fa0ce3f
HDFS-14321. Fix -Xcheck:jni issues in libhdfs, run ctest with -Xcheck:jni enabled. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-04 12:37:57 -08:00
387dbe587a
HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star.
2019-03-04 10:43:44 -08:00
18ea0c1493
HDFS-14182. Datanode usage histogram is clicked to show ip list. Contributed by fengchuang.
2019-03-04 17:34:24 +08:00
6c4d566955
Revert "HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng."
...
This reverts commit 5c10630ad8
2019-03-04 16:59:20 +09:00
fc17ba172b
HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen.
2019-03-04 16:52:04 +09:00
80b77deb42
HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao.
2019-03-01 09:18:51 -08:00
7a0db2f92b
HDFS-13997. Secondary NN Web UI displays nothing, and the console log shows moment is not defined. Contributed by Ayush Saxena
2019-03-01 12:24:52 +05:30
05e4ddeee6
HDFS-14324. Fix TestDataNodeVolumeFailure. Contributed by Ayush Saxena.
2019-02-28 10:03:05 -08:00
0feba4396f
HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao.
2019-02-28 08:48:10 -08:00
a6ab37192a
HDFS-14130. [SBN read] Make ZKFC ObserverNode aware. Contributed by xiangheng and Konstantin Shvachko.
2019-02-25 14:35:02 -08:00
f4ae00c530
HDFS-7133. Support clearing namespace quota on '/'. Contributed by Ayush Saxena."
2019-02-26 00:36:14 +05:30
f7a27cdee4
HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li.
2019-02-23 09:35:36 -08:00
f19c844e75
HDFS-14285. libhdfs hdfsRead copies entire array even if its only partially filled. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-22 17:54:32 -08:00
bad3ffd290
HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen.
2019-02-22 13:58:49 -08:00
7d3b567194
HDFS-14298. Improve log messages of ECTopologyVerifier. Contributed by Kitti Nanasi.
2019-02-23 01:08:15 +05:30
92b53c40f0
HDFS-14216. NullPointerException happens in NamenodeWebHdfs. Contributed by lujie.
2019-02-21 20:36:34 +05:30
1bea785020
HDFS-14081. hdfs dfsadmin -metasave metasave_test results NPE. Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 14:28:37 -08:00
f5b4e0f971
HDFS-14302. Refactor NameNodeWebHdfsMethods#generateDelegationToken() to allow better extensibility. Contributed by CR Hota.
2019-02-20 13:55:13 -08:00
a30059bb61
HDFS-14267. Add test_libhdfs_ops to libhdfs tests, mark libhdfs_read/write.c as examples. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 11:38:37 -08:00
41e18feda3
HDFS-14235. Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon. Contributed by Ranith Sardar.
2019-02-20 16:56:10 +05:30
51950f149e
Logging stale datanode information. Contributed by Karthik Palanisamy.
2019-02-19 17:01:07 -08:00
14282e311b
HDFS-14188. Make hdfs ec -verifyClusterSetup command accept an erasure coding policy as a parameter. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-19 12:04:56 -08:00
7587f97127
HDFS-14296. Prefer ArrayList over LinkedList in VolumeScanner. Contributed by BELUGA BEHR.
2019-02-18 17:55:43 -08:00
1de25d134f
HDFS-9596. Remove Shuffle Method From DFSUtil. Contributed by BELUGA BEHR.
2019-02-18 15:30:38 -08:00
235e3da90a
HDFS-14287. DataXceiverServer May Double-Close PeerServer. Contributed by BELUGA BEHR.
2019-02-18 11:00:04 -08:00
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
dde0ab55aa
HDFS-14258. Introduce Java Concurrent Package To DataXceiverServer Class. Contributed by BELUGA BEHR.
2019-02-15 16:32:27 -08:00
75e15cc0c4
HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects.
...
Contributed by Elek, Marton.
2019-02-14 23:33:25 -08:00
64f28f9efa
HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.
2019-02-14 11:22:04 -08:00
0d7a5ac5f5
HDFS-13209. DistributedFileSystem.create should allow an option to provide StoragePolicy. Contributed by Ayush Saxena.
2019-02-14 22:13:14 +05:30
080a421911
HDFS-14262. [SBN read] Make Log.WARN message in GlobalStateIdContext more informative. Contributed by Shweta Yakkali.
2019-02-14 08:05:06 -08:00
024c87291c
HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang
2019-02-13 12:40:31 -08:00
29b411d5f0
HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore
2019-02-13 23:41:27 +05:30
35d4f32b32
HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena.
2019-02-13 23:06:46 +05:30
7806403842
HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena.
2019-02-12 21:57:57 +05:30
5c10630ad8
HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-11 11:02:32 -08:00
0ceb1b70f3
HDFS-14260. Replace synchronized method in BlockReceiver with atomic value. Contributed by BELUGA BEHR.
2019-02-11 10:09:44 -08:00
177131793a
HDFS-14172. Avoid NPE when SectionName#fromString returns null. Contributed by Xiang Li.
2019-02-08 20:51:30 +08:00
4be87353e3
HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-07 16:51:59 -08:00
668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth."
...
(accidentally mixed in two patches)
This reverts commit fa8cd1bf28
2019-02-07 21:57:22 +00:00
6aa63452b3
HDFS-14242. OIV WebImageViewer: NPE when param op is not specified. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-06 12:19:21 -08:00
d3de8e162b
HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun.
2019-02-06 09:54:47 -08:00
49ddd8a6ed
HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-05 15:14:19 -08:00
308f3168fa
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
...
Contributed by Da Zhou and Junhua Gu.
2019-02-05 19:23:15 +00:00
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
...
S3A to implement S3 Select through this API.
The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.
The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.
This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.
In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).
Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
5f15a60e32
HDFS-14125. Use parameterized log format in ECTopologyVerifier. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-04 14:52:07 -08:00
0e79a86582
HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar.
2019-02-04 11:59:48 -08:00
9aa3dc872c
HDFS-14158. Checkpointer ignores configured time period > 5 minutes
...
This closes #449
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-04 13:51:07 +09:00
ec77e95bbd
HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka.
2019-02-04 10:12:09 +09:00
0ab7fc9200
HDFS-14187. Make warning message more clear when there are not enough data nodes for EC write. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-31 16:07:24 -08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
d1714c20e9
Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
...
This reverts commit 1d523279da
2019-01-29 15:43:09 -08:00
dc5af4c6b7
HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki.
2019-01-26 05:34:32 +09:00
1d523279da
HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 09:02:57 -08:00
45caeee6cf
HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 07:37:01 -08:00
f3e642d92b
HDFS-14185. Cleanup method calls to static Assert methods in TestAddStripedBlocks
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I0d533f575a405ed3affd05994a4208bde7d9cbe9
2019-01-23 15:57:39 -08:00
951cdd7e4c
HDFS-14061. Check if the cluster topology supports the EC policy before setting, enabling or adding it. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-23 14:40:57 -08:00
221e308cb5
HDFS-14153. [SPS] : Add Support for Storage Policy Satisfier in WEBHDFS. Contributed by Ayush Saxena.
2019-01-23 17:00:37 +05:30
7d6792e5d2
HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena.
2019-01-23 10:03:06 +09:00
6f0756fc0e
HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal.
2019-01-21 20:45:09 -08:00
1ff658b2ef
HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal.
2019-01-21 20:44:37 -08:00
de34fc148c
HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui.
2019-01-22 12:01:08 +08:00
e9962240f0
Make 3.2.0 aware to other branches - jdiff
2019-01-21 22:41:26 +05:30
f048512bb8
HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena.
2019-01-16 10:14:22 -08:00
fb8932a727
HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.
2019-01-11 10:54:49 -08:00
9aeaaa0479
HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena.
2019-01-11 14:36:55 +05:30
4ab5260b7e
HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8
2019-01-08 17:04:57 -08:00
1a08302897
HDFS-14189. Fix intermittent failure of TestNameNodeMetrics. Contributed by Ayush Saxena.
2019-01-08 10:38:06 -08:00
f660e5eaa3
HDFS-14163. Debug Admin Command Should Support Generic Options. Contributed by Ayush Saxena.
2019-01-03 18:58:03 +05:30
21fe77e3ab
HDFS-14184. [SPS] Add support for URI based path in satisfystoragepolicy command. Contributed by Ayush Saxena.
2019-01-03 18:36:53 +05:30
f30473f2ef
HDFS-14183. [SPS] Remove the -w parameter from the -satisfystoragepolicy usage. Contributed by Ayush Saxena.
2018-12-31 03:16:26 +09:00
e9a005dfcf
HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang.
2018-12-28 10:34:18 -08:00
bd2a59e508
HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko.
2018-12-24 09:39:20 -08:00
c546b12e45
HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
c9d7737431
HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko.
2018-12-24 09:34:01 -08:00
fa8550337d
HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
c3efc299fe
HDFS-14116. [SBN read] Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
ef3e1929dd
HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
b73fb70f97
HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko.
2018-12-24 09:34:01 -08:00
b8ad6c85a5
HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
2904aa3808
HDFS-14131. [SBN read] Create user guide for Consistent Reads from Observer feature. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
71cf66e29b
HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
3bb92a1d9a
HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena.
2018-12-24 09:34:01 -08:00
cbc2f932eb
HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
a3aab48df0
HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
652b257478
HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
1a9ba9616a
HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
8c49135078
HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
b6f20c36c2
HDFS-13925. Unit Test for transitioning between different states. Contributed by Sherwood Zheng.
2018-12-24 09:34:00 -08:00
1c4265d7bc
HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
b5b9b77707
HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
a65bb97f5d
HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen.
2018-12-24 09:34:00 -08:00
b74a7dbf88
HDFS-13898. [SBN read] Throw retriable exception for getBlockLocations when ObserverNameNode is in safemode. Contributed by Chao Sun.
2018-12-24 09:34:00 -08:00
25b63e8da8
HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun.
2018-12-24 09:34:00 -08:00
28820fb6b1
HDFS-13778. [SBN read] TestStateAlignmentContextWithHA should use real ObserverReadProxyProvider instead of AlignmentContextProxyProvider. Contributed by Konstantin Shvachko and Plamen Jeliazkov.
2018-12-24 09:34:00 -08:00
d2e85b0b6d
HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
aa42fb0db7
HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen.
2018-12-24 09:34:00 -08:00
f9fc01cd7f
HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
25d8e39b5c
HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
66ec3a67e8
HDFS-13767. Add msync server implementation. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
4c9baba066
HDFS-13789. Reduce logging frequency of QuorumJournalManager#selectInputStreams. Contributed by Erik Krogen.
2018-12-24 09:34:00 -08:00
eae0a5d54a
HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
1e22f2bfbb
HDFS-13610. [SBN read] Edit Tail Fast Path Part 4: Cleanup. Integration test, documentation, remove unnecessary dummy sync, minors fixups. Contributed by Erik Krogen.
2018-12-24 09:34:00 -08:00
64b7cf59bd
HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
0105d7f597
HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko.
2018-12-24 09:33:59 -08:00
00e99c6594
HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen.
2018-12-24 09:33:59 -08:00
151c8ddbe4
HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen.
2018-12-24 09:33:59 -08:00
c81ac2ff02
HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen.
2018-12-24 09:33:59 -08:00
e880660a20
HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
a109f2b32f
HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
091ad974cd
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
9bf0696c73
HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
c9d73437e8
HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
3a78e5ffd8
HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
686fcd4db3
HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar.
2018-12-24 11:02:36 +05:30
d944d5ec46
HDFS-14165. In NameNode UI under DataNode tab ,the Capacity column is Non-Aligned. Contributed by Shubham Dewan.
2018-12-23 15:06:12 +05:30
feb2664ac4
HDFS-13946. Log longest FSN write/read lock held stack trace.
2018-12-22 23:09:59 +08:00
788e7473a4
HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena.
2018-12-17 11:24:57 +05:30
00d5e631b5
HDFS-14006. Refactor name node to allow different token verification implementations. Contributed by CR Hota.
2018-12-14 11:10:54 -08:00
f858f18554
HDFS-14088. RequestHedgingProxyProvider can throw NullPointerException when failover due to no lock on currentUsedProxy. Contributed by Yuxuan Wang.
2018-12-14 11:04:36 -08:00
cb3382f667
HDFS-14145. TestBlockStorageMovementAttemptedItems#testNoBlockMovementAttemptFinishedReportAdded fails sporadically in Trunk. Contributed by Ayush Saxena.
2018-12-14 10:59:02 -08:00
aa12859890
HDFS-14121. Log message about the old hosts file format is misleading
...
(Contributed by Zsolt Venczel via Daniel Templeton)
Change-Id: I7ff548f6c82e0aeb08a7a50ca7c2c827db8726bb
2018-12-14 13:54:57 +01:00
ca379e1c43
HDFS-13970. Use MultiMap for CacheManager Directives to simplify the code. Contributed by BELUGA BEHR.
2018-12-14 14:36:26 +09:00
4aa0609fb0
HDFS-13985. Clearer error message for ReplicaNotFoundException
...
(Contributed by Adam Antal via Daniel Templeton)
Change-Id: I68ae7a5bedecbc204c161fc3b3ce8878fe5d493d
2018-12-13 16:21:45 +01:00
561b61cfb1
HDFS-14144. TestPred fails in Trunk. Contributed by Ayush Saxena.
2018-12-12 11:16:16 -08:00
60af851e59
HDFS-14137. TestMaintenanceState fails with ArrayIndexOutOfBound Exception. Contributed by Ayush Saxena.
2018-12-11 11:22:32 -08:00
39dc7345b8
HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena.
2018-12-11 17:59:04 +05:30
80e59e7876
HDFS-14101. Fixing underflow error in test. Contributed by Zsolt Venczel.
2018-12-10 14:03:08 -07:00
8fc0d04517
HDFS-14001. [PROVIDED Storage] bootstrapStandby should manage the InMemoryAliasMap. Contributed by Virajith Jalaparti.
2018-12-07 18:30:36 -08:00
c03024a530
HDFS-14113. EC : Add Configuration to restrict UserDefined Policies. Contributed by Ayush Saxena.
2018-12-06 18:20:28 +05:30
69489ff2d1
HDFS-14119. Improve GreedyPlanner Parameter Logging. Contributed by Beluga Behr.
2018-12-03 12:55:52 -08:00
fb10803dfa
HDFS-13818. Extend OIV to detect FSImage corruption. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-12-03 10:34:39 -08:00
dd5e7c6b72
HDFS-12946. Add a tool to check rack configuration against EC policies. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-12-03 10:01:09 -08:00
042c8ef593
HDFS-14075. Terminate the namenode when failed to start log segment. Contributed by Ayush Saxena.
2018-12-02 12:31:08 +05:30
99e201dfe2
HDFS-14105. Replace TreeSet in NamenodeFsck with HashSet. Contributed by Beluga Behr.
2018-11-30 11:07:11 -08:00
b09cfad432
HDFS-14106. Refactor NamenodeFsck#copyBlock. Contributed by Beluga Behr.
2018-11-30 10:47:59 -08:00
0e36e935d9
HDFS-13870. WebHDFS: Document ALLOWSNAPSHOT and DISALLOWSNAPSHOT API doc. Contributed by Siyao Meng.
2018-11-30 11:31:34 +08:00
0081b02e35
HDFS-14112. Avoid recursive call to external authorizer for getContentSummary.
2018-11-29 13:55:21 -08:00
f534736867
HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena.
2018-11-30 00:18:27 +05:30
c1d24f8483
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
...
Contributed by Ewan Higgs and Steve Loughran.
2018-11-29 15:12:17 +00:00
300f560fcc
HDFS-14102. Performance improvement in BlockPlacementPolicyDefault. Contributed by Beluga Behr.
2018-11-28 11:33:22 -08:00
4ca3a6b21a
HDFS-14108. Performance improvement in BlockManager Data Structures. Contributed by Beluga Behr.
2018-11-28 11:25:47 -08:00
b098281454
HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B.
2018-11-26 18:22:32 +05:30
892b33e054
HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena.
2018-11-22 00:13:56 +05:30
1a00b4e325
Add 2.9.2 release notes and changes documents.
2018-11-20 14:25:18 +09:00
e56d9f2618
HDFS-14015. Improve error handling in hdfsThreadDestructor in native thread local storage
...
Change-Id: Ida1e888c9231b9e46081338e3a206d8f6faabd36
2018-11-16 16:24:40 -08:00
ad5256e44d
HDFS-14063. Support noredirect param for CREATE/APPEND/OPEN/GETFILECHECKSUM in HttpFS. Contributed by Íñigo Goiri.
2018-11-17 01:12:22 +08:00
68d5dfdc78
HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena.
2018-11-16 18:05:10 +05:30
8571507efa
HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena.
2018-11-16 17:26:36 +05:30
1e15c7e855
HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager compilation fails due to the usage of '_' as identifier. Contributed by Zsolt Venczel.
2018-11-15 12:42:31 -08:00
460a94a10f
HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang.
2018-11-15 10:58:57 -08:00
993c2140cc
HDFS-14054. TestLeaseRecovery2: testHardLeaseRecoveryAfterNameNodeRestart2 and testHardLeaseRecoveryWithRenameAfterNameNodeRestart are flaky. Contributed by Zsolt Venczel.
2018-11-15 09:29:14 -08:00
9da6054ca4
Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel."
...
This reverts commit 7dc79a8b5b
2018-11-13 12:44:25 -08:00
e7b63baca1
HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota
2018-11-13 12:45:44 +05:30
b6d4e19f34
HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena.
2018-11-12 15:31:42 -08:00
8d99648c20
HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-11-08 10:01:19 -08:00
0b6625a973
MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan
2018-11-07 08:20:49 -06:00
887244de4a
HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen.
2018-11-06 11:18:15 -08:00
f0ce072934
HDFS-14051. Refactor NameNodeHttpServer#initWebHdfs to specify local keytab. Contributed by CR Hota.
2018-11-06 10:05:58 -08:00
349168c4b3
HDFS-14047. [libhdfs++] Fix hdfsGetLastExceptionRootCause bug in test_libhdfs_threaded.c
...
(Contributed by Antoli Shein via Daniel Templeton)
Change-Id: Ia5546fd5f8270f285c373b9023a9bd9835059845
2018-11-06 09:22:27 -08:00
6430c9890a
HADOOP-15904. [JDK 11] Javadoc build failed due to bad use of '>' in hadoop-hdfs.
2018-11-06 18:42:38 +09:00
ffc9c50e07
HDFS-14053. Provide ability for NN to re-replicate based on topology changes. Contributed by Hrishikesh Gadre.
2018-11-05 21:38:39 -08:00
f3296501e0
HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak.
2018-11-05 16:48:37 -08:00
f3f5e7ad00
HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti.
2018-11-05 11:02:31 -08:00
4e3df75eb7
HDFS-14049. TestHttpFSServerWebServer fails on Windows because of missing winutils.exe. Contributed by Inigo Goiri.
2018-11-04 09:15:53 +08:00
8fe85af63b
HDFS-13996. Make HttpFS' ACLs RegEx configurable. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-11-01 17:36:50 -07:00
d98b881e9a
HDFS-14008. NN should log snapshotdiff report. Contributed by Pranay Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-11-01 17:26:00 -07:00
9c438abe52
HDFS-14033. [libhdfs++] Disable libhdfs++ build on systems that do not support thread_local. Contributed by Anatoli Shein.
2018-10-31 12:32:49 +05:30
fac9f91b29
HDFS-13942. [JDK10] Fix javadoc errors in hadoop-hdfs module. Contributed by Dinesh Chitlangia.
2018-10-31 14:43:58 +09:00
db7e636824
HDFS-14027. DFSStripedOutputStream should implement both hsync methods.
2018-10-29 19:06:15 -07:00
63e7134df6
Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao."
...
This reverts commit fb2b72e6fc
2018-10-28 17:04:59 -07:00
4f10d7e23f
HDFS-14028. HDFS OIV temporary dir deletes folder.
...
Contributed by Adam Antal.
2018-10-26 16:41:23 -07:00
1851d06eb3
HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena.
2018-10-26 09:29:12 -07:00
9b899f1ebd
HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal.
2018-10-25 15:14:13 -07:00
fb2b72e6fc
HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao.
2018-10-25 09:33:31 -07:00
97bd49fc36
HDFS-14026. Overload BlockPoolTokenSecretManager.checkAccess to make storageId and storageType optional. Contributed by Arpit Agarwal.
2018-10-24 21:55:42 -07:00
936fc3f3c2
HDFS-14025. TestPendingReconstruction.testPendingAndInvalidate fails. Contributed by Ayush Saxena.
2018-10-24 16:18:43 -07:00
c1874046e2
HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen.
2018-10-24 09:20:38 -07:00
635786a511
HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang.
2018-10-23 14:53:45 -07:00
efdfe679d6
HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena.
2018-10-23 14:28:37 -07:00
c58811c77d
HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar.
2018-10-22 14:44:28 -07:00
f069d38c8d
HDFS-13983. TestOfflineImageViewer crashes in windows. Contributed by Vinayakumar B.
2018-10-19 17:16:04 -07:00
88cce32551
HDFS-9872. HDFS bytes-default configurations should accept multiple size units. Contributed by Yiqun Lin.
2018-10-19 10:38:04 -07:00
8b64fbab1a
HDFS-13994. Improve DataNode BlockSender waitForMinLength. Contributed by BELUGA BEHR.
2018-10-19 10:36:03 -07:00
be1cffb085
HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma.
2018-10-18 15:25:53 -07:00
098d0b86fe
HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri.
2018-10-18 14:25:17 -07:00
0e56c883cd
HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin.
2018-10-18 10:53:30 -07:00
1e78dfca40
HDFS-13990. Synchronization Issue With HashResolver. Contributed by BELUGA BEHR.
2018-10-18 10:05:27 -07:00
7347fa2df3
HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota.
2018-10-17 16:40:25 -07:00
533138718c
HDFS-13662. TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky
2018-10-16 19:33:55 -07:00
f90c64e624
HDFS-13993. TestDataNodeVolumeFailure#testTolerateVolumeFailuresAfterAddingMoreVolumes is flaky. Contributed by Ayush Saxena.
2018-10-16 11:22:57 -07:00
b6fc72a025
HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
2018-10-15 10:51:55 -07:00
ee1c80ea32
HDFS-13987. RBF: Review of RandomResolver Class. Contributed by BELUGA BEHR.
2018-10-15 09:51:26 -07:00
b60ca37914
Fix potential FSImage corruption. Contributed by Daryn Sharp.
2018-10-15 15:48:26 +05:30
f63ee083db
HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui.
2018-10-12 10:10:12 -07:00
5ec86b445c
HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
...
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
2018-10-12 09:35:52 -07:00
6e0e6daaf3
HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena.
2018-10-12 17:34:10 +05:30
de44e1064f
HDFS-13156. HDFS Block Placement Policy - Client Local Rack. Contributed by Ayush Saxena.
2018-10-12 17:27:23 +05:30
c0118105b7
HDFS-13973. getErasureCodingPolicy should log path in audit event. Contributed by Shweta.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-11 20:39:41 -07:00
6dcfef79af
HDFS-13878. HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-11 15:01:50 -07:00
3532aa3886
HDFS-13968. BlockReceiver Array-Based Queue. Contributed by BELUGA BEHR.
2018-10-11 13:49:39 -07:00
2addebb94f
HDFS-13951. HDFS DelegationTokenFetcher can't print non-HDFS tokens in a tokenfile.
...
Contributed by Steve Loughran.
2018-10-11 20:03:56 +01:00
bf3d591f0c
HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-10 10:23:07 -07:00
3ead525c71
HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-10 10:11:10 -07:00
10185d9a77
HDFS-13882. Set a maximum delay for retrying locateFollowingBlock. Contributed by Kitti Nanasi.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-10 08:55:32 -07:00
b39b802422
HDFS-13949. Correct the description of dfs.datanode.disk.check.timeout in hdfs-default.xml. Contributed by Toshihiro Suzuki.
2018-10-10 12:29:07 +05:30
6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter.
2018-10-10 10:16:57 +09:00
605622c87b
HDFS-11396. TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out. Contributed by Ayush Saxena.
2018-10-09 17:03:21 -07:00
a05bd1288c
HDFS-13979. Review StateStoreFileSystemImpl Class. Contributed by BELUGA BEHR.
2018-10-09 12:00:21 -07:00
d4626b4d18
HDFS-13967. HDFS Router Quota Class Review. Contributed by BELUGA BEHR.
2018-10-09 16:11:07 +08:00
08bb6c49a5
HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads.
...
Contributed by Xiao Chen, Hrishikesh Gadre.
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-08 20:31:57 -07:00
1043795f7f
HDFS-13962. Add null check for add-replica pool to avoid lock acquiring. Contributed by Surendra Singh Lilhore.
2018-10-09 10:33:13 +08:00
3bfff0f79e
HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena.
2018-10-06 10:40:49 -07:00
73c660b43f
HDFS-13958. Miscellaneous Improvements for FsVolumeSpi. Contributed by BELUGA BEHR.
2018-10-05 09:32:19 -07:00
619e490333
HDFS-13957. Fix incorrect option used in description of InMemoryAliasMap.
2018-10-05 09:55:08 +08:00
cc2babc1f7
HDFS-13950. ACL documentation update to indicate that ACL entries are capped by 32. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-04 15:22:44 -07:00
396ce0d9f4
HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-04 15:17:47 -07:00
1dc0adfac0
HDFS-13947. Review of DirectoryScanner Class. Contributed by BELUGA BEHR.
2018-10-03 11:19:57 -07:00
fa7f7078a7
HDFS-13944. [JDK10] Fix javadoc errors in hadoop-hdfs-rbf module. Contributed by Inigo Goiri.
2018-10-03 12:48:45 +09:00
58fa96b697
Changed version in trunk to 3.3.0-SNAPSHOT.
2018-10-02 22:41:41 +05:30
5689355783
HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore.
2018-10-02 09:43:14 +08:00
f6c5ef9903
HDFS-13943. [JDK10] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Akira Ajisaka.
2018-10-02 09:49:48 +09:00
e2113500df
HDFS-8196. Post enabled Erasure Coding Policies on NameNode UI. Contributed by Kitti Nanasi and Kai Sasaki.
2018-09-29 00:53:35 +09:00
284aca8b63
HDFS-13938. Add a missing break in BaseTestHttpFSWith. Contributed by Siyao Meng.
2018-09-28 15:44:46 +09:00
f9c0221623
HDFS-13932. [JDK 11] Casts to BlockStoragePolicy[] in unit tests raise ClassCastExceptions. Contributed by Andrew Purtell.
2018-09-27 11:44:42 +09:00
e0ff8e2c10
HDFS-13840. RBW Blocks which are having less GS should be added to Corrupt. Contributed by Brahma Reddy Battula
2018-09-26 23:44:16 +05:30
6275b42870
HDFS-13927. Addendum: Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena.
2018-09-26 09:50:09 -07:00
8de5c923b4
HDFS-13876. HttpFS: Implement ALLOWSNAPSHOT, DISALLOWSNAPSHOT. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-09-24 10:40:30 -07:00
4758b4b6da
HDFS-13927. Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena.
2018-09-21 15:32:28 -07:00
6fc293fece
HDFS-13892. Disk Balancer: Make execute command documentation better.
...
Contributed by Ranith Sardar.
2018-09-19 20:48:41 -07:00
2eb597b151
HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena.
2018-09-19 13:48:27 -07:00
a30b4f9e71
HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen.
2018-09-19 13:22:37 -07:00
090272d7de
HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang.
2018-09-19 13:00:30 -07:00
56e0d635e0
HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena.
2018-09-19 09:30:25 -07:00
28ceb34a72
HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh.
2018-09-19 03:12:20 -07:00
27978bcb66
HDFS-13833. Improve BlockPlacementPolicyDefault's consider load logic. Contributed by Shweta.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-09-18 20:23:50 -07:00
44857476fa
HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng.
2018-09-18 15:33:02 -07:00
6ff509c32a
HDFS-11719. Arrays.fill() wrong index in BlockSender.readChecksum() exception handling. Contributed by Tao Zhang
2018-09-18 10:25:28 -07:00
d1541932db
HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng.
2018-09-17 14:42:03 -07:00
fdf5a3fd63
HDFS-13919. Documentation: Improper formatting in Disk Balancer for Settings.
...
Contributed by Ayush Saxena.
2018-09-17 10:08:23 -07:00
a1de8cbac5
HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng.
2018-09-14 05:22:56 +08:00
e1b242a984
HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang.
2018-09-13 09:16:58 -07:00
78bd3b1db9
HDFS-13846. Safe blocks counter is not decremented correctly if the block is striped
...
(Contributed by Kitti Nanasi via Daniel Templeton)
Change-Id: Id41747a67dc946fdf0dbde90643bb1ea7e9e0f70
2018-09-12 11:18:55 -07:00
b2432d254c
HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang.
2018-09-12 06:30:57 +05:30
8e54da1511
HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp.
2018-09-10 22:34:23 -07:00
96892c469b
HDFS-13237. [Documentation] RBF: Mount points across multiple subclusters. Contributed Íñigo Goiri
2018-09-11 10:12:34 +05:30
0da49642fc
HDFS-13884. Improve the description of the setting dfs.image.compress. Contributed by Ryan Wu.
2018-09-10 13:57:36 +08:00
eef3bafae8
HDFS-13895. EC: Fix Intermittent Failure in TestDFSStripedOutputStreamWithFailureWithRandomECPolicy. Contributed by Ayush Saxena.
2018-09-10 09:22:59 +05:30
30eceec342
HDFS-13806. EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory. Contributed by Ayush Saxena.
2018-09-10 09:10:51 +05:30
8944407d93
HDFS-13862. RBF: Router logs are not capturing few of the dfsrouteradmin commands. Contributed by Ayush Saxena
2018-09-09 23:20:26 +05:30
335a8139f5
HDFS-13820. Add an ability to disable CacheReplicationMonitor. Contributed by Hrishikesh Gadre.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-09-07 14:59:06 -07:00
410dd3faa5
HDFS-13744. OIV tool should better handle control characters present in file or directory names. Contributed by Zsolt Venczel.
2018-09-07 12:39:33 -06:00
527288ef89
HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng.
2018-09-06 16:47:54 -07:00
eca1a4bfe9
HDFS-13695. Move logging to slf4j in HDFS package. Contributed by Ian Pickering.
2018-09-06 14:48:00 -07:00
9af96d4ed4
HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak.
2018-09-05 10:50:25 -07:00
e780556ae9
HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang.
2018-09-05 09:52:35 -07:00
9315db5f5d
HDFS-13815. RBF: Add check to order command. Contributed by Ranith Sardar.
2018-09-05 23:33:27 +08:00
6ccb809c2d
HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-09-04 21:53:42 -07:00
54f2044595
HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng.
2018-09-04 12:17:17 -07:00
6e5ffb74dd
HDFS-13885. Add debug logs in dfsclient around decrypting EDEK. Contributed by Kitti Nanasi.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-09-03 22:37:37 -07:00
3801436e49
HDFS-13774. EC: 'hdfs ec -getPolicy' is not retrieving policy details when the special REPLICATION policy set on the directory. Contributed by Ayush Saxena.
2018-09-03 14:37:57 +05:30
780df9034f
HDFS-13867. RBF: Add validation for max arguments for Router admin ls, clrQuota, setQuota, rm and nameservice commands. Contributed by Ayush Saxena.
2018-09-03 14:28:31 +05:30
8aa6c4f079
Revert "HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng."
...
This reverts commit 26c2a97c56
2018-08-30 11:44:20 -07:00
c36d69a7b3
HDFS-13027. Handle possible NPEs due to deleted blocks in race condition. Contributed by Vinayakumar B.
...
(cherry picked from commit 65977e5d81
2018-08-30 22:15:51 +05:30
582cb10ec7
HDFS-13863. FsDatasetImpl should log DiskOutOfSpaceException. Contributed by Fei Hui.
2018-08-30 11:21:13 +08:00
a0ebb6b39f
HDFS-13634. RBF: Configurable value in xml for async connection request queue size. Contributed by CR Hota.
2018-08-29 16:15:22 +08:00
64ad0298d4
HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. Contributed by yanghuafeng.
2018-08-29 08:29:50 +05:30
3e18b957eb
HDFS-13731. ReencryptionUpdater fails with ConcurrentModificationException during processCheckpoints. Contributed by Zsolt Venczel.
2018-08-28 15:13:43 -07:00
33f42efc94
HDFS-13837. Enable debug log for LeaseRenewer in TestDistributedFileSystem. Contributed by Shweta.
2018-08-28 13:56:32 -07:00
cb9d371ae2
HDFS-13861. RBF: Illegal Router Admin command leads to printing usage for all commands. Contributed by Ayush Saxena.
2018-08-29 00:29:05 +05:30
75691ad600
HDFS-13858. RBF: Add check to have single valid argument to safemode command. Contributed by Ayush Saxena.
2018-08-28 09:21:07 +05:30
26c2a97c56
HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng.
2018-08-27 16:02:35 -07:00
7b1fa5693e
HDFS-13849. Migrate logging to slf4j in hadoop-hdfs-httpfs, hadoop-hdfs-nfs, hadoop-hdfs-rbf, hadoop-hdfs-native-client. Contributed by Ian Pickering.
2018-08-27 10:18:05 -07:00
b9b964d253
HDFS-13831. Make block increment deletion number configurable. Contributed by Ryan Wu.
2018-08-27 14:55:46 +08:00
a4121c71c2
HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko.
2018-08-24 18:27:30 -07:00
96c4575d73
HDFS-13805. Journal Nodes should allow to format non-empty directories with -force option. Contributed by Surendra Singh Lilhore.
2018-08-24 08:14:57 +05:30
81847392ba
HDFS-13821. RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache. Contributed by Fei Hui.
2018-08-22 11:43:40 +08:00
770d9d9bb6
HDFS-13772. Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled. Contributed by Ayush Saxena
2018-08-21 09:33:19 +05:30
01ff817814
HDFS-13750. RBF: Router ID in RouterRpcClient is always null. Contributed by Takanobu Asanuma.
2018-08-20 23:01:59 +08:00
6425ed27ea
HDFS-13790. RBF: Move ClientProtocol APIs to its own module. Contributed by Chao Sun.
2018-08-20 12:20:29 +05:30
79c97f6a0b
HADOOP-14624. Add GenericTestUtils.DelayAnswer that accept slf4j logger API. Contributed by Ian Pickering and Wenxin He.
2018-08-17 14:40:00 -07:00
a17eed1b87
HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati
2018-08-17 12:25:36 -05:00
fb5b3dce61
Revert "HDFS-13790. RBF: Move ClientProtocol APIs to its own module. Contributed by Chao Sun."
...
This reverts commit fa121eb66b
2018-08-17 08:01:44 -07:00
fa121eb66b
HDFS-13790. RBF: Move ClientProtocol APIs to its own module. Contributed by Chao Sun.
2018-08-17 15:22:55 +05:30
77b015000a
HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor.
2018-08-17 11:10:29 +02:00
c67b0650ea
HDFS-13747. Statistic for list_located_status is incremented incorrectly by listStatusIterator. Contributed by Antal Mihalyi.
2018-08-16 23:14:21 -07:00
1290e3c647
HDFS-10240. Race between close/recoverLease leads to missing block. Contributed by Jinglun, zhouyingchao and Wei-Chiu Chuang.
2018-08-16 16:29:38 -07:00
8512e1a91b
HDFS-13746. Still occasional "Should be different group" failure in TestRefreshUserMappings#testGroupMappingRefresh
...
(Contributed by Siyao Meng via Daniel Templeton)
Change-Id: I9fad1537ace38367a463d9fe67aaa28d3178fc69
2018-08-16 15:00:45 -07:00
6df606f1b4
HDFS-13829. Remove redundant condition judgement in DirectoryScanner#scan. Contributed by liaoyuxiangqin.
2018-08-16 18:44:18 +08:00
7dc79a8b5b
HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel.
2018-08-15 13:53:47 -07:00
ef02f9b233
HDFS-13217. Audit log all EC policy names during addErasureCodingPolicies. Contributed by liaoyuxiangqin.
2018-08-15 09:23:05 -07:00
3e3963b035
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering.
2018-08-16 00:31:59 +09:00
4a5006b1d0
HDFS-13819. TestDirectoryScanner#testDirectoryScannerInFederatedCluster is flaky
...
Change-Id: I1cea6e67fcec72702ad202775dee3373261ac5cd
2018-08-14 17:22:15 -07:00
363bd16e31
HADOOP-14212. Expose SecurityEnabled boolean field in JMX for other services besides NameNode. Contributed by Adam Antal.
2018-08-14 17:19:00 -07:00
cede33997f
HDFS-13788. Update EC documentation about rack fault tolerance. Contributed by Kitti Nanasi.
2018-08-14 11:57:22 -07:00
61a9b4f58b
HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan.
2018-08-14 11:51:27 -07:00
4023eeba05
HDFS-13738. fsck -list-corruptfileblocks has infinite loop if user is not privileged. Contributed by Yuen-Kuei Hsueh.
2018-08-13 17:36:13 -07:00
23854443ef
HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng.
2018-08-13 16:12:37 -07:00
f760a544a7
HDFS-13823. NameNode UI : "Utilities -> Browse the file system -> open a file -> Head the file" is not working. Contributed by Nanda kumar.
2018-08-13 11:35:19 -07:00
Rakesh Radhakrishnan
Surendra Singh Lilhore
Chen Liang
Vinayakumar B
Shweta
Giovanni Matteo Fumarola
Inigo Goiri
Siyao Meng
Sahil Takiar
Arpit Agarwal
Sean Mackrory
Erik Krogen
lys0716
Puleya7
Sahil Takiar
Adam Antal
Akira Ajisaka
Stephen O'Donnell
Gabor Bota
Wei-Chiu Chuang
Lokesh Jain
Uma Maheswara Rao G
Takanobu Asanuma
Ajay Kumar
Sahil Takiar
Yiqun Lin
Brahma Reddy Battula
Konstantin V Shvachko
Yongjun Zhang
Bharat Viswanadham
Kitti Nanasi
Anu Engineer
Steve Loughran
Wangda Tan
Da Zhou
tiwalter
Masatake Iwasaki
Pranay Singh
Dinesh Chitlangia
Weiwei Yang
Sunil G
Chao Sun
Plamen Jeliazkov
Zsolt Venczel
Daniel Templeton
Tsz Wo Nicholas Sze
Ewan Higgs
Xiao Chen
Jason Lowe
Ajay Kumar
Shweta
Hrishikesh Gadre
Nanda kumar
Mingliang Liu
Giovanni Matteo Fumarola
Xiaoyu Yao
Márton Elek