49ddd8a6ed
HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-05 15:14:19 -08:00
308f3168fa
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
...
Contributed by Da Zhou and Junhua Gu.
2019-02-05 19:23:15 +00:00
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
...
S3A to implement S3 Select through this API.
The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.
The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.
This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.
In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).
Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
5f15a60e32
HDFS-14125. Use parameterized log format in ECTopologyVerifier. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-04 14:52:07 -08:00
0e79a86582
HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar.
2019-02-04 11:59:48 -08:00
9aa3dc872c
HDFS-14158. Checkpointer ignores configured time period > 5 minutes
...
This closes #449
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-02-04 13:51:07 +09:00
ec77e95bbd
HDFS-14232. libhdfs is not included in binary tarball. Contributed by Akira Ajisaka.
2019-02-04 10:12:09 +09:00
0ab7fc9200
HDFS-14187. Make warning message more clear when there are not enough data nodes for EC write. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-31 16:07:24 -08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
d1714c20e9
Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
...
This reverts commit 1d523279da
2019-01-29 15:43:09 -08:00
dc5af4c6b7
HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki.
2019-01-26 05:34:32 +09:00
1d523279da
HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 09:02:57 -08:00
45caeee6cf
HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 07:37:01 -08:00
f3e642d92b
HDFS-14185. Cleanup method calls to static Assert methods in TestAddStripedBlocks
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I0d533f575a405ed3affd05994a4208bde7d9cbe9
2019-01-23 15:57:39 -08:00
951cdd7e4c
HDFS-14061. Check if the cluster topology supports the EC policy before setting, enabling or adding it. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-23 14:40:57 -08:00
221e308cb5
HDFS-14153. [SPS] : Add Support for Storage Policy Satisfier in WEBHDFS. Contributed by Ayush Saxena.
2019-01-23 17:00:37 +05:30
7d6792e5d2
HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena.
2019-01-23 10:03:06 +09:00
6f0756fc0e
HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal.
2019-01-21 20:45:09 -08:00
1ff658b2ef
HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal.
2019-01-21 20:44:37 -08:00
de34fc148c
HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui.
2019-01-22 12:01:08 +08:00
e9962240f0
Make 3.2.0 aware to other branches - jdiff
2019-01-21 22:41:26 +05:30
f048512bb8
HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena.
2019-01-16 10:14:22 -08:00
fb8932a727
HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.
2019-01-11 10:54:49 -08:00
9aeaaa0479
HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena.
2019-01-11 14:36:55 +05:30
4ab5260b7e
HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8
2019-01-08 17:04:57 -08:00
1a08302897
HDFS-14189. Fix intermittent failure of TestNameNodeMetrics. Contributed by Ayush Saxena.
2019-01-08 10:38:06 -08:00
f660e5eaa3
HDFS-14163. Debug Admin Command Should Support Generic Options. Contributed by Ayush Saxena.
2019-01-03 18:58:03 +05:30
21fe77e3ab
HDFS-14184. [SPS] Add support for URI based path in satisfystoragepolicy command. Contributed by Ayush Saxena.
2019-01-03 18:36:53 +05:30
f30473f2ef
HDFS-14183. [SPS] Remove the -w parameter from the -satisfystoragepolicy usage. Contributed by Ayush Saxena.
2018-12-31 03:16:26 +09:00
e9a005dfcf
HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang.
2018-12-28 10:34:18 -08:00
bd2a59e508
HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko.
2018-12-24 09:39:20 -08:00
c546b12e45
HDFS-14154. [SBN read] Document dfs.ha.tail-edits.period in user guide. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
c9d7737431
HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko.
2018-12-24 09:34:01 -08:00
fa8550337d
HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
c3efc299fe
HDFS-14116. [SBN read] Fix class cast error in NNThroughputBenchmark with ObserverReadProxyProvider. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
ef3e1929dd
HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
b73fb70f97
HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko.
2018-12-24 09:34:01 -08:00
b8ad6c85a5
HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
2904aa3808
HDFS-14131. [SBN read] Create user guide for Consistent Reads from Observer feature. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
71cf66e29b
HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
3bb92a1d9a
HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena.
2018-12-24 09:34:01 -08:00
cbc2f932eb
HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
a3aab48df0
HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
652b257478
HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
1a9ba9616a
HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
8c49135078
HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
b6f20c36c2
HDFS-13925. Unit Test for transitioning between different states. Contributed by Sherwood Zheng.
2018-12-24 09:34:00 -08:00
1c4265d7bc
HDFS-13523. Support observer nodes in MiniDFSCluster. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
b5b9b77707
HDFS-13961. [SBN read] TestObserverNode refactoring. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
Kitti Nanasi
Wangda Tan
Da Zhou
Steve Loughran
Inigo Goiri
tiwalter
Takanobu Asanuma
Akira Ajisaka
Wei-Chiu Chuang
Masatake Iwasaki
Pranay Singh
Dinesh Chitlangia
shwetayakkali@cloudera.com
Surendra Singh Lilhore
Arpit Agarwal
Weiwei Yang
Sunil G
Giovanni Matteo Fumarola
Konstantin V Shvachko
Chao Sun
Erik Krogen
Chen Liang