ddc0ee27fa
HADOOP-17304. KMS ACL: Allow DeleteKey Operation to Invalidate Cache. Contributed by Xiaoyu.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-10-14 16:00:37 +08:00
59f01a548e
HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same
...
Contributed by Gabor Bota.
2020-10-13 16:17:44 +01:00
b3786d6c3c
HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.
2020-10-12 17:26:49 -07:00
518a212cff
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer ( #2363 )
2020-10-09 16:12:22 +08:00
52db86b0bb
HADOOP-17021. Add concat fs command ( #1993 )
...
Contributed by Jinglun
2020-10-08 10:36:07 +01:00
82522d60fb
HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem ( #2354 )
...
Contains HADOOP-17300: FileSystem.DirListingIterator.next() call should
return NoSuchElementException
Contributed by Mukund Thakur
2020-10-07 13:59:06 +01:00
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec ( #2297 )
...
This switches the SnappyCodec to use the java-snappy codec, rather than the native one.
To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.
This comesin as an avro dependency, so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7
Contributed by DB Tsai and Liang-Chi Hsieh
2020-10-06 17:07:54 +01:00
d0d10f7e8f
HADOOP-17276. Extend CallerContext to make it include many items ( #2327 )
2020-10-04 02:02:28 +09:00
d68d2a5c1e
HADOOP-17287. Support new Instance by non default constructor by ReflectionUtils ( #2341 )
2020-09-30 14:22:55 -07:00
a490d87eb7
HADOOP-17280. Service-user cost shouldn't be accumulated to totalDecayedCallCost and totalRawCallCost. Contributed by Jinglun.
2020-09-30 12:12:52 +08:00
4c5ad57818
HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider. ( #2334 )
2020-09-29 20:15:27 -07:00
75d10f8499
HADOOP-17267. Add debug-level logs in Filesystem.close() ( #2321 )
...
When a filesystem is closed, the FileSystem log will, at debug level,
log the method calling close/closeAll.
At trace level: the full calling stack.
Contributed by Karen Coppage.
2020-09-29 16:06:50 +01:00
82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng
2020-09-27 19:27:13 +05:30
899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. ( #2339 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-25 21:21:01 -07:00
ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS ( #2189 )
...
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
2020-09-24 14:27:04 +05:30
474fa80bfb
HADOOP-17277. Correct spelling errors for separator ( #2322 )
...
Contributed by Hui Fei.
2020-09-22 17:10:33 +01:00
6b5d9e2334
HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … ( #2301 )
2020-09-21 12:41:06 -07:00
c8c1cc43d3
HADOOP-17088.Failed to load XInclude files with relative path. ( #2097 )
...
Contributed by Yushi Hayasaka.
2020-09-21 19:10:51 +01:00
364b958085
HADOOP-17270. Fix testCompressorDecompressorWithExeedBufferLimit to c… ( #2311 )
2020-09-20 00:47:02 +09:00
fc2435cb5c
HADOOP-15136. Correct typos in filesystem.md ( #2314 )
...
Contributed by David Tucker
2020-09-18 18:28:14 +01:00
6adf8462ba
HADOOP-17208. LoadBalanceKMSClientProvider#deleteKey should invalidateCache via all KMSClientProvider instances. ( #2259 )
2020-09-17 10:39:19 -07:00
e4cb0d3514
HDFS-15578: Fix the rename issues with fallback fs enabled ( #2305 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-16 22:43:00 -07:00
d2779de3f5
HDFS-15532: listFiles on root/InternalDir will fail if fallback root has file. ( #2298 ). Contributed by Uma Maheswara Rao G.
2020-09-12 17:06:39 -07:00
958cab804e
Revert "HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )"
...
This reverts commit 9960c01a25
2020-09-11 18:07:49 +01:00
56ebabd426
HADOOP-17222. Create socket address leveraging URI cache ( #2241 )
...
Contributed by fanrui.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-10 22:30:52 -07:00
12a316cdf9
HADOOP-15891. provide Regex Based Mount Point In Inode Tree ( #2185 ). Contributed by Zhenzhao Wang.
...
Co-authored-by: Zhenzhao Wang <zhenzhaowang@gmail.com>
2020-09-10 21:20:32 -07:00
9960c01a25
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )
...
This changes directory tree deletion so that only files are incrementally deleted
from S3Guard after the objects are deleted; the directories are left alone
until metadataStore.deleteSubtree(path) is invoke.
This avoids directory tombstones being added above files/child directories,
which stop the treewalk and delete phase from working.
Also:
* Callback to delete objects splits files and dirs so that
any problems deleting the dirs doesn't trigger s3guard updates
* New statistic to measure #of objects deleted, alongside request count.
* Callback listFilesAndEmptyDirectories renamed listFilesAndDirectoryMarkers
to clarify behavior.
* Test enhancements to replicate the failure and verify the fix
Contributed by Steve Loughran
2020-09-10 17:03:52 +01:00
e5fe326270
HADOOP-17165. Implement service-user feature in DecayRPCScheduler. ( #2240 )
2020-09-09 09:56:58 -07:00
2029556dbb
HADOOP-17181. Handle transient stream read failures in FileSystem contract tests ( #2286 )
...
Contributed by Steve Loughran.
* Fixes AbstractContractSeekTest test to use readFully
* Doesn't do this to AbstractContractUnbufferTest test as it changes the test too much.
Instead just notes in the error that this may be transient
The issue is that read(buffer) doesn't guarantee that the buffer is filled, only that it will
read up to a point, and that may be just be the amount of data left in the TCP packet.
readFully corrects for this, but using it in the unbuffer test runs the risk that what
is tested for in terms of unbuffering doesn't actually get validated.
2020-09-09 11:58:26 +01:00
8a3952436c
HADOOP-17245. Add OzoneFileSystem classes to core-default.xml. ( #2276 )
2020-09-04 18:38:48 -07:00
b3660d0147
HDFS-15529: getChildFilesystems should include fallback fs as well ( #2234 ). Contributed by Uma Maheswara Rao G.
2020-09-03 11:06:20 -07:00
7e671f3b65
HADOOP-17235. Erasure Coding: Remove dead code from common side. Contributed by Fei Hui.
2020-09-01 11:35:55 +08:00
c392d9022a
HDFS-15471. TestHDFSContractMultipartUploader failing ( #2252 )
...
Contributed by Steve Loughran
(Was: broken by Steve Loughran)
2020-08-28 15:39:09 +01:00
bfd99f31fd
HADOOP-17232. Erasure Coding: Typo in document. Contributed by Fei Hui.
2020-08-28 17:38:23 +05:30
d8aaa8c338
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time ( #2249 )
...
Contributed by Sandeep Guggilam.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
2020-08-26 23:45:21 -07:00
5e52955112
Revert "HADOOP-17159 Ability for forceful relogin in UserGroupInformation class ( #2197 )"
...
This reverts commit a932796d0c
2020-08-26 10:41:10 -07:00
a932796d0c
HADOOP-17159 Ability for forceful relogin in UserGroupInformation class ( #2197 )
...
Contributed by Sandeep Guggilam.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
2020-08-24 23:39:57 -07:00
960fb0aa4f
HADOOP-16925. MetricsConfig incorrectly loads the configuration whose value is String list in the properties file ( #1896 )
...
Contributed by Jiayi Liu
2020-08-24 14:01:48 +01:00
17cd8a1b16
HADOOP-17209. Erasure Coding: Native library memory leak. Contriubted by Sean Chow
2020-08-24 12:00:35 +01:00
dd013f2fdf
HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. ( #2229 ). Contributed by Uma Maheswara Rao G.
2020-08-19 09:30:41 -07:00
5092ea62ec
HADOOP-13230. S3A to optionally retain directory markers.
...
This adds an option to disable "empty directory" marker deletion,
so avoid throttling and other scale problems.
This feature is *not* backwards compatible.
Consult the documentation and use with care.
Contributed by Steve Loughran.
Change-Id: I69a61e7584dc36e485d5e39ff25b1e3e559a1958
2020-08-15 12:51:08 +01:00
141c62584b
HADOOP-17204. Fix typo in Hadoop KMS document. Contributed by Xieming Li.
2020-08-12 16:08:36 +09:00
3fd3aeb621
HDFS-15492. Make trash root inside each snapshottable directory ( #2176 )
2020-08-11 08:52:16 -07:00
909f1e82d3
HADOOP-17196. Fix C/C++ standard warnings ( #2208 )
...
* Passing C/C++ standard flags -std is
not cross-compiler friendly as not all
compilers support all values.
* Thus, we need to make use of the
appropriate flags provided by CMake in
order to specify the C/C++ standards.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-08-11 16:35:08 +09:00
8955a6ceb7
HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false ( #2205 )
...
* HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false
* Used LambdaTestUtils#intercept in test
2020-08-11 00:01:58 -07:00
2986058e7f
HADOOP-17164. UGI loginUserFromKeytab doesn't set the last login time ( #2178 )
...
Contributed by Sandeep Guggilam.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
2020-08-04 10:30:06 -07:00
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors ( #2098 )
2020-08-03 10:46:51 +09:00
e60096c377
HADOOP-17141. Add Capability To Get Text Length ( #2157 )
...
Contributed by David Mollitor
2020-07-24 10:37:28 +01:00
d5b4766158
HADOOP-17147. Dead link in hadoop-kms/index.md.vm. Contributed by Xieming Li.
2020-07-23 00:39:49 +09:00
ac9a07b51a
HDFS-15478: When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target fs. ( #2160 ). Contributed by Uma Maheswara Rao G.
2020-07-21 23:29:10 -07:00
He Xiaoqiao
Gabor Bota
Konstantin V Shvachko
Hui Fei
Jinglun
Mukund Thakur
Liang-Chi Hsieh
maobaolong
Xiaoyu Yao
Karen Coppage
Vinayakumar B
Uma Maheswara Rao G
huangtianhua
crossfire
Masatake Iwasaki
David Tucker
Steve Loughran
1996fanrui
zz
Takanobu Asanuma
Bharat Viswanadham
Ayush Saxena
sguggilam
Mingliang Liu
Joey
S O'Donnell
Akira Ajisaka
Siyao Meng
Gautham B A
belugabehr