ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS ( #2189 )
...
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
2020-09-24 14:27:04 +05:30
474fa80bfb
HADOOP-17277. Correct spelling errors for separator ( #2322 )
...
Contributed by Hui Fei.
2020-09-22 17:10:33 +01:00
6b5d9e2334
HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … ( #2301 )
2020-09-21 12:41:06 -07:00
c8c1cc43d3
HADOOP-17088.Failed to load XInclude files with relative path. ( #2097 )
...
Contributed by Yushi Hayasaka.
2020-09-21 19:10:51 +01:00
364b958085
HADOOP-17270. Fix testCompressorDecompressorWithExeedBufferLimit to c… ( #2311 )
2020-09-20 00:47:02 +09:00
fc2435cb5c
HADOOP-15136. Correct typos in filesystem.md ( #2314 )
...
Contributed by David Tucker
2020-09-18 18:28:14 +01:00
6adf8462ba
HADOOP-17208. LoadBalanceKMSClientProvider#deleteKey should invalidateCache via all KMSClientProvider instances. ( #2259 )
2020-09-17 10:39:19 -07:00
e4cb0d3514
HDFS-15578: Fix the rename issues with fallback fs enabled ( #2305 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-16 22:43:00 -07:00
d2779de3f5
HDFS-15532: listFiles on root/InternalDir will fail if fallback root has file. ( #2298 ). Contributed by Uma Maheswara Rao G.
2020-09-12 17:06:39 -07:00
958cab804e
Revert "HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )"
...
This reverts commit 9960c01a25
2020-09-11 18:07:49 +01:00
56ebabd426
HADOOP-17222. Create socket address leveraging URI cache ( #2241 )
...
Contributed by fanrui.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-10 22:30:52 -07:00
12a316cdf9
HADOOP-15891. provide Regex Based Mount Point In Inode Tree ( #2185 ). Contributed by Zhenzhao Wang.
...
Co-authored-by: Zhenzhao Wang <zhenzhaowang@gmail.com>
2020-09-10 21:20:32 -07:00
9960c01a25
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )
...
This changes directory tree deletion so that only files are incrementally deleted
from S3Guard after the objects are deleted; the directories are left alone
until metadataStore.deleteSubtree(path) is invoke.
This avoids directory tombstones being added above files/child directories,
which stop the treewalk and delete phase from working.
Also:
* Callback to delete objects splits files and dirs so that
any problems deleting the dirs doesn't trigger s3guard updates
* New statistic to measure #of objects deleted, alongside request count.
* Callback listFilesAndEmptyDirectories renamed listFilesAndDirectoryMarkers
to clarify behavior.
* Test enhancements to replicate the failure and verify the fix
Contributed by Steve Loughran
2020-09-10 17:03:52 +01:00
e5fe326270
HADOOP-17165. Implement service-user feature in DecayRPCScheduler. ( #2240 )
2020-09-09 09:56:58 -07:00
2029556dbb
HADOOP-17181. Handle transient stream read failures in FileSystem contract tests ( #2286 )
...
Contributed by Steve Loughran.
* Fixes AbstractContractSeekTest test to use readFully
* Doesn't do this to AbstractContractUnbufferTest test as it changes the test too much.
Instead just notes in the error that this may be transient
The issue is that read(buffer) doesn't guarantee that the buffer is filled, only that it will
read up to a point, and that may be just be the amount of data left in the TCP packet.
readFully corrects for this, but using it in the unbuffer test runs the risk that what
is tested for in terms of unbuffering doesn't actually get validated.
2020-09-09 11:58:26 +01:00
8a3952436c
HADOOP-17245. Add OzoneFileSystem classes to core-default.xml. ( #2276 )
2020-09-04 18:38:48 -07:00
b3660d0147
HDFS-15529: getChildFilesystems should include fallback fs as well ( #2234 ). Contributed by Uma Maheswara Rao G.
2020-09-03 11:06:20 -07:00
7e671f3b65
HADOOP-17235. Erasure Coding: Remove dead code from common side. Contributed by Fei Hui.
2020-09-01 11:35:55 +08:00
c392d9022a
HDFS-15471. TestHDFSContractMultipartUploader failing ( #2252 )
...
Contributed by Steve Loughran
(Was: broken by Steve Loughran)
2020-08-28 15:39:09 +01:00
bfd99f31fd
HADOOP-17232. Erasure Coding: Typo in document. Contributed by Fei Hui.
2020-08-28 17:38:23 +05:30
d8aaa8c338
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time ( #2249 )
...
Contributed by Sandeep Guggilam.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
2020-08-26 23:45:21 -07:00
5e52955112
Revert "HADOOP-17159 Ability for forceful relogin in UserGroupInformation class ( #2197 )"
...
This reverts commit a932796d0c
2020-08-26 10:41:10 -07:00
a932796d0c
HADOOP-17159 Ability for forceful relogin in UserGroupInformation class ( #2197 )
...
Contributed by Sandeep Guggilam.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
2020-08-24 23:39:57 -07:00
960fb0aa4f
HADOOP-16925. MetricsConfig incorrectly loads the configuration whose value is String list in the properties file ( #1896 )
...
Contributed by Jiayi Liu
2020-08-24 14:01:48 +01:00
17cd8a1b16
HADOOP-17209. Erasure Coding: Native library memory leak. Contriubted by Sean Chow
2020-08-24 12:00:35 +01:00
dd013f2fdf
HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. ( #2229 ). Contributed by Uma Maheswara Rao G.
2020-08-19 09:30:41 -07:00
5092ea62ec
HADOOP-13230. S3A to optionally retain directory markers.
...
This adds an option to disable "empty directory" marker deletion,
so avoid throttling and other scale problems.
This feature is *not* backwards compatible.
Consult the documentation and use with care.
Contributed by Steve Loughran.
Change-Id: I69a61e7584dc36e485d5e39ff25b1e3e559a1958
2020-08-15 12:51:08 +01:00
141c62584b
HADOOP-17204. Fix typo in Hadoop KMS document. Contributed by Xieming Li.
2020-08-12 16:08:36 +09:00
3fd3aeb621
HDFS-15492. Make trash root inside each snapshottable directory ( #2176 )
2020-08-11 08:52:16 -07:00
909f1e82d3
HADOOP-17196. Fix C/C++ standard warnings ( #2208 )
...
* Passing C/C++ standard flags -std is
not cross-compiler friendly as not all
compilers support all values.
* Thus, we need to make use of the
appropriate flags provided by CMake in
order to specify the C/C++ standards.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-08-11 16:35:08 +09:00
8955a6ceb7
HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false ( #2205 )
...
* HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false
* Used LambdaTestUtils#intercept in test
2020-08-11 00:01:58 -07:00
2986058e7f
HADOOP-17164. UGI loginUserFromKeytab doesn't set the last login time ( #2178 )
...
Contributed by Sandeep Guggilam.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
2020-08-04 10:30:06 -07:00
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors ( #2098 )
2020-08-03 10:46:51 +09:00
e60096c377
HADOOP-17141. Add Capability To Get Text Length ( #2157 )
...
Contributed by David Mollitor
2020-07-24 10:37:28 +01:00
d5b4766158
HADOOP-17147. Dead link in hadoop-kms/index.md.vm. Contributed by Xieming Li.
2020-07-23 00:39:49 +09:00
ac9a07b51a
HDFS-15478: When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target fs. ( #2160 ). Contributed by Uma Maheswara Rao G.
2020-07-21 23:29:10 -07:00
1b29c9bfee
HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. ( #2155 )
2020-07-22 13:40:20 +09:00
3833c616e0
HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang.
2020-07-20 12:49:58 -07:00
f2033de234
HADOOP-17119. Jetty upgrade to 9.4.x causes MR app fail with IOException. Contributed by Bilwa S T.
2020-07-20 22:13:48 +05:30
6bcb24d269
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-18 14:33:43 +05:30
b21cb91c7f
HADOOP-17130. Configuration.getValByRegex() shouldn't be updating the results while fetching. ( #2142 )
...
Contributed by Mukund Thakur
2020-07-16 18:09:59 +01:00
1f71c4ae71
HADOOP-17099. Replace Guava Predicate with Java8+ Predicate
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 11:39:06 -05:00
317fe4584a
HADOOP-17127. Use RpcMetrics.TIMEUNIT to initialize rpc queueTime and processingTime. Contributed by Jim Brennan.
2020-07-14 11:22:25 -07:00
e62d8f8412
HADOOP-17116. Skip Retry INFO logging on first failover from a proxy
2020-07-13 12:55:34 -07:00
b9fa5e0182
HDFS-13934. Multipart uploaders to be created through FileSystem/FileContext.
...
Contributed by Steve Loughran.
Change-Id: Iebd34140c1a0aa71f44a3f4d0fee85f6bdf123a3
2020-07-13 13:30:02 +01:00
84b74b335c
HDFS-15447 RBF: Add top real owners metrics for delegation tokens ( #2110 )
2020-07-12 00:10:12 -07:00
3e70006639
HDFS-15464: ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links ( #2132 ). Contributed by Uma Maheswara Rao G.
2020-07-11 23:50:04 -07:00
0e694b20b9
HDFS-15462. Add fs.viewfs.overload.scheme.target.ofs.impl to core-default.xml ( #2131 )
2020-07-09 12:38:52 -07:00
f91a8ad88b
HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. ( #2085 )
2020-07-09 11:33:37 -07:00
dc0626b5f2
HDFS-15449. Optionally ignore port number in mount-table name when picking from initialized uri. Contributed by Uma Maheswara Rao G.
2020-07-06 18:50:03 -07:00
huangtianhua
Hui Fei
Xiaoyu Yao
crossfire
Masatake Iwasaki
David Tucker
Uma Maheswara Rao G
Steve Loughran
1996fanrui
zz
Takanobu Asanuma
Bharat Viswanadham
He Xiaoqiao
Ayush Saxena
sguggilam
Mingliang Liu
Joey
S O'Donnell
Akira Ajisaka
Siyao Meng
Gautham B A
belugabehr
Chen Liang
Mukund Thakur
Ahmed Hussein
Erik Krogen
Hanisha Koneru
lfengnan