45434c93e8
HADOOP-17096. Fix ZStandardCompressor input buffer offset ( #2104 ). Contributed by Stephen Jung (Stripe).
2020-11-10 11:38:22 -08:00
f1d5652359
HDFS-15668. RBF: Fix RouterRPCMetrics annocation and document misplaced error. Contributed by Hongbing Wang.
2020-11-10 13:09:03 +08:00
1ea3f74246
HADOOP-17360. Log the remote address for authentication success ( #2441 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
2020-11-09 14:05:08 -06:00
ae7b00a998
HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. ( #2427 ). Contributed by Steve Loughran.
2020-11-07 09:43:24 +05:30
6eacaffeea
HDFS-15623. Respect configured values of rpc.engine ( #2403 ) Contributed by Hector Chaverri.
2020-11-06 14:25:35 -08:00
af389d9897
HADOOP-17342. Creating a token identifier should not do kerberos name
...
resolution. Contributed by Jim Brennan.
2020-11-05 21:46:14 +00:00
638f1fc2b6
Revert "HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 ( #2387 )"
...
This reverts commit d259928035
2020-11-05 17:11:39 +00:00
7e52c0975f
Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. ( #2291 )"
...
This reverts commit 7f5caca04c
2020-11-04 16:18:03 -08:00
7f5caca04c
HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. ( #2291 )
...
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-11-03 11:20:23 -08:00
f17e067d52
HADOOP-17329. mvn site commands fails due to MetricsSystemImpl changes. Contributed by Xiaoqiao He.
2020-10-29 07:19:49 +05:30
da93cd962c
MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka.
2020-10-28 10:35:30 +01:00
872440610f
HADOOP-17328. LazyPersist Overwrite fails in direct write mode. ( #2413 )
2020-10-27 01:37:37 +09:00
d259928035
HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 ( #2387 )
2020-10-23 11:30:02 +05:30
6a9ceedfb3
HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module. ( #2397 )
2020-10-23 03:15:45 +09:00
7b4359657f
HADOOP-17315. Use shaded guava in ClientCache.java ( #2398 )
...
Added checkstyle rules to warn guava imports
2020-10-20 23:25:24 +09:00
30f06e0c74
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. ( #2394 )
2020-10-20 19:08:59 +09:00
31477a224f
YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko
2020-10-19 13:17:51 +02:00
4c651103f2
HADOOP-17310. Touch command with -c option is broken. ( #2393 ). Contributed by Ayush Saxena.
2020-10-19 10:54:18 +05:30
42e7e0380f
HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina.
2020-10-18 18:37:46 +05:30
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. ( #2342 ). Contributed by Ayush Saxena.
2020-10-17 12:01:18 +05:30
ddc0ee27fa
HADOOP-17304. KMS ACL: Allow DeleteKey Operation to Invalidate Cache. Contributed by Xiaoyu.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-10-14 16:00:37 +08:00
59f01a548e
HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same
...
Contributed by Gabor Bota.
2020-10-13 16:17:44 +01:00
b3786d6c3c
HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.
2020-10-12 17:26:49 -07:00
518a212cff
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer ( #2363 )
2020-10-09 16:12:22 +08:00
52db86b0bb
HADOOP-17021. Add concat fs command ( #1993 )
...
Contributed by Jinglun
2020-10-08 10:36:07 +01:00
82522d60fb
HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem ( #2354 )
...
Contains HADOOP-17300: FileSystem.DirListingIterator.next() call should
return NoSuchElementException
Contributed by Mukund Thakur
2020-10-07 13:59:06 +01:00
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec ( #2297 )
...
This switches the SnappyCodec to use the java-snappy codec, rather than the native one.
To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.
This comesin as an avro dependency, so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7
Contributed by DB Tsai and Liang-Chi Hsieh
2020-10-06 17:07:54 +01:00
d0d10f7e8f
HADOOP-17276. Extend CallerContext to make it include many items ( #2327 )
2020-10-04 02:02:28 +09:00
d68d2a5c1e
HADOOP-17287. Support new Instance by non default constructor by ReflectionUtils ( #2341 )
2020-09-30 14:22:55 -07:00
a490d87eb7
HADOOP-17280. Service-user cost shouldn't be accumulated to totalDecayedCallCost and totalRawCallCost. Contributed by Jinglun.
2020-09-30 12:12:52 +08:00
4c5ad57818
HADOOP-17284. Support BCFKS keystores for Hadoop Credential Provider. ( #2334 )
2020-09-29 20:15:27 -07:00
75d10f8499
HADOOP-17267. Add debug-level logs in Filesystem.close() ( #2321 )
...
When a filesystem is closed, the FileSystem log will, at debug level,
log the method calling close/closeAll.
At trace level: the full calling stack.
Contributed by Karen Coppage.
2020-09-29 16:06:50 +01:00
82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng
2020-09-27 19:27:13 +05:30
899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. ( #2339 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-25 21:21:01 -07:00
ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS ( #2189 )
...
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
2020-09-24 14:27:04 +05:30
474fa80bfb
HADOOP-17277. Correct spelling errors for separator ( #2322 )
...
Contributed by Hui Fei.
2020-09-22 17:10:33 +01:00
6b5d9e2334
HADOOP-17259. Allow SSLFactory fallback to input config if ssl-*.xml … ( #2301 )
2020-09-21 12:41:06 -07:00
c8c1cc43d3
HADOOP-17088.Failed to load XInclude files with relative path. ( #2097 )
...
Contributed by Yushi Hayasaka.
2020-09-21 19:10:51 +01:00
364b958085
HADOOP-17270. Fix testCompressorDecompressorWithExeedBufferLimit to c… ( #2311 )
2020-09-20 00:47:02 +09:00
fc2435cb5c
HADOOP-15136. Correct typos in filesystem.md ( #2314 )
...
Contributed by David Tucker
2020-09-18 18:28:14 +01:00
6adf8462ba
HADOOP-17208. LoadBalanceKMSClientProvider#deleteKey should invalidateCache via all KMSClientProvider instances. ( #2259 )
2020-09-17 10:39:19 -07:00
e4cb0d3514
HDFS-15578: Fix the rename issues with fallback fs enabled ( #2305 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-16 22:43:00 -07:00
d2779de3f5
HDFS-15532: listFiles on root/InternalDir will fail if fallback root has file. ( #2298 ). Contributed by Uma Maheswara Rao G.
2020-09-12 17:06:39 -07:00
958cab804e
Revert "HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )"
...
This reverts commit 9960c01a25
2020-09-11 18:07:49 +01:00
56ebabd426
HADOOP-17222. Create socket address leveraging URI cache ( #2241 )
...
Contributed by fanrui.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-10 22:30:52 -07:00
12a316cdf9
HADOOP-15891. provide Regex Based Mount Point In Inode Tree ( #2185 ). Contributed by Zhenzhao Wang.
...
Co-authored-by: Zhenzhao Wang <zhenzhaowang@gmail.com>
2020-09-10 21:20:32 -07:00
9960c01a25
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )
...
This changes directory tree deletion so that only files are incrementally deleted
from S3Guard after the objects are deleted; the directories are left alone
until metadataStore.deleteSubtree(path) is invoke.
This avoids directory tombstones being added above files/child directories,
which stop the treewalk and delete phase from working.
Also:
* Callback to delete objects splits files and dirs so that
any problems deleting the dirs doesn't trigger s3guard updates
* New statistic to measure #of objects deleted, alongside request count.
* Callback listFilesAndEmptyDirectories renamed listFilesAndDirectoryMarkers
to clarify behavior.
* Test enhancements to replicate the failure and verify the fix
Contributed by Steve Loughran
2020-09-10 17:03:52 +01:00
e5fe326270
HADOOP-17165. Implement service-user feature in DecayRPCScheduler. ( #2240 )
2020-09-09 09:56:58 -07:00
2029556dbb
HADOOP-17181. Handle transient stream read failures in FileSystem contract tests ( #2286 )
...
Contributed by Steve Loughran.
* Fixes AbstractContractSeekTest test to use readFully
* Doesn't do this to AbstractContractUnbufferTest test as it changes the test too much.
Instead just notes in the error that this may be transient
The issue is that read(buffer) doesn't guarantee that the buffer is filled, only that it will
read up to a point, and that may be just be the amount of data left in the TCP packet.
readFully corrects for this, but using it in the unbuffer test runs the risk that what
is tested for in terms of unbuffering doesn't actually get validated.
2020-09-09 11:58:26 +01:00
8a3952436c
HADOOP-17245. Add OzoneFileSystem classes to core-default.xml. ( #2276 )
2020-09-04 18:38:48 -07:00
Stephen Jung
Hui Fei
Ahmed Hussein
Steve Loughran
hchaverr
Eric Badger
Jim Brennan
Wei-Chiu Chuang
Akira Ajisaka
Sunil G
Peter Bacsko
Ayush Saxena
Vinayakumar B
Takanobu Asanuma
Szilard Nemeth
hemanthboyina
He Xiaoqiao
Gabor Bota
Konstantin V Shvachko
Hui Fei
Jinglun
Mukund Thakur
Liang-Chi Hsieh
maobaolong
Xiaoyu Yao
Karen Coppage
Uma Maheswara Rao G
huangtianhua
crossfire
Masatake Iwasaki
David Tucker
1996fanrui
zz
Bharat Viswanadham