6de1a8eb67
HADOOP-13571. ServerSocketUtil.getPort() should use loopback address, not 0.0.0.0. Contributed by Eric Badger
2020-12-11 20:16:56 +00:00
ad40715690
HDFS-15221. Add checking of effective filesystem during initializing storage locations. Contributed by Yang Yun.
2020-12-07 15:00:19 +05:30
e2c1268ebd
HDFS-15240. Erasure Coding: dirty buffer causes reconstruction block error. Contributed by HuangTao.
2020-12-04 09:20:09 +08:00
8c234fc0d4
HADOOP-17389. KMS should log full UGI principal. ( #2476 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Reviewed-by: Jim Brennan <jbrennan@apache.org>
2020-12-04 09:33:30 +09:00
db73e994ed
HADOOP-16881. KerberosAuthentication does not disconnect HttpURLConnection leading to CLOSE_WAIT cnxns. Contributed by Attila Magyar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-12-03 12:05:20 -08:00
f94e927bfb
HADOOP-17392. Remote exception messages should not include the exception class ( #2486 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-12-03 10:55:51 -06:00
9170eb566b
YARN-10511. Update yarn.nodemanager.env-whitelist value in docs ( #2512 )
...
Reviewed-by: Adam Antal <adamantal@apache.org>
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-12-04 00:14:04 +09:00
44910b53f3
HDFS-15699 Remove lz4 references in vcxproj ( #2498 )
2020-11-29 22:53:54 -08:00
2ce2198287
HADOOP-17394. [JDK 11] Fix error in mvn package -Pdocs ( #2488 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2020-11-26 11:32:10 +09:00
65002c9486
Revert "[JDK 11] Fix error in mvn package -Pdocs ( #2488 )" because JIRA ID is missing.
...
This reverts commit ce5b3d7975
2020-11-26 11:30:53 +09:00
ce5b3d7975
[JDK 11] Fix error in mvn package -Pdocs ( #2488 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2020-11-26 11:27:30 +09:00
ac7045b75f
HADOOP-17313. FileSystem.get to support slow-to-instantiate FS clients. ( #2396 )
...
This adds a semaphore to throttle the number of FileSystem instances which
can be created simultaneously, set in "fs.creation.parallel.count".
This is designed to reduce the impact of many threads in an application calling
FileSystem.get() on a filesystem which takes time to instantiate -for example
to an object where HTTPS connections are set up during initialization.
Many threads trying to do this may create spurious delays by conflicting
for access to synchronized blocks, when simply limiting the parallelism
diminishes the conflict, so speeds up all threads trying to access
the store.
The default value, 64, is larger than is likely to deliver any speedup -but
it does mean that there should be no adverse effects from the change.
If a service appears to be blocking on all threads initializing connections to
abfs, s3a or store, try a smaller (possibly significantly smaller) value.
Contributed by Steve Loughran.
2020-11-25 14:31:02 +00:00
8fa699b53f
HDFS-15635. ViewFileSystemOverloadScheme support specifying mount table loader imp through conf ( #2389 ). Contributed by Junfan Zhang.
2020-11-19 20:21:16 -08:00
07050339e0
HADOOP-17367. Add InetAddress api to ProxyUsers.authorize ( #2449 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-11-19 14:37:14 -06:00
34aa6137bd
HADOOP-17292. Using lz4-java in Lz4Codec ( #2350 )
...
Contributed by Liang-Chi Hsieh.
2020-11-18 12:03:25 -08:00
ce7827c82a
HADOOP-17318. Support concurrent S3A commit jobs with same app attempt ID. ( #2399 )
...
See also [SPARK-33402]: Jobs launched in same second have duplicate MapReduce JobIDs
Contributed by Steve Loughran.
Change-Id: Iae65333cddc84692997aae5d902ad8765b45772a
2020-11-18 13:34:51 +00:00
e3c08f285a
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2310 )
...
This fixes the S3Guard/Directory Marker Retention integration so that when
fs.s3a.directory.marker.retention=keep, failures during multipart delete
are handled correctly, as are incremental deletes during
directory tree operations.
In both cases, when a directory marker with children is deleted from
S3, the directory entry in S3Guard is not deleted, because it is still
critical to representing the structure of the store.
Contributed by Steve Loughran.
Change-Id: I4ca133a23ea582cd42ec35dbf2dc85b286297d2f
2020-11-18 12:18:11 +00:00
ebe1d1fbf7
HADOOP-17362. reduce RPC calls doing ls on HAR file ( #2444 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-11-13 14:22:35 -06:00
5ce18101cb
HADOOP-17346. Fair call queue is defeated by abusive service principals ( #2431 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
2020-11-12 13:13:12 -06:00
6f10a0506f
HADOOP-17365. Contract test for renaming over existing file is too lenient ( #2447 )
...
Contributed by Attila Doroszlai.
2020-11-11 21:20:09 +00:00
71071e5c0f
HADOOP-17358. Improve excessive reloading of Configurations ( #2436 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
2020-11-11 12:39:03 -08:00
45434c93e8
HADOOP-17096. Fix ZStandardCompressor input buffer offset ( #2104 ). Contributed by Stephen Jung (Stripe).
2020-11-10 11:38:22 -08:00
f1d5652359
HDFS-15668. RBF: Fix RouterRPCMetrics annocation and document misplaced error. Contributed by Hongbing Wang.
2020-11-10 13:09:03 +08:00
1ea3f74246
HADOOP-17360. Log the remote address for authentication success ( #2441 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
2020-11-09 14:05:08 -06:00
ae7b00a998
HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. ( #2427 ). Contributed by Steve Loughran.
2020-11-07 09:43:24 +05:30
6eacaffeea
HDFS-15623. Respect configured values of rpc.engine ( #2403 ) Contributed by Hector Chaverri.
2020-11-06 14:25:35 -08:00
af389d9897
HADOOP-17342. Creating a token identifier should not do kerberos name
...
resolution. Contributed by Jim Brennan.
2020-11-05 21:46:14 +00:00
638f1fc2b6
Revert "HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 ( #2387 )"
...
This reverts commit d259928035
2020-11-05 17:11:39 +00:00
7e52c0975f
Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. ( #2291 )"
...
This reverts commit 7f5caca04c
2020-11-04 16:18:03 -08:00
7f5caca04c
HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. ( #2291 )
...
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-11-03 11:20:23 -08:00
f17e067d52
HADOOP-17329. mvn site commands fails due to MetricsSystemImpl changes. Contributed by Xiaoqiao He.
2020-10-29 07:19:49 +05:30
da93cd962c
MAPREDUCE-7302. Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail. Contributed by Peter Bacsko. Reviewed by Akira Ajisaka.
2020-10-28 10:35:30 +01:00
872440610f
HADOOP-17328. LazyPersist Overwrite fails in direct write mode. ( #2413 )
2020-10-27 01:37:37 +09:00
d259928035
HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 ( #2387 )
2020-10-23 11:30:02 +05:30
6a9ceedfb3
HADOOP-17175. [JDK 11] Fix javadoc errors in hadoop-common module. ( #2397 )
2020-10-23 03:15:45 +09:00
7b4359657f
HADOOP-17315. Use shaded guava in ClientCache.java ( #2398 )
...
Added checkstyle rules to warn guava imports
2020-10-20 23:25:24 +09:00
30f06e0c74
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. ( #2394 )
2020-10-20 19:08:59 +09:00
31477a224f
YARN-10460. Upgrading to JUnit 4.13 causes tests in TestNodeStatusUpdater to fail. Contributed by Peter Bacsko
2020-10-19 13:17:51 +02:00
4c651103f2
HADOOP-17310. Touch command with -c option is broken. ( #2393 ). Contributed by Ayush Saxena.
2020-10-19 10:54:18 +05:30
42e7e0380f
HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina.
2020-10-18 18:37:46 +05:30
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. ( #2342 ). Contributed by Ayush Saxena.
2020-10-17 12:01:18 +05:30
ddc0ee27fa
HADOOP-17304. KMS ACL: Allow DeleteKey Operation to Invalidate Cache. Contributed by Xiaoyu.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-10-14 16:00:37 +08:00
59f01a548e
HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same
...
Contributed by Gabor Bota.
2020-10-13 16:17:44 +01:00
b3786d6c3c
HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.
2020-10-12 17:26:49 -07:00
518a212cff
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer ( #2363 )
2020-10-09 16:12:22 +08:00
52db86b0bb
HADOOP-17021. Add concat fs command ( #1993 )
...
Contributed by Jinglun
2020-10-08 10:36:07 +01:00
82522d60fb
HADOOP-17281 Implement FileSystem.listStatusIterator() in S3AFileSystem ( #2354 )
...
Contains HADOOP-17300: FileSystem.DirListingIterator.next() call should
return NoSuchElementException
Contributed by Mukund Thakur
2020-10-07 13:59:06 +01:00
c9ea344f98
HADOOP-17125. Use snappy-java in SnappyCodec ( #2297 )
...
This switches the SnappyCodec to use the java-snappy codec, rather than the native one.
To use the codec, snappy-java.jar (from org.xerial.snappy) needs to be on the classpath.
This comesin as an avro dependency, so it is already on the hadoop-common classpath,
as well as in hadoop-common/lib.
The version used is now managed in the hadoop-project POM; initially 1.1.7.7
Contributed by DB Tsai and Liang-Chi Hsieh
2020-10-06 17:07:54 +01:00
d0d10f7e8f
HADOOP-17276. Extend CallerContext to make it include many items ( #2327 )
2020-10-04 02:02:28 +09:00
d68d2a5c1e
HADOOP-17287. Support new Instance by non default constructor by ReflectionUtils ( #2341 )
2020-09-30 14:22:55 -07:00
Jim Brennan
Ayush Saxena
Hui Fei
Ahmed Hussein
Attila Magyar
Andrea Scarpino
Gautham B A
Akira Ajisaka
Steve Loughran
zhang_jf
Liang-Chi Hsieh
Doroszlai, Attila
Stephen Jung
hchaverr
Eric Badger
Wei-Chiu Chuang
Sunil G
Peter Bacsko
Vinayakumar B
Takanobu Asanuma
Szilard Nemeth
hemanthboyina
He Xiaoqiao
Gabor Bota
Konstantin V Shvachko
Hui Fei
Jinglun
Mukund Thakur
maobaolong