e4bc64cce0
HADOOP-17343. Upgrade AWS SDK to 1.11.901 ( #2468 )
...
Contributed by Steve Loughran.
2020-11-23 14:09:14 +00:00
ced08fd87f
MAPREDUCE-7304. Enhance the map-reduce Job end notifier to be able to notify the given URL via a custom class. Contributed by Zoltan Erdmann
2020-11-20 13:14:49 +01:00
401cadbac5
HADOOP-17388. AbstractS3ATokenIdentifier to issue date in UTC. ( #2477 )
...
Followup to HADOOP-17379.
Contributed by Jungtaek Lim.
Change-Id: I7b2fce36028d297c1e095499691a08caba92d9fd
2020-11-20 10:56:57 +00:00
e24a6b550e
HADOOP-17367. Add InetAddress api to ProxyUsers.authorize ( #2449 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-11-19 21:26:47 +00:00
4687c25389
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2310 )
...
This fixes the S3Guard/Directory Marker Retention integration so that when
fs.s3a.directory.marker.retention=keep, failures during multipart delete
are handled correctly, as are incremental deletes during
directory tree operations.
In both cases, when a directory marker with children is deleted from
S3, the directory entry in S3Guard is not deleted, because it is still
critical to representing the structure of the store.
Contributed by Steve Loughran.
Change-Id: I4ca133a23ea582cd42ec35dbf2dc85b286297d2f
2020-11-18 12:30:43 +00:00
4bb9d593da
HADOOP-17261. s3a rename() needs s3:deleteObjectVersion permission ( #2303 )
...
Contributed by Steve Loughran.
Change-Id: I8e89a402a24bd9fb958e0fa93d1a28191093851d
2020-11-18 12:20:12 +00:00
71c106b412
MAPREDUCE-7305. [JDK 11] TestMRJobsWithProfiler fails. ( #2463 )
...
(cherry picked from commit 2045a9df13
2020-11-18 14:16:45 +09:00
22039a14ff
HADOOP-17379. AbstractS3ATokenIdentifier to set issue date == now. ( #2466 )
...
Unless you explicitly set it, the issue date of a delegation token identifier is 0, which confuses spark renewal (SPARK-33440). This patch makes sure that all S3A DT identifiers have the current time as issue date, fixing the problem as far as S3A tokens are concerned.
Contributed by Jungtaek Lim.
Change-Id: Ic80ac7895612a1aa669459c73a78a9c17ecf0c0d
2020-11-17 14:56:58 +00:00
be108159b1
HDFS-15685. [JDK 14] TestConfiguredFailoverProxyProvider#testResolveDomainNameUsingDNS fails. ( #2465 )
...
(cherry picked from commit b57f04cd5b
2020-11-17 11:01:23 +09:00
df4edb99f7
HADOOP-17360. Log the remote address for authentication success ( #2441 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 1ea3f74246
2020-11-16 21:48:37 +00:00
bc634c077b
YARN-10485. TimelineConnector swallows InterruptedException ( #2450 ). Contributed by Ahmed Hussein
...
(cherry picked from commit 0b2510ee1f
2020-11-16 21:09:36 +00:00
bf2ff35a04
HADOOP-17376. ITestS3AContractRename failing against stricter tests. ( #2462 )
...
Contributed by Attila Doroszlai.
Change-Id: Ie15624ec07b1c5e34ca7fde0a72a54431d79e746
2020-11-16 11:26:06 +00:00
75ca0c0f23
HADOOP-17362. reduce RPC calls doing ls on HAR file ( #2444 ). Contributed by Daryn Sharp and Ahmed Hussein
...
(cherry picked from commit ebe1d1fbf7
2020-11-13 21:14:47 +00:00
d78bf82722
HDFS-15538. Fix the documentation for dfs.namenode.replication.max-streams in hdfs-default.xml. Contributed by Xieming Li.
...
(cherry picked from commit f56cd88d7d
2020-11-13 11:05:20 +09:00
23fe3bdab3
HADOOP-17358. Improve excessive reloading of Configurations ( #2436 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0f
2020-11-12 10:35:28 -08:00
47131cdf7c
HADOOP-17365. Contract test for renaming over existing file is too lenient ( #2447 )
...
Contributed by Attila Doroszlai.
Change-Id: I21c29256b52449b7fea335704b3afa02e39c6a39
2020-11-11 21:21:11 +00:00
8e4b1cd192
HADOOP-17373. hadoop-client-integration-tests doesn't work when building with skipShade ( #2458 )
2020-11-11 09:49:20 -08:00
64344fb302
HADOOP-17324. Don't relocate org.bouncycastle in shaded client jars ( #2411 )
...
Contributed by Chao Sun.
2020-11-11 09:48:57 -08:00
0712505b59
HADOOP-17096. Fix ZStandardCompressor input buffer offset ( #2104 ). Contributed by Stephen Jung (Stripe).
...
(cherry picked from commit 45434c93e8
2020-11-10 11:41:21 -08:00
9a3f2bb545
YARN-10480. replace href tags with ng-href ( #2426 )
...
(cherry picked from commit 61f8c5767e
2020-11-10 11:00:00 -08:00
12bfd9194a
HADOOP-17142. Fix outdated properties of JournalNode when performing rollback. Contributed by Deegue.
...
(cherry picked from commit 82de07c794
2020-11-10 09:14:12 -08:00
0361837c9e
YARN-10458. Hive On Tez queries fails upon submission to dynamically created pools. Contributed by Peter Bacsko.
2020-11-10 13:16:52 +01:00
aa3807ed1d
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. ( #2433 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811
2020-11-10 14:00:08 +09:00
7cb5325dda
HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. ( #2427 ). Contributed by Steve Loughran.
2020-11-07 17:05:23 +05:30
043cca01b1
HDFS-15623. Respect configured values of rpc.engine ( #2403 ) Contributed by Hector Chaverri.
...
(cherry picked from commit 6eacaffeea
2020-11-06 14:31:31 -08:00
1e22929885
YARN-10479. RMProxy should retry on SocketTimeout Exceptions. Contributed by Jim Brennan (Jim_Brennan)
...
(cherry picked from commit 55339c2bdd
2020-11-05 22:08:51 +00:00
c6fee0a2c8
HADOOP-17342. Creating a token identifier should not do kerberos name
...
resolution. Contributed by Jim Brennan.
(cherry picked from commit af389d9897
2020-11-05 21:56:46 +00:00
41d58d190d
Revert "HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 ( #2387 )"
...
This reverts commit e21b81276e
2020-11-05 17:31:39 +00:00
cfa0986d00
Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. ( #2291 )"
...
This reverts commit dd1634ec3b
2020-11-04 16:18:23 -08:00
cb11fd8793
HDFS-15643. EC: Fix checksum computation in case of native encoders. ( #2424 ). Contributed by Ayush Saxena.
2020-11-04 09:44:08 +05:30
e48dd9daea
HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit d07dc7afb4
2020-11-03 14:52:05 -08:00
dd1634ec3b
HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. ( #2291 )
...
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04c
2020-11-03 11:22:48 -08:00
2473e8b711
YARN-10475: Scale RM-NM heartbeat interval based on node utilization. Contributed by Jim Brennan (Jim_Brennan).
2020-11-02 17:16:28 +00:00
cdaef111d5
YARN-10471. Prevent logs for any container from becoming larger than a configurable size. Contributed by Eric Payne
...
(cherry picked from commit 8ee6bc2518
2020-10-29 19:52:43 +00:00
91a3d298b9
HADOOP-17329. mvn site commands fails due to MetricsSystemImpl changes. Contributed by Xiaoqiao He.
...
(cherry picked from commit f17e067d52
2020-10-29 07:20:46 +05:30
f95c0824b0
YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen
...
(cherry picked from commit bab5bf9743
2020-10-28 10:33:41 -07:00
fe8b22ca8f
HADOOP-17236. Bump up snakeyaml to 1.26 to mitigate CVE-2017-18640. Contributed by Brahma Reddy Battula.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit eb84793af1
2020-10-28 09:28:09 -07:00
8e0922edb2
HDFS-15657. RBF: TestRouter#testNamenodeHeartBeatEnableDefault fails by BindException ( #2418 )
...
(cherry picked from commit b3ba74d72d
2020-10-28 19:14:19 +09:00
d4129c6794
HDFS-9776. testMultipleAppendsDuringCatchupTailing is flaky ( #2410 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit ae74407ac4
2020-10-27 20:49:15 +09:00
1429c65533
HDFS-15461. TestDFSClientRetries testGetFileChecksum fails ( #2404 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit afaab3d332
2020-10-27 16:58:04 +09:00
af5f90623c
HADOOP-17328. LazyPersist Overwrite fails in direct write mode. ( #2413 )
...
(cherry picked from commit 872440610f
2020-10-27 01:40:25 +09:00
b8cc09d7c7
HDFS-15641. DataNode could meet deadlock if invoke refreshNameNode. Contributed by Hongbing Wang.
...
(cherry picked from commit 12c908c827
2020-10-26 22:25:03 +08:00
8312f230eb
HADOOP-17308. WASB PageBlobOutputStream.flush succeeds even when flush to storage fails ( #2392 )
...
Contributed by Anoop Sam John.
2020-10-26 13:31:53 +00:00
c8eb86ee3d
HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein.
...
(cherry picked from commit 74634eb002
2020-10-23 12:06:23 -07:00
c55d639e8f
HDFS-15459. TestBlockTokenWithDFSStriped fails intermittently. Contributed by Ahmed Hussein.
...
(cherry picked from commit e7aa4da0a2
2020-10-23 09:48:23 -07:00
e21b81276e
HADOOP-17306. RawLocalFileSystem's lastModifiedTime() looses milli seconds in JDK < 10.b09 ( #2387 )
2020-10-23 11:34:14 +05:30
02709cb054
HDFS-15622. Deleted blocks linger in the replications queue. Contributed by Ahmed Hussein.
...
(cherry picked from commit da1b6e3cc2
2020-10-22 19:41:38 -05:00
c40f0f1eb3
MAPREDUCE-7303. Fix TestJobResourceUploader failures after HADOOP-16878. Contributed by Peter Bacsko.
...
(cherry picked from commit 7bc305db5d
2020-10-23 04:41:37 +09:00
cf932a7e2d
HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein.
2020-10-22 09:55:28 -05:00
0bb1f0df27
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. ( #2394 )
...
(cherry picked from commit 30f06e0c74
2020-10-20 19:12:26 +09:00
Steve Loughran
Peter Bacsko
Jungtaek Lim
Jim Brennan
Akira Ajisaka
Ahmed Hussein
Doroszlai, Attila
Takanobu Asanuma
Chao Sun
Stephen Jung
Gabriel Medeiros Coelho
Yizhong Zhang
hchaverr
Eric E Payne
Eric Badger
Wei-Chiu Chuang
Ayush Saxena
Konstantin V Shvachko
Sunil G
Jonathan Hung
Brahma Reddy Battula
Hui Fei
Anoop Sam John
Inigo Goiri
Vinayakumar B
Kihwal Lee