b0bfd09c41
HDFS-16798. HDFS-16798. SerialNumberMap should decrease current counter if the item exist. ( #4987 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-10-09 18:58:09 +08:00
5eddec8c46
HADOOP-18468: Upgrade jettison to 1.5.1 to fix CVE-2022-40149 ( #4937 )
...
Contributed by PJ Fanning
2022-10-07 15:44:01 +01:00
062c50db6b
MAPREDUCE-7370. Parallelize MultipleOutputs#close call ( #4248 ). Contributed by Ashutosh Gupta.
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-10-06 15:23:05 -07:00
8336b91329
HADOOP-18469. Add secure XML parser factories to XMLUtils ( #4940 )
...
Add to XMLUtils a set of methods to create secure XML Parsers/transformers, locking down DTD, schema, XXE exposure.
Use these wherever XML parsers are created.
Contributed by PJ Fanning
2022-10-06 19:30:51 +01:00
b31b3ea0f6
YARN-11187. Remove WhiteBox in yarn module. ( #4463 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-06 23:13:33 +09:00
7ec762a5fd
HADOOP-18465. Fix S3A SSE test skip when encryption is disabled ( #4925 )
...
Contributed by Daniel Carl Jones
2022-10-06 12:42:01 +01:00
1675a28e5a
HADOOP-18378. Implement lazy seek in S3A prefetching. ( #4955 )
...
Make S3APrefetchingInputStream.seek() completely lazy. Calls to seek() will not affect the current buffer nor interfere with prefetching, until read() is called.
This change allows various usage patterns to benefit from prefetching, e.g. when calling readFully(position, buffer) in a loop for contiguous positions the intermediate internal calls to seek() will be noops and prefetching will have the same performance as in a sequential read.
Contributed by Alessandro Passaro.
2022-10-06 12:00:41 +01:00
38b2ed2151
HADOOP-18442. Remove openstack support ( #4855 )
...
Contributed by Steve Loughran
2022-10-06 11:49:38 +01:00
1a9faf123d
YARN-11313. [Federation] Add SQLServer Script and Supported DB Version in Federation.md. ( #4927 )
2022-10-05 13:25:20 -07:00
a708ff96f1
YARN-11324. Fix some PBImpl classes to avoid NPE. ( #4961 )
2022-10-04 14:30:20 -04:00
874a004347
YARN-11318. Improve FederationInterceptorREST#createInterceptorForSubCluster Use WebAppUtils. ( #4939 )
2022-10-04 09:18:01 -07:00
22bd5e3b53
YARN-11238. Optimizing FederationClientInterceptor Call with Parallelism. ( #4904 )
2022-10-04 09:17:00 -07:00
07581f1ab2
YARN-6169 message on empty configuration file improved ( #4952 )
2022-10-03 23:31:06 -04:00
4891bf5049
HDFS-13369. Fix for FSCK Report broken with RequestHedgingProxyProvider ( #4917 )
...
Contributed-by: navinko <nakumr@cloudera.com>
2022-09-30 23:28:12 +08:00
e22f5e75ae
HADOOP-18463. Add an integration test to process data asynchronously during vectored read. ( #4921 )
...
part of HADOOP-18103.
Contributed by: Mukund Thakur
2022-09-28 23:16:47 +05:30
42d883937d
YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page. ( #4924 )
2022-09-27 13:31:52 -07:00
bfd6415827
YARN-11290. Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster. ( #4846 )
2022-09-27 13:28:52 -07:00
735e35d648
HADOOP-18347. S3A Vectored IO to use bounded thread pool. ( #4918 )
...
part of HADOOP-18103.
Also introducing a config fs.s3a.vectored.active.ranged.reads
to configure the maximum number of number of range reads a
single input stream can have active (downloading, or queued)
to the central FileSystem instance's pool of queued operations.
This stops a single stream overloading the shared thread pool.
Contributed by: Mukund Thakur
2022-09-27 21:13:07 +05:30
d9f435f6ac
HDFS-16766. XML External Entity (XXE) attacks can occur while processing XML received from an untrusted source ( #4886 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-27 15:44:25 +09:00
5d20988f9f
YARN-11308. Router Page display the db username and password in mask mode. ( #4908 )
2022-09-26 15:54:17 -07:00
0e65f4cc04
YARN-11316. Fix Yarn federation.md table format. ( #4935 )
2022-09-26 15:47:08 -07:00
aeba204fa2
YARN-11306. [Federation] Refactor NM#FederationInterceptor#recover Code. ( #4897 )
2022-09-26 15:46:06 -07:00
648071e197
HADOOP-18466. Limit the findbugs suppression IS2_INCONSISTENT_SYNC to S3AFileSystem field ( #4926 )
...
Follow-on to HADOOP-18455.
Contributed by Viraj Jasani
2022-09-26 18:56:58 +01:00
7923cac86b
HADOOP-18443. Upgrade snakeyaml to 1.32 ( #4906 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:49:48 +09:00
603e9bd745
YARN-11271. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-common ( #4774 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:30:58 +09:00
fd0415c44a
YARN-11270. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-client ( #4773 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:26:06 +09:00
747fb92107
HADOOP-18444 Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash ( #4869 )
...
* HADOOP-18444 Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash
Signed-off-by: Xing Lin <xinglin@linkedin.com>
2022-09-23 10:37:51 -07:00
0676495950
HADOOP-18456. NullPointerException in ObjectListingIterator. ( #4909 )
...
This problem surfaced in impala integration tests
IMPALA-11592. TestLocalCatalogRetries.test_fetch_metadata_retry fails in S3 build
after the change
HADOOP-17461. Add thread-level IOStatistics Context
The actual GC race condition came with
HADOOP-18091. S3A auditing leaks memory through ThreadLocal references
The fix for this is, if our hypothesis is correct, in WeakReferenceMap.create()
where a strong reference to the new value is kept in a local variable
*and referred to later* so that the JVM will not GC it.
Along with the fix, extra assertions ensure that if the problem is not fixed,
applications will fail faster/more meaningfully.
Contributed by Steve Loughran.
2022-09-23 09:54:31 +01:00
9a29075f91
HDFS-16776 Erasure Coding: The length of targets should be checked when DN gets a reconstruction task ( #4901 )
2022-09-23 12:27:56 +09:00
e526f48fa4
YARN-11296. [Federation] Fix SQLFederationStateStore#Sql script bug. ( #4858 )
2022-09-22 16:40:47 -07:00
e6d2c336cb
HADOOP-18341: upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 ( #4578 )
...
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-22 09:45:20 +09:00
084b68e380
HADOOP-18455. S3A prefetching executor should be closed ( #4879 )
...
follow-on patch to HADOOP-18186.
Contributed by: Viraj Jasani
2022-09-22 00:22:41 +05:30
740e1ef357
HDFS-16706. ViewFS doc points to wrong mount table name ( #4803 )
2022-09-21 16:55:20 +05:30
917aef75fc
YARN-11255. Support loading alternative docker client config from system environment ( #4884 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-09-21 16:54:02 +05:30
4c5a7cc6fc
YARN-11307. Fix Yarn Router Broken Link. ( #4905 )
2022-09-20 23:10:09 -04:00
4d9bb81b16
HADOOP-18451. Update hsqldb.version from 2.3.4 to 2.5.2. ( #4880 )
2022-09-20 11:10:51 -07:00
40ab9c8ba7
HDFS-16341. Fix BlockPlacementPolicy details in hdfs defaults. ( #3691 ). Contributed by guophilipse.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:16:24 +05:30
2950c5405b
HADOOP-16674. Fix when TestDNS.testRDNS can fail with ServiceUnavailableException ( #4802 ). Contributed by Ashutosh Gupta.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:09:59 +05:30
fd687bb4c4
YARN-11305. Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241. ( #4893 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:06:24 +05:30
a134628d1b
YARN-11303. Upgrade jquery ui to 1.13.2 to mitigate CVE-2022-31160 ( #4895 )
2022-09-20 09:50:03 -07:00
5b1657278c
HADOOP-18377. hadoop-aws build to add a -prefetch profile to run all tests with prefetching ( #4914 )
...
Contributed by Viraj Jasani
2022-09-20 10:26:13 +01:00
e68006cd70
HDFS-16772. refreshHostsReader should use the latest configuration ( #4890 )
2022-09-19 13:27:07 -07:00
f52b900a5f
YARN-11283. Fix Typo of NodeManager amrmproxy. ( #4899 )
2022-09-19 13:16:25 -07:00
342c4856b8
YARN-11293. [Federation] StoreNewMasterKey/removeStoredMasterKey With MemoryStateStore. ( #4852 )
2022-09-19 13:14:55 -07:00
620dd37712
HADOOP-18118. [Follow on] Fix test failure in TestHttpServer ( #4900 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-09-19 09:10:00 -07:00
30c36ef25a
HADOOP-18400. Fix file split duplicating records from a succeeding split when reading BZip2 text files ( #4732 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-19 13:45:05 +09:00
a73c4804d8
HADOOP-18446. [SBN read] Add a re-queue metric to RpcMetrics to quantify the number of re-queued RPCs ( #4871 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-16 10:09:01 -07:00
0f03299eba
HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails ( #4842 )
...
The patch provides detailed diagnostics of file creation failure in LocalDirAllocator.
Contributed by: Ashutosh Gupta
2022-09-16 13:17:26 +05:30
43c1ebae16
HDFS-16771. Follow-up for HDFS-16659: JN should tersely print logs about NewerTxnIdException ( #4882 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-15 12:44:36 -07:00
59d3c20118
MAPREDUCE-7407. Avoid stopContainer() on dead node ( #4779 )
2022-09-15 10:30:36 -07:00
ZanderXu
PJ Fanning
Ashutosh Gupta
slfan1989
Daniel Carl Jones
Alessandro Passaro
Steve Loughran
Riya Khandelwal
Navink
Mukund Thakur
Viraj Jasani
Xing Lin
Kidd5368
Samrat
GuoPhilipse
ZanderXu