48b6f9f335
YARN-11328. Refactoring part of the code of SQLFederationStateStore. ( #4976 )
2022-10-19 16:11:28 -07:00
8aa04b0b24
HADOOP-18189 S3APrefetchingInputStream to support status probes when closed ( #5036 )
...
Contributed by Viraj Jasani
2022-10-19 14:38:11 +01:00
6207ac47e0
HADOOP-18304. Improve user-facing S3A committers documentation ( #4478 )
...
Contributed by: Daniel Carl Jones
2022-10-19 12:56:47 +05:30
d80db6c9e5
HADOOP-18476. Abfs and S3A FileContext bindings to close wrapped filesystems in finalizer ( #4966 )
...
This is to try and close the underlying filesystems when the FileContext APIs are used.
Without this, threads may be leaked
2022-10-18 14:53:02 +01:00
ee886cacd7
YARN-11247. Remove unused classes introduced by YARN-9615. ( #4720 )
2022-10-18 09:40:39 -04:00
babb050fa3
HADOOP-18497. Upgrade commons-text version to fix CVE-2022-42889. ( #5037 ). Contributed by PJ Fanning.
...
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2022-10-18 11:28:56 +08:00
2d91daab5e
HADOOP-18156. Address JavaDoc warnings in classes like MarkerTool, S3ObjectAttributes, etc ( #4965 )
...
Contributed by Ankit Saurabh
2022-10-17 18:10:47 +01:00
9a8aff69ff
HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS ( #4750 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-10-17 18:56:15 +08:00
136291d2d5
HADOOP-18462. InstrumentedWriteLock should consider Reentrant case ( #4919 ). Contributed by ZanderXu.
...
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-10-17 12:44:25 +08:00
4ff6c9b8de
HADOOP-18493: upgrade jackson-databind to 2.12.7.1 ( #5011 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-17 10:03:10 +05:30
b0d5182c31
YARN-10680. Revisit try blocks without catch blocks but having finally blocks. Contributed by Susheel Gupta
2022-10-15 21:51:08 +02:00
77e551a478
HADOOP-18481. AWS v2 SDK upgrade log to not about standard AWS Credential Providers. ( #4973 )
...
The AWS SDKV2 upgrade log no longer warns about instantiation
of the v1 SDK credential providers which are commonly used in
s3a configurations:
* com.amazonaws.auth.EnvironmentVariableCredentialsProvider
* com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper
* com.amazonaws.auth.InstanceProfileCredentialsProvider
When the hadoop-aws module moves to the v2 SDK, references to these
credential providers will be rewritten to their v2 equivalents.
Follow-on to HADOOP-18382. "Upgrade AWS SDK to V2 - Prerequisites"
Contributed by Ahmar Suhail
2022-10-14 10:48:09 +01:00
5b52123c9d
YARN-8041. [Router] Federation: Improve Router REST API Metrics. ( #4938 )
2022-10-13 16:54:36 -07:00
1962851356
YARN-11294. [Federation] Router Support DelegationToken store/update/remove Token With MemoryStateStore. ( #4915 )
2022-10-13 16:52:22 -07:00
647457e6ab
YARN-11327. [Federation] Refactoring Yarn Router's Node Web Page. ( #5009 )
2022-10-13 14:05:30 -07:00
bfce21ee08
YARN-11330. use secure XML parsers ( #4981 )
...
Move construction of XML parsers in YARN
modules to using the locked-down parser factory
of HADOOP-18469.
One exception: GpuDeviceInformationParser still supports DTD resolution;
all other features are disabled.
Contributed by P J Fanning
2022-10-13 18:19:19 +01:00
9439d8e4e4
HADOOP-18292. Fix s3 select tests when running against unsupported storage class ( #4489 )
...
Follow-on from HADOOP-12020.
Contributed by Monthon Klongklaew
2022-10-13 13:13:36 +01:00
3ff8f58f8c
HADOOP-18360. Update commons-csv from 1.0 to 1.9.0. ( #4928 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-13 12:10:54 +05:30
1ff7e84caf
YARN-11334. Improve SubClusterState#fromString parameter and LogMessage. ( #4988 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-13 12:06:24 +05:30
d6b1e1eeb6
HDFS-16777. datatables@1.10.17 sonatype-2020-0988 vulnerability ( #5003 )
2022-10-12 14:51:12 -07:00
5694d7e25f
Add Dockerfile_windows_10 ( #4936 )
2022-10-12 09:30:54 -07:00
d78b0b39a6
YARN-11323. [Federation] Improve ResourceManager Handler FinishApps. ( #4954 )
2022-10-11 14:53:02 -07:00
9e16f1f883
YARN-11317. [Federation] Refactoring Yarn Router's About Web Page. ( #4946 )
2022-10-11 13:30:48 -07:00
82a88a8ae6
YARN-11315. [Federation] YARN Federation Router Supports Cross-Origin. ( #4934 )
2022-10-11 10:18:50 -07:00
2122733c30
Add .yetus/excludes.txt ( #4984 )
2022-10-11 09:23:34 -07:00
03d600fa82
HADOOP-17779: Lock File System Creator Semaphore Uninterruptibly ( #3158 )
2022-10-11 11:56:32 +01:00
d14b88c698
HDFS-16774.Improve async delete replica on datanode ( #4903 )
...
HDFS-16774. Improve async delete replica on datanode to reduce the probability of ReplicationNotFoundException
Co-authored-by: Haiyang Hu <haiyang.hu@shopee.com>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-10-11 16:22:56 +08:00
4fe079f85f
HDFS-16795. Use secure XML parsers ( #4979 )
...
Contributed by P J Fanning
2022-10-10 18:56:35 +01:00
0c515b0ef0
YARN-6766. Add helper method in FairSchedulerAppsBlock to print app info. Contributed by Riya Khandelwal
2022-10-10 15:30:33 +02:00
62ff4e36cf
HDFS-16787. Remove the redundant lock in DataSetLockManager#removeLock ( #4948 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-10-10 20:05:02 +08:00
b0b2cb4a16
HDFS-16783. Remove the redundant lock in deepCopyReplica and getFinalizedBlocks ( #4942 ). Contributed by ZanderXu.
...
Reviewed-by: Mingxiang Li <liaiphag0@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-10-10 19:52:22 +08:00
be70bbb4be
HADOOP-18460. checkIfVectoredIOStopped before populating the buffers ( #4986 )
...
Contributed by Mukund Thakur
2022-10-10 11:17:45 +01:00
540a660429
HADOOP-18480. Upgrade aws sdk to 1.12.316 ( #4972 )
...
Contributed by Steve Loughran
2022-10-10 10:23:50 +01:00
9a7d0e7ed0
YARN-11260. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice ( #4775 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-10 00:28:03 +09:00
b0bfd09c41
HDFS-16798. HDFS-16798. SerialNumberMap should decrease current counter if the item exist. ( #4987 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-10-09 18:58:09 +08:00
5eddec8c46
HADOOP-18468: Upgrade jettison to 1.5.1 to fix CVE-2022-40149 ( #4937 )
...
Contributed by PJ Fanning
2022-10-07 15:44:01 +01:00
062c50db6b
MAPREDUCE-7370. Parallelize MultipleOutputs#close call ( #4248 ). Contributed by Ashutosh Gupta.
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-10-06 15:23:05 -07:00
8336b91329
HADOOP-18469. Add secure XML parser factories to XMLUtils ( #4940 )
...
Add to XMLUtils a set of methods to create secure XML Parsers/transformers, locking down DTD, schema, XXE exposure.
Use these wherever XML parsers are created.
Contributed by PJ Fanning
2022-10-06 19:30:51 +01:00
b31b3ea0f6
YARN-11187. Remove WhiteBox in yarn module. ( #4463 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-06 23:13:33 +09:00
7ec762a5fd
HADOOP-18465. Fix S3A SSE test skip when encryption is disabled ( #4925 )
...
Contributed by Daniel Carl Jones
2022-10-06 12:42:01 +01:00
1675a28e5a
HADOOP-18378. Implement lazy seek in S3A prefetching. ( #4955 )
...
Make S3APrefetchingInputStream.seek() completely lazy. Calls to seek() will not affect the current buffer nor interfere with prefetching, until read() is called.
This change allows various usage patterns to benefit from prefetching, e.g. when calling readFully(position, buffer) in a loop for contiguous positions the intermediate internal calls to seek() will be noops and prefetching will have the same performance as in a sequential read.
Contributed by Alessandro Passaro.
2022-10-06 12:00:41 +01:00
38b2ed2151
HADOOP-18442. Remove openstack support ( #4855 )
...
Contributed by Steve Loughran
2022-10-06 11:49:38 +01:00
1a9faf123d
YARN-11313. [Federation] Add SQLServer Script and Supported DB Version in Federation.md. ( #4927 )
2022-10-05 13:25:20 -07:00
a708ff96f1
YARN-11324. Fix some PBImpl classes to avoid NPE. ( #4961 )
2022-10-04 14:30:20 -04:00
874a004347
YARN-11318. Improve FederationInterceptorREST#createInterceptorForSubCluster Use WebAppUtils. ( #4939 )
2022-10-04 09:18:01 -07:00
22bd5e3b53
YARN-11238. Optimizing FederationClientInterceptor Call with Parallelism. ( #4904 )
2022-10-04 09:17:00 -07:00
07581f1ab2
YARN-6169 message on empty configuration file improved ( #4952 )
2022-10-03 23:31:06 -04:00
4891bf5049
HDFS-13369. Fix for FSCK Report broken with RequestHedgingProxyProvider ( #4917 )
...
Contributed-by: navinko <nakumr@cloudera.com>
2022-09-30 23:28:12 +08:00
e22f5e75ae
HADOOP-18463. Add an integration test to process data asynchronously during vectored read. ( #4921 )
...
part of HADOOP-18103.
Contributed by: Mukund Thakur
2022-09-28 23:16:47 +05:30
42d883937d
YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page. ( #4924 )
2022-09-27 13:31:52 -07:00
slfan1989
Viraj Jasani
Daniel Carl Jones
Steve Loughran
Hexiaoqiao
Ankit Saurabh
Ashutosh Gupta
ZanderXu
PJ Fanning
Szilard Nemeth
ahmarsuhail
monthonk
Gautham B A
belugabehr
huhaiyang
Szilard Nemeth
Mukund Thakur
Alessandro Passaro
Riya Khandelwal
Navink