42d883937d
YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page. ( #4924 )
2022-09-27 13:31:52 -07:00
bfd6415827
YARN-11290. Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster. ( #4846 )
2022-09-27 13:28:52 -07:00
735e35d648
HADOOP-18347. S3A Vectored IO to use bounded thread pool. ( #4918 )
...
part of HADOOP-18103.
Also introducing a config fs.s3a.vectored.active.ranged.reads
to configure the maximum number of number of range reads a
single input stream can have active (downloading, or queued)
to the central FileSystem instance's pool of queued operations.
This stops a single stream overloading the shared thread pool.
Contributed by: Mukund Thakur
2022-09-27 21:13:07 +05:30
d9f435f6ac
HDFS-16766. XML External Entity (XXE) attacks can occur while processing XML received from an untrusted source ( #4886 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-27 15:44:25 +09:00
5d20988f9f
YARN-11308. Router Page display the db username and password in mask mode. ( #4908 )
2022-09-26 15:54:17 -07:00
0e65f4cc04
YARN-11316. Fix Yarn federation.md table format. ( #4935 )
2022-09-26 15:47:08 -07:00
aeba204fa2
YARN-11306. [Federation] Refactor NM#FederationInterceptor#recover Code. ( #4897 )
2022-09-26 15:46:06 -07:00
648071e197
HADOOP-18466. Limit the findbugs suppression IS2_INCONSISTENT_SYNC to S3AFileSystem field ( #4926 )
...
Follow-on to HADOOP-18455.
Contributed by Viraj Jasani
2022-09-26 18:56:58 +01:00
7923cac86b
HADOOP-18443. Upgrade snakeyaml to 1.32 ( #4906 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:49:48 +09:00
603e9bd745
YARN-11271. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-common ( #4774 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:30:58 +09:00
fd0415c44a
YARN-11270. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-client ( #4773 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:26:06 +09:00
747fb92107
HADOOP-18444 Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash ( #4869 )
...
* HADOOP-18444 Add Support for localized trash for ViewFileSystem in Trash.moveToAppropriateTrash
Signed-off-by: Xing Lin <xinglin@linkedin.com>
2022-09-23 10:37:51 -07:00
0676495950
HADOOP-18456. NullPointerException in ObjectListingIterator. ( #4909 )
...
This problem surfaced in impala integration tests
IMPALA-11592. TestLocalCatalogRetries.test_fetch_metadata_retry fails in S3 build
after the change
HADOOP-17461. Add thread-level IOStatistics Context
The actual GC race condition came with
HADOOP-18091. S3A auditing leaks memory through ThreadLocal references
The fix for this is, if our hypothesis is correct, in WeakReferenceMap.create()
where a strong reference to the new value is kept in a local variable
*and referred to later* so that the JVM will not GC it.
Along with the fix, extra assertions ensure that if the problem is not fixed,
applications will fail faster/more meaningfully.
Contributed by Steve Loughran.
2022-09-23 09:54:31 +01:00
9a29075f91
HDFS-16776 Erasure Coding: The length of targets should be checked when DN gets a reconstruction task ( #4901 )
2022-09-23 12:27:56 +09:00
e526f48fa4
YARN-11296. [Federation] Fix SQLFederationStateStore#Sql script bug. ( #4858 )
2022-09-22 16:40:47 -07:00
e6d2c336cb
HADOOP-18341: upgrade commons-configuration2 to 2.8.0 and commons-text to 1.9 ( #4578 )
...
Reviewed-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-22 09:45:20 +09:00
084b68e380
HADOOP-18455. S3A prefetching executor should be closed ( #4879 )
...
follow-on patch to HADOOP-18186.
Contributed by: Viraj Jasani
2022-09-22 00:22:41 +05:30
740e1ef357
HDFS-16706. ViewFS doc points to wrong mount table name ( #4803 )
2022-09-21 16:55:20 +05:30
917aef75fc
YARN-11255. Support loading alternative docker client config from system environment ( #4884 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-09-21 16:54:02 +05:30
4c5a7cc6fc
YARN-11307. Fix Yarn Router Broken Link. ( #4905 )
2022-09-20 23:10:09 -04:00
4d9bb81b16
HADOOP-18451. Update hsqldb.version from 2.3.4 to 2.5.2. ( #4880 )
2022-09-20 11:10:51 -07:00
40ab9c8ba7
HDFS-16341. Fix BlockPlacementPolicy details in hdfs defaults. ( #3691 ). Contributed by guophilipse.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:16:24 +05:30
2950c5405b
HADOOP-16674. Fix when TestDNS.testRDNS can fail with ServiceUnavailableException ( #4802 ). Contributed by Ashutosh Gupta.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:09:59 +05:30
fd687bb4c4
YARN-11305. Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241. ( #4893 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-20 23:06:24 +05:30
a134628d1b
YARN-11303. Upgrade jquery ui to 1.13.2 to mitigate CVE-2022-31160 ( #4895 )
2022-09-20 09:50:03 -07:00
5b1657278c
HADOOP-18377. hadoop-aws build to add a -prefetch profile to run all tests with prefetching ( #4914 )
...
Contributed by Viraj Jasani
2022-09-20 10:26:13 +01:00
e68006cd70
HDFS-16772. refreshHostsReader should use the latest configuration ( #4890 )
2022-09-19 13:27:07 -07:00
f52b900a5f
YARN-11283. Fix Typo of NodeManager amrmproxy. ( #4899 )
2022-09-19 13:16:25 -07:00
342c4856b8
YARN-11293. [Federation] StoreNewMasterKey/removeStoredMasterKey With MemoryStateStore. ( #4852 )
2022-09-19 13:14:55 -07:00
620dd37712
HADOOP-18118. [Follow on] Fix test failure in TestHttpServer ( #4900 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-09-19 09:10:00 -07:00
30c36ef25a
HADOOP-18400. Fix file split duplicating records from a succeeding split when reading BZip2 text files ( #4732 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-19 13:45:05 +09:00
a73c4804d8
HADOOP-18446. [SBN read] Add a re-queue metric to RpcMetrics to quantify the number of re-queued RPCs ( #4871 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-16 10:09:01 -07:00
0f03299eba
HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails ( #4842 )
...
The patch provides detailed diagnostics of file creation failure in LocalDirAllocator.
Contributed by: Ashutosh Gupta
2022-09-16 13:17:26 +05:30
43c1ebae16
HDFS-16771. Follow-up for HDFS-16659: JN should tersely print logs about NewerTxnIdException ( #4882 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-15 12:44:36 -07:00
59d3c20118
MAPREDUCE-7407. Avoid stopContainer() on dead node ( #4779 )
2022-09-15 10:30:36 -07:00
6422eaf301
HDFS-16767: RBF: Support observer node in Router-Based Federation.
...
Fixes #4127
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-14 16:00:14 -07:00
ce54b7e55d
HADOOP-18118. Fix KMS Accept Queue Size default value to 500 ( #3972 )
2022-09-15 00:53:25 +08:00
272844ee57
HADOOP-15072 - Update Apache Kerby to 2.0.2 ( #4473 )
2022-09-15 00:43:25 +08:00
86b84ed74e
HADOOP-18452. Fix TestKMS#testKMSHAZooKeeperDelegationToken Failed By Hadoop-18427. ( #4885 )
2022-09-14 09:13:58 -07:00
8ce71c8882
[Documentation] RBF: Duplicate statement to be removed for better readabilty ( #4881 )
2022-09-13 09:24:05 -07:00
88545a875d
YARN-11301. Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory after YARN-11269. ( #4878 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-13 17:08:16 +09:00
55d8a91b2c
YARN-11261. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy ( #4777 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-13 17:05:24 +09:00
3ce353395b
YARN-7614. [RESERVATION] Support ListReservation APIs in Federation Router. ( #4843 )
2022-09-12 12:33:21 -07:00
65a027b112
YARN-11241. Add uncleaning option for local app log file with log-aggregation enabled ( #4703 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-12 23:32:20 +09:00
cde1f3af21
HADOOP-18302. Remove WhiteBox in hadoop-common module. ( #4457 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-12 23:28:16 +09:00
1923096adb
Allow block reconstruction pending timeout to be refreshable ( #4567 )
...
Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-12 11:45:01 +09:00
21bae31d58
YARN-11265. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-sharedcachemanager ( #4772 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-11 03:31:28 +09:00
cdcb448b78
YARN-11286. Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable. ( #4824 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-10 23:00:50 +05:30
b2760520c3
YARN-11274. Impove Nodemanager#NodeStatusUpdaterImpl Log. ( #4783 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-10 22:57:06 +05:30
e77d54d1ee
HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients.
...
Fixes #4311
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-09 15:43:28 -07:00
slfan1989
Mukund Thakur
Ashutosh Gupta
Viraj Jasani
Xing Lin
Steve Loughran
Kidd5368
PJ Fanning
Samrat
GuoPhilipse
ZanderXu
Simbarashe Dzinamarira
Colm O hEigeartaigh
Renukaprasad C
caozhiqiang