f52b900a5f
YARN-11283. Fix Typo of NodeManager amrmproxy. ( #4899 )
2022-09-19 13:16:25 -07:00
342c4856b8
YARN-11293. [Federation] StoreNewMasterKey/removeStoredMasterKey With MemoryStateStore. ( #4852 )
2022-09-19 13:14:55 -07:00
620dd37712
HADOOP-18118. [Follow on] Fix test failure in TestHttpServer ( #4900 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-09-19 09:10:00 -07:00
30c36ef25a
HADOOP-18400. Fix file split duplicating records from a succeeding split when reading BZip2 text files ( #4732 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-19 13:45:05 +09:00
a73c4804d8
HADOOP-18446. [SBN read] Add a re-queue metric to RpcMetrics to quantify the number of re-queued RPCs ( #4871 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-16 10:09:01 -07:00
0f03299eba
HADOOP-16769. LocalDirAllocator to provide diagnostics when file creation fails ( #4842 )
...
The patch provides detailed diagnostics of file creation failure in LocalDirAllocator.
Contributed by: Ashutosh Gupta
2022-09-16 13:17:26 +05:30
43c1ebae16
HDFS-16771. Follow-up for HDFS-16659: JN should tersely print logs about NewerTxnIdException ( #4882 )
...
Signed-off-by: Erik Krogen <xkrogen@apache.org>
Co-authored-by: zengqiang.xu <zengqiang.xu@shopee.com>
2022-09-15 12:44:36 -07:00
59d3c20118
MAPREDUCE-7407. Avoid stopContainer() on dead node ( #4779 )
2022-09-15 10:30:36 -07:00
6422eaf301
HDFS-16767: RBF: Support observer node in Router-Based Federation.
...
Fixes #4127
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-14 16:00:14 -07:00
ce54b7e55d
HADOOP-18118. Fix KMS Accept Queue Size default value to 500 ( #3972 )
2022-09-15 00:53:25 +08:00
272844ee57
HADOOP-15072 - Update Apache Kerby to 2.0.2 ( #4473 )
2022-09-15 00:43:25 +08:00
86b84ed74e
HADOOP-18452. Fix TestKMS#testKMSHAZooKeeperDelegationToken Failed By Hadoop-18427. ( #4885 )
2022-09-14 09:13:58 -07:00
8ce71c8882
[Documentation] RBF: Duplicate statement to be removed for better readabilty ( #4881 )
2022-09-13 09:24:05 -07:00
88545a875d
YARN-11301. Fix NoClassDefFoundError: org/junit/platform/launcher/core/LauncherFactory after YARN-11269. ( #4878 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-13 17:08:16 +09:00
55d8a91b2c
YARN-11261. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-web-proxy ( #4777 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-13 17:05:24 +09:00
3ce353395b
YARN-7614. [RESERVATION] Support ListReservation APIs in Federation Router. ( #4843 )
2022-09-12 12:33:21 -07:00
65a027b112
YARN-11241. Add uncleaning option for local app log file with log-aggregation enabled ( #4703 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-12 23:32:20 +09:00
cde1f3af21
HADOOP-18302. Remove WhiteBox in hadoop-common module. ( #4457 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-12 23:28:16 +09:00
1923096adb
Allow block reconstruction pending timeout to be refreshable ( #4567 )
...
Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2022-09-12 11:45:01 +09:00
21bae31d58
YARN-11265. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-sharedcachemanager ( #4772 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-11 03:31:28 +09:00
cdcb448b78
YARN-11286. Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable. ( #4824 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-10 23:00:50 +05:30
b2760520c3
YARN-11274. Impove Nodemanager#NodeStatusUpdaterImpl Log. ( #4783 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-10 22:57:06 +05:30
e77d54d1ee
HDFS-13522: Add federated nameservices states to client protocol and propagate it between routers and clients.
...
Fixes #4311
Signed-off-by: Owen O'Malley <oomalley@linkedin.com>
2022-09-09 15:43:28 -07:00
e76ffbf102
YARN-11297. [Federation] Improve Yarn Router Reservation Submission Code. ( #4863 )
2022-09-09 10:39:00 -07:00
8732625f50
HADOOP-18439. Fix VectoredIO for LocalFileSystem when checksum is enabled. ( #4862 )
...
part of HADOOP-18103.
While merging the ranges in CheckSumFs, they are rounded up based on the
value of checksum bytes size which leads to some ranges crossing the EOF
thus they need to be fixed else it will cause EOFException during actual reads.
Contributed By: Mukund Thakur
2022-09-09 21:46:08 +05:30
5b85af87f0
YARN-11278. Fixed Ambiguous error message in mutation API. Contributed by Ashutosh Gupta.
2022-09-09 14:38:41 +02:00
56387cce57
HADOOP-18186. s3a prefetching to use SemaphoredDelegatingExecutor for submitting work ( #4796 )
...
Contributed by Viraj Jasani
2022-09-09 11:32:20 +01:00
4a01fadb94
HDFS-16756. RBF proxies the client's user by the login user to enable CacheEntry ( #4853 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-09 12:47:36 +05:30
29c4d8d8f7
YARN-11298. Improve Yarn Router Junit Test Close MockRM. ( #4870 )
2022-09-08 11:42:36 -07:00
0db3ee5b4b
HADOOP-18427. Improve ZKDelegationTokenSecretManager#startThead With recommended methods. ( #4812 )
2022-09-08 11:41:21 -07:00
03961b10c2
HADOOP-18416. fix ITestS3AIOStatisticsContext test failure ( #4806 )
...
Follow on to HADOOP-17461.
Contributed by: Mehakmeet Singh
2022-09-08 21:03:18 +05:30
832d0e0d76
HADOOP-18443. Upgrade snakeyaml to 1.31 to mitigate CVE-2022-25857 ( #4856 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Brahma Reddy Battula <brahma@apache.org>
2022-09-08 19:58:38 +05:30
42c8f61fec
HADOOP-18441. Remove hadoop custom ServicesResourceTransformer ( #4850 ). Contributed by PJ Fanning.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-07 17:11:12 +05:30
c664f953c9
HADOOP-18426. Use weighted calculation for MutableStat mean/variance to fix accuracy. ( #4844 ). Contributed by Erik Krogen.
...
Co-authored-by: Shuyan Zhang <zqingchai@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-09-07 13:49:56 +08:00
cc41ad63f9
HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping. ( #4798 )
...
* HADOOP-18388. Allow dynamic groupSearchFilter in LdapGroupsMapping.
2022-09-06 18:38:51 -04:00
c947c326e8
HDFS-16659. JournalNode should throw NewerTxnIdException when SinceTxId is bigger than HighestWrittenTxId ( #4560 )
...
Co-authored-by: Zander Xu <zengqiang.xu@shopee.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2022-09-06 10:12:55 -07:00
7bcf853ff4
HADOOP-17873. ABFS: Fix transient failures in ITestAbfsStreamStatistics and ITestAbfsRestOperationException ( #3699 )
...
Successor for the reverted PR #3341 , using the hadoop @VisibleForTesting attribute
Contributed by Sumangala Patki
2022-09-06 11:00:52 +01:00
be4c638e4c
HDFS-16748. RBF: DFSClient should uniquely identify writing files by namespace id and iNodeId via RBF ( #4813 ). Contributed by ZanderXu.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-09-05 17:46:15 +05:30
ac42519ade
HDFS-16593. Correct the BlocksRemoved metric on DataNode side ( #4353 ). Contributed by ZanderXu.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-09-05 19:35:48 +08:00
7bf95d7949
YARN-11289. [Federation] Improve NM FederationInterceptor removeAppFromRegistry. ( #4836 )
2022-09-02 10:41:31 -07:00
1965708d49
YARN-11273. Federation StateStore: Support storage/retrieval of Reservations With SQL. ( #4817 )
2022-09-02 10:39:58 -07:00
b266f852d7
YARN-11284. [Federation] Improve UnmanagedAMPoolManager WithoutBlock ServiceStop ( #4814 )
2022-09-02 10:28:38 -07:00
3a96de7756
YARN-6667. Handle containerId duplicate without failing the heartbeat in Federation Interceptor. ( #4810 )
2022-09-02 10:25:26 -07:00
7b239a80fe
HDFS-16750. NameNode should use NameNode.getRemoteUser() to log audit event to avoid possible NPE ( #4821 )
2022-09-02 10:23:03 -07:00
c48ed3e96c
HADOOP-18408. ABFS: ITestAbfsManifestCommitProtocol fails on nonHNS configuration ( #4758 )
...
ITestAbfsManifestCommitProtocol to set requireRenameResilience to false for nonHNS configuration (#4758 )
Contributed by Sree Bhattacharyya
2022-09-02 12:33:12 +01:00
37e213c3fc
YARN-11177. Support getNewReservation, submit / update/ Reservation API's for Federation. ( #4764 )
2022-09-01 16:35:20 -07:00
20560401ec
HADOOP-18339. S3A storage class option only picked up when buffering writes to disk. ( #4669 )
...
Follow-up to HADOOP-12020 Support configuration of different S3 storage classes;
S3 storage class is now set when buffering to heap/bytebuffers, and when
creating directory markers
Contributed by Monthon Klongklaew
2022-09-01 18:14:32 +01:00
2dd8b1342e
HDFS-16755. TestQJMWithFaults.testUnresolvableHostName() can fail due to unexpected host resolution ( #4833 )
...
Use ".invalid" domain from IETF RFC 2606 to ensure that the host doesn't resolve.
Contributed by Steve Vaughan Jr
2022-09-01 14:00:15 +01:00
33edbed54c
YARN-11272. Federation StateStore: Support storage/retrieval of Reservations With Zk. ( #4781 )
2022-08-31 10:15:15 -07:00
19830c98bc
HADOOP-18391. Improvements in VectoredReadUtils#readVectored() for direct buffers ( #4787 )
...
part of HADOOP-18103.
Contributed By: Mukund Thakur
2022-08-31 21:41:41 +05:30
slfan1989
GuoPhilipse
Ashutosh Gupta
ZanderXu
Simbarashe Dzinamarira
Colm O hEigeartaigh
Renukaprasad C
caozhiqiang
Mukund Thakur
9uapaw
Viraj Jasani
Mehakmeet Singh
PJ Fanning
Erik Krogen
Ayush Saxena
Sumangala Patki
sreeb-msft
monthonk
Steve Vaughan