af20841fb1
YARN-11217. [Federation] Add dumpSchedulerLogs REST APIs for Router. ( #5272 )
2023-02-08 11:48:38 -08:00
08f58ecf07
HDFS-16904. Close webhdfs during TestSymlinkHdfs teardown ( #5342 )
...
Contributed by Steve Vaughan Jr
2023-02-08 17:15:42 +00:00
3ba058a894
HDFS-16898. Remove write lock for processCommandFromActor of DataNode to reduce impact on heartbeat ( #5330 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-02-08 11:19:07 +08:00
7e919212c4
Revert "HDFS-16898. Make write lock fine-grain in method processCommandFromActor ( #5330 ). Contributed by ZhangHB."
...
This reverts commit eb04ecd29d
2023-02-08 11:15:22 +08:00
eb04ecd29d
HDFS-16898. Make write lock fine-grain in method processCommandFromActor ( #5330 ). Contributed by ZhangHB.
...
Reviewed-by: zhangshuyan <zqingchai@gmail.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2023-02-08 10:33:28 +08:00
d310642626
HDFS-16895. [RBF] NamenodeHeartbeatService should use credentials of logged in user
2023-02-07 18:45:05 +00:00
8714403dc7
HADOOP-18621. Resource leak in CryptoOutputStream.close() ( #5347 )
...
When closing we need to wrap the flush() in a try .. finally, otherwise
when flush throws it will stop completion of the remainder of the
close activities and in particular the close of the underlying wrapped
stream object resulting in a resource leak.
Contributed by Colm Dougan
2023-02-07 12:01:57 +00:00
f02c452cf1
HDFS-16907. Add LastHeartbeatResponseTime for BP service actor ( #5349 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-07 09:54:40 +08:00
a6a9fe17e0
YARN-3657. Federation maintenance mechanisms (simple CLI and command propagation). ( #5348 )
2023-02-06 11:47:07 -08:00
5f5157ac53
HADOOP-18612. Avoid mixing canonical and non-canonical when performing comparisons ( #5339 )
...
Contributed by Steve Vaughan Jr
2023-02-06 18:28:29 +00:00
aed6fcee5b
HADOOP-18576. Java 11 JavaDoc fails due to missing package comments ( #5344 )
...
Add JavaDoc comments to package-info.java to avoid errors resulting from the use of Hadoop annotations.
Contributed by Steve Vaughan Jr
2023-02-06 18:17:57 +00:00
be564f5c20
[HDFS-16903]. Fix javadoc of LightWeightResizableGSet class ( #5338 )
2023-02-06 13:21:28 +09:00
0ae075a2af
HDFS-16848. RBF: Improve StateStoreZooKeeperImpl performance ( #5147 )
2023-02-05 09:33:05 +08:00
dad73b76c0
YARN-11419. Remove redundant exception capture in NMClientAsyncImpl and improve readability in ContainerShellWebSocket, etc ( #5309 )
...
Co-authored-by: smallzhongfeng <982458633@qq.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-04 10:29:19 +08:00
bce388fd3f
HDFS-16902 Add Namenode status to BPServiceActor metrics and improve logging in offerservice ( #5334 )
...
Reviewed-by: Mingliang Liu <liuml07@apache.org>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-02-03 09:11:02 +08:00
22f6d55b71
HADOOP-18246. Reduce lower limit on fs.s3a.prefetch.block.size to 1 byte. ( #5120 )
...
The minimum value of fs.s3a.prefetch.block.size is now 1
Contributed by Ankit Saurabh
2023-02-02 18:45:21 +00:00
ad0cff2f97
HADOOP-18592. Sasl connection failure should log remote address. ( #5294 )
...
Contributed by Viraj Jasani <vjasani@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2023-02-01 10:15:20 -08:00
6d325d9d09
HADOOP-18598. maven site generation doesn't include javadocs. ( #5319 )
...
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit 004121f9cc
2023-01-31 10:50:41 +00:00
a70f84098f
HADOOP-18601. Fix build failure with docs profile. ( #5331 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2023-01-31 19:44:19 +09:00
88c8ac750d
HDFS-16888. BlockManager#maxReplicationStreams, replicationStreamsHardLimit, blocksReplWorkMultiplier and PendingReconstructionBlocks#timeout should be volatile ( #5296 )
...
Reviewed-by: Tao Li <tomscut@apache.org>
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2023-01-31 17:46:38 +09:00
9d47108b50
HADOOP-18584. [NFS GW] Fix regression after netty4 migration. ( #5252 )
...
Reviewed-by: Tsz-Wo Nicholas Sze <szetszwo@apache.org>
2023-01-31 01:17:04 +08:00
952d707240
HADOOP-18604. Add compile platform in the hadoop version output. ( #5327 ). Contributed by Ayush Saxena.
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-28 14:19:19 +05:30
b677d40ab5
HADOOP-18602. Remove netty3 dependency
2023-01-27 16:32:50 +01:00
970ebaeded
HADOOP-17717. Update wildfly openssl to 1.1.3.Final. ( #5310 )
...
Contributed by Wei-Chiu Chuang
2023-01-27 11:50:17 +00:00
468135a4d9
YARN-11218. [Federation] Add getActivities, getBulkActivities REST APIs for Router. ( #5284 )
2023-01-26 11:14:05 -08:00
cf1b3711cb
YARN-10965. Centralize queue resource calculation based on CapacityVectors. Contributed by Andras Gyori
2023-01-26 19:45:54 +01:00
815cde9810
YARN-6971. Clean up different ways to create resources. Contributed by Riya Khandelwal
2023-01-25 17:28:29 +01:00
29f2230cb6
YARN-5607. Document TestContainerResourceUsage#waitForContainerCompletion. Contributed by Susheel Gupta
2023-01-25 15:13:24 +01:00
8eda456d37
YARN-11404. Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure. Contributed by Susheel Gupta
2023-01-25 15:06:20 +01:00
3b7b79b37a
HADOOP-18582. skip unnecessary cleanup logic in distcp ( #5251 )
...
Co-authored-by: 万康 <mingge@xiaohongshu.com>
Reviewed-by: Steve Loughran <stevel@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-24 15:49:32 -08:00
3f767a61b1
YARN-8900. [Follow Up] Fix FederationInterceptorREST#invokeConcurrent Inaccurate Order of Subclusters. ( #5260 )
2023-01-19 17:13:55 -08:00
72b760130a
HDFS-16893. Standardize the usage of DFSClient debug log ( #5303 )
...
Co-authored-by: Zhtttylz <hualong.z@hotmail.com>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-01-19 07:56:41 +08:00
04f3573f6a
HDFS-16891 Avoid the overhead of copy-on-write exception list while loading inodes sub sections in parallel ( #5300 )
...
Reviewed-by: Stephen O'Donnell <sodonnell@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-18 13:13:41 -08:00
442a5fb285
YARN-11320. [Federation] Add getSchedulerInfo REST APIs for Router. ( #5217 )
2023-01-17 09:36:19 -08:00
4de31123ce
Fix "the the" and friends typos ( #5267 )
...
Signed-off-by: Nikita Eshkeev <neshkeev@yandex.ru>
2023-01-17 03:33:59 +08:00
d81d98388c
HADOOP-18575: followup: try to avoid repeatedly hitting exceptions when transformer factories do not support attributes ( #5253 )
...
Part of HADOOP-18469 and the hardening of XML/XSL parsers.
Followup to the main HADOOP-18575 patch, to improve performance when
working with xml/xsl engines which don't support the relevant attributes.
Include this change when backporting.
Contributed by PJ Fanning.
2023-01-16 13:15:37 +00:00
38453f8589
MAPREDUCE-7413. Upgrade Junit 4 to 5 in hadoop-mapreduce-client-hs-plugins ( #5023 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2023-01-16 18:19:28 +09:00
082266516a
MAPREDUCE-7417. Upgrade Junit 4 to 5 in hadoop-mapreduce-client-uploader ( #5019 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2023-01-16 17:22:04 +09:00
168fa07801
YARN-11409. Fix Typo of ResourceManager#webapp moudle. ( #5285 )
2023-01-13 11:53:13 +08:00
4520448327
YARN-11374. [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation. ( #5193 )
2023-01-12 15:04:09 -08:00
1263e024b9
HDFS-16887 Log start and end of phase/step in startup progress ( #5292 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-12 14:26:52 -08:00
36bf54aba0
MAPREDUCE-7375 JobSubmissionFiles don't set right permission after mkdirs ( #4237 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-12 13:48:29 -08:00
a90e424d9f
HADOOP-18591. Fix a typo in Trash ( #5291 )
...
Signed-off-by: Tao Li <tomscut@apache.org>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-12 13:21:21 -08:00
3d21cff263
YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412. ( #5289 )
...
* YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412.
* YARN-11413. Fix CheckStyle.
* YARN-11413. Fix CheckStyle.
Co-authored-by: slfan1989 <louj1988@@>
2023-01-12 14:29:05 +01:00
f26d8bc9bd
HDFS-16886: Fixes error in documentation for StateStoreRecordsOperations. ( #5290 )
2023-01-11 19:46:51 +00:00
7f6cc196f8
YARN-11190. CS Mapping rule bug: User matcher does not work correctly for usernames with dot ( #4471 )
2023-01-11 13:23:04 +01:00
e3b09b7512
HDFS-16884. Fix TestFsDatasetImpl#testConcurrentWriteAndDeleteBlock failed ( #5280 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
Signed-off-by: Tao Li <tomscut@apache.org>
2023-01-11 09:12:26 +08:00
4cf304de45
HDFS-16872. Fix log throttling by declaring LogThrottlingHelper as static members ( #5246 )
...
Co-authored-by: Chengbing Liu <liuchengbing@qiyi.com>
Signed-off-by: Erik Krogen <xkrogen@apache.org>
2023-01-10 10:03:25 -08:00
f3cff032e6
HDFS-16885. Fix TestHdfsConfigFields#testCompareConfigurationClassAgainstXml failed ( #5283 )
2023-01-10 09:02:56 +08:00
6f99558c2e
HADOOP-18590. Publish SBOM artifacts ( #5281 )
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2023-01-09 16:41:06 -08:00
slfan1989
Steve Vaughan
He Xiaoqiao
hfutatzhanghb
hchaverri
gardenia
Viraj Jasani
sunhao
jokercurry
Ankit Saurabh
Masatake Iwasaki
huhaiyang
Wei-Chiu Chuang
Ayush Saxena
Szilard Nemeth
Steve Loughran
kevin wan
zhtttylz
Nikita Eshkeev
PJ Fanning
Ashutosh Gupta
skysiders
huangxiaoping
Simbarashe Dzinamarira
Szilard Nemeth
Chengbing Liu
Dongjoon Hyun