2e997d818d
YARN-11370. [Federation] Refactor MemoryFederationStateStore code. ( #5126 )
2023-02-22 12:37:35 -08:00
49b8ac19f2
YARN-11408. Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method ( #5278 )
...
* added a check of autoQueueCreationV2Disabled
Change-Id: If1e36c5969d270c1b81a4bbd2e883fa819c81f20
* added check of AutoCreateChildQueueDisabled
Change-Id: Ia011b4393ccd8d4d419a2e46b06a5237d050851c
* removed auto-create-child-queue-enabled check and emit
Change-Id: I7a154124519ecbd81379b46a238707c16db1e82a
2023-02-22 09:46:42 +01:00
4e6e2f318c
YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. ( #5288 )
...
Contributed by Shilun Fan
2023-02-21 14:39:32 +00:00
7e486038ea
YARN-11439. Fix Typo of hadoop-yarn-ui README.md. ( #5405 )
...
Co-authored-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:14:01 +08:00
10e7ca481c
YARN-11441. Revert YARN-10495.
...
This reverts commit 7d3c8ef606
2023-02-17 15:05:06 +00:00
a5f48eacca
YARN-11425. [Federation] Router Supports SubClusterCleaner. ( #5326 )
2023-02-15 14:40:34 -08:00
c3706597a3
YARN-11349. [Federation] Router Support DelegationToken With SQL. ( #5244 )
2023-02-15 14:38:41 -08:00
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
4fcceff535
HADOOP-18620 Avoid using grizzly-http-* APIs ( #5356 )
2023-02-09 10:45:07 +08:00
af20841fb1
YARN-11217. [Federation] Add dumpSchedulerLogs REST APIs for Router. ( #5272 )
2023-02-08 11:48:38 -08:00
a6a9fe17e0
YARN-3657. Federation maintenance mechanisms (simple CLI and command propagation). ( #5348 )
2023-02-06 11:47:07 -08:00
dad73b76c0
YARN-11419. Remove redundant exception capture in NMClientAsyncImpl and improve readability in ContainerShellWebSocket, etc ( #5309 )
...
Co-authored-by: smallzhongfeng <982458633@qq.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-04 10:29:19 +08:00
a70f84098f
HADOOP-18601. Fix build failure with docs profile. ( #5331 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2023-01-31 19:44:19 +09:00
468135a4d9
YARN-11218. [Federation] Add getActivities, getBulkActivities REST APIs for Router. ( #5284 )
2023-01-26 11:14:05 -08:00
cf1b3711cb
YARN-10965. Centralize queue resource calculation based on CapacityVectors. Contributed by Andras Gyori
2023-01-26 19:45:54 +01:00
815cde9810
YARN-6971. Clean up different ways to create resources. Contributed by Riya Khandelwal
2023-01-25 17:28:29 +01:00
29f2230cb6
YARN-5607. Document TestContainerResourceUsage#waitForContainerCompletion. Contributed by Susheel Gupta
2023-01-25 15:13:24 +01:00
3f767a61b1
YARN-8900. [Follow Up] Fix FederationInterceptorREST#invokeConcurrent Inaccurate Order of Subclusters. ( #5260 )
2023-01-19 17:13:55 -08:00
442a5fb285
YARN-11320. [Federation] Add getSchedulerInfo REST APIs for Router. ( #5217 )
2023-01-17 09:36:19 -08:00
4de31123ce
Fix "the the" and friends typos ( #5267 )
...
Signed-off-by: Nikita Eshkeev <neshkeev@yandex.ru>
2023-01-17 03:33:59 +08:00
168fa07801
YARN-11409. Fix Typo of ResourceManager#webapp moudle. ( #5285 )
2023-01-13 11:53:13 +08:00
4520448327
YARN-11374. [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation. ( #5193 )
2023-01-12 15:04:09 -08:00
3d21cff263
YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412. ( #5289 )
...
* YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412.
* YARN-11413. Fix CheckStyle.
* YARN-11413. Fix CheckStyle.
Co-authored-by: slfan1989 <louj1988@@>
2023-01-12 14:29:05 +01:00
7f6cc196f8
YARN-11190. CS Mapping rule bug: User matcher does not work correctly for usernames with dot ( #4471 )
2023-01-11 13:23:04 +01:00
dd49077aed
YARN-6412 aux-services classpath not documented ( #5242 )
2023-01-09 15:30:24 +01:00
0926fa5a2c
YARN-11225. [Federation] Add postDelegationToken postDelegationTokenExpiration cancelDelegationToken REST APIs for Router. ( #5185 )
2023-01-03 02:14:02 -08:00
c44c9f984b
YARN-11393. Fs2cs could be extended to set ULF to -1 upon conversion ( #5201 )
2023-01-02 15:35:16 +01:00
6b67373d10
YARN-11388: Prevent resource leaks in TestClientRMService. ( #5187 )
...
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2022-12-28 11:00:27 -08:00
9668a85d40
YARN-11392 Audit Log missing in ClientRMService ( #5250 ). Contributed by Beibei Zhao.
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-12-27 15:58:53 -08:00
049d1762bd
MAPREDUCE-7428. Fix failing MapReduce tests due to the JUnit upgrades in WebServicesTestUtils ( #5243 )
...
Removed JUnit APIs from WebServicesTestUtils and TestContainerLogsUtils.
They are used by MapReduce modules as well as YARN modules, so the
APIs need to be removed to upgrade the JUnit version on a per-module basis.
Also, this effectively reverts the prior fix in #5209 because it didn't actually
fix the issue.
2022-12-24 04:33:35 +09:00
bf8ab83cd0
YARN-11395. RM UI, RMAttemptBlock can not render FINAL_SAVING. Contributed by Bence Kosztolnik
...
- In the YARN-1345 remove of FINAL_SAVING was missed from RMAttemptBlock
- Same issue was present after YARN-1345 in YARN-4411
- YARN-4411 logic was applied in this commit for FINAL_SAVING
2022-12-23 17:15:36 +01:00
17035da46e
YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router. ( #5175 )
2022-12-22 11:25:09 -08:00
e6056d128a
YARN-10879. Incorrect WARN text in ACL check for application tag based placement ( #5231 )
...
Change-Id: Id892e38fe4c834b1743a0df2f0a40146d3d5a878
2022-12-22 17:20:53 +01:00
6172c3192d
YARN-11358. [Federation] Add FederationInterceptor#allow-partial-result config. ( #5056 )
2022-12-14 14:37:56 -08:00
63b9a6a2b6
YARN-11350. [Federation] Router Support DelegationToken With ZK. ( #5131 )
2022-12-14 09:09:38 -08:00
a71aaef9a9
YARN-11385. Fix hadoop-yarn-server-common module Java Doc Errors. ( #5182 )
2022-12-10 15:03:49 -08:00
ee7d1787cd
YARN-11390. TestResourceTrackerService.testNodeRemovalNormally: Shutdown nodes should be 0 now expected: <1> but was: <0> ( #5190 )
...
Reviewed-by: Peter Szucs
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-12-08 09:52:19 -08:00
f71fd885be
YARN-11373. [Federation] Support refreshQueues refreshNodes API's for Federation. ( #5146 )
2022-12-06 08:17:05 -08:00
dadd3d9138
YARN-11386. Fix issue with classpath resolution ( #5183 )
...
* This PR ensures that all the special notations such as
<CPS> are resolved before getting added to classpath.
2022-12-06 16:32:26 +05:30
60e0fe8709
YARN-11381. Fix hadoop-yarn-common module Java Doc Errors. ( #5179 )
2022-12-02 10:56:17 -08:00
4af4997e11
YARN-11158. Support (Create/Renew/Cancel) DelegationToken API's for Federation. ( #5104 )
2022-12-01 13:20:21 -08:00
5440c75c4a
YARN-10946. AbstractCSQueue: Create separate class for constructing Queue API objects. Contributed by Peter Szucs
2022-12-01 15:11:58 +01:00
f93167e678
YARN-11380. Fix hadoop-yarn-api module Java Doc Errors. ( #5152 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-28 18:54:32 +05:30
1ddc9091f6
YARN-11381. Fix hadoop-yarn-common module Java Doc Errors. ( #5153 ). Contributed by Shilun Fan.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-11-26 21:01:07 +05:30
7cb22eb72d
YARN-11371. [Federation] Refactor FederationInterceptorREST#createNewApplication\submitApplication Use FederationActionRetry. ( #5130 )
2022-11-22 14:38:24 -08:00
3c37a01654
YARN-8262. get_executable in container-executor should provide meaningful error codes. Contributed by Susheel Gupta
2022-11-22 13:37:55 +01:00
2e993fdf4e
YARN-6946. Upgrade JUnit from 4 to 5 in hadoop-yarn-common ( #4717 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-11-21 14:40:02 +09:00
eccd2d0492
YARN-11359. [Federation] Routing admin invocations transparently to multiple RMs. ( #5057 )
2022-11-16 18:00:38 -08:00
22c9f28f4d
YARN-10005. Code improvements in MutableCSConfigurationProvider. Contributed by Peter Szucs
2022-11-12 18:35:49 +01:00
b398a7b003
YARN-11367. [Federation] Fix DefaultRequestInterceptorREST Client NPE. ( #5100 )
2022-11-09 10:25:10 -08:00
845cf8bc28
YARN-11368. [Federation] Improve Yarn Router's Federation Page style. ( #5105 )
2022-11-07 10:13:23 -08:00
5d6ab15860
YARN-11354. [Federation] Add Yarn Router's NodeLabel Web Page. ( #5073 )
2022-11-04 14:39:57 -07:00
2ba982a061
MAPREDUCE-7386. Maven parallel builds (skipping tests) fail ( #4415 )
...
Contributed by Steve Vaughan Jr
2022-11-04 11:50:43 +00:00
b90dfdff3f
YARN-11366. Improve equals, hashCode(), toString() methods of the Federation Base Object. ( #5096 )
2022-11-03 21:33:53 -07:00
e62ba16a02
HADOOP-18484. Upgrade hsqldb to v2.7.1 to mitigate CVE-2022-41853 ( #4991 )
2022-11-02 08:41:27 +01:00
83acb55981
YARN-11364. Docker Container to accept docker Image name with sha256 digest ( #5092 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 14:44:35 -07:00
69225ae5b9
YARN-11363. Remove unused TimelineVersionWatcher and TimelineVersion from hadoop-yarn-server-tests ( #5091 )
...
Reviewed-by: slfan1989 <55643692+slfan1989@users.noreply.github.com>
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-11-01 14:02:06 -07:00
b1f418f802
YARN-11365. Fix NM class not found on Windows ( #5093 )
2022-10-31 09:19:07 -07:00
cbe02c2e77
YARN-11264. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-tests ( #4776 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 04:33:57 +09:00
c096803387
YARN-11339. Upgrade Junit 4 to 5 in hadoop-yarn-services-api ( #4995 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 03:07:16 +09:00
2aae7ffe08
YARN-11337. Upgrade Junit 4 to 5 in hadoop-yarn-applications-mawo ( #4993 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-30 02:25:53 +09:00
070a2d4880
YARN-11332. [Federation] Improve FederationClientInterceptor#ThreadPool thread pool configuration. ( #4982 )
2022-10-28 15:39:04 -07:00
b1cd88c598
YARN-11229. [Federation] Add checkUserAccessToQueue REST APIs for Router. ( #4929 )
2022-10-28 15:37:35 -07:00
bfb84cd7f6
YARN-11360: Add number of decommissioning/shutdown nodes to YARN cluster metrics. ( #5060 )
2022-10-28 11:07:01 -07:00
ba77530ff4
YARN-11357. Fix FederationClientInterceptor#submitApplication Can't Update SubClusterId ( #5055 )
2022-10-26 16:42:22 -07:00
562b693374
YARN-11356. Upgrade DataTables to 1.11.5 to fix CVEs. Contributed by Bence Kosztolnik.
2022-10-26 22:29:01 +02:00
e6edbf1b4b
YARN-11338. Upgrade Junit 4 to 5 in hadoop-yarn-applications-unmanaged-am-launcher ( #4994 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-25 17:21:12 +09:00
21b7790866
YARN-11336. Upgrade Junit 4 to 5 in hadoop-yarn-applications-catalog-webapp ( #4992 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-25 16:52:31 +09:00
454157a384
YARN-11345. [Federation] Refactoring Yarn Router's Application Web Page. ( #5030 )
2022-10-24 09:35:16 -07:00
d93e6f0cbb
YARN-11295. [Federation] Router Support DelegationToken in MemoryStore mode. ( #5032 )
2022-10-20 13:14:38 -07:00
9adf0ca089
YARN-11342. [Federation] Refactor getNewApplication, submitApplication Use FederationActionRetry. ( #5005 )
2022-10-20 09:22:24 -07:00
48b6f9f335
YARN-11328. Refactoring part of the code of SQLFederationStateStore. ( #4976 )
2022-10-19 16:11:28 -07:00
ee886cacd7
YARN-11247. Remove unused classes introduced by YARN-9615. ( #4720 )
2022-10-18 09:40:39 -04:00
5b52123c9d
YARN-8041. [Router] Federation: Improve Router REST API Metrics. ( #4938 )
2022-10-13 16:54:36 -07:00
1962851356
YARN-11294. [Federation] Router Support DelegationToken store/update/remove Token With MemoryStateStore. ( #4915 )
2022-10-13 16:52:22 -07:00
647457e6ab
YARN-11327. [Federation] Refactoring Yarn Router's Node Web Page. ( #5009 )
2022-10-13 14:05:30 -07:00
bfce21ee08
YARN-11330. use secure XML parsers ( #4981 )
...
Move construction of XML parsers in YARN
modules to using the locked-down parser factory
of HADOOP-18469.
One exception: GpuDeviceInformationParser still supports DTD resolution;
all other features are disabled.
Contributed by P J Fanning
2022-10-13 18:19:19 +01:00
3ff8f58f8c
HADOOP-18360. Update commons-csv from 1.0 to 1.9.0. ( #4928 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-13 12:10:54 +05:30
1ff7e84caf
YARN-11334. Improve SubClusterState#fromString parameter and LogMessage. ( #4988 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-10-13 12:06:24 +05:30
d78b0b39a6
YARN-11323. [Federation] Improve ResourceManager Handler FinishApps. ( #4954 )
2022-10-11 14:53:02 -07:00
9e16f1f883
YARN-11317. [Federation] Refactoring Yarn Router's About Web Page. ( #4946 )
2022-10-11 13:30:48 -07:00
82a88a8ae6
YARN-11315. [Federation] YARN Federation Router Supports Cross-Origin. ( #4934 )
2022-10-11 10:18:50 -07:00
0c515b0ef0
YARN-6766. Add helper method in FairSchedulerAppsBlock to print app info. Contributed by Riya Khandelwal
2022-10-10 15:30:33 +02:00
9a7d0e7ed0
YARN-11260. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice ( #4775 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-10 00:28:03 +09:00
5eddec8c46
HADOOP-18468: Upgrade jettison to 1.5.1 to fix CVE-2022-40149 ( #4937 )
...
Contributed by PJ Fanning
2022-10-07 15:44:01 +01:00
b31b3ea0f6
YARN-11187. Remove WhiteBox in yarn module. ( #4463 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-10-06 23:13:33 +09:00
1a9faf123d
YARN-11313. [Federation] Add SQLServer Script and Supported DB Version in Federation.md. ( #4927 )
2022-10-05 13:25:20 -07:00
a708ff96f1
YARN-11324. Fix some PBImpl classes to avoid NPE. ( #4961 )
2022-10-04 14:30:20 -04:00
874a004347
YARN-11318. Improve FederationInterceptorREST#createInterceptorForSubCluster Use WebAppUtils. ( #4939 )
2022-10-04 09:18:01 -07:00
22bd5e3b53
YARN-11238. Optimizing FederationClientInterceptor Call with Parallelism. ( #4904 )
2022-10-04 09:17:00 -07:00
07581f1ab2
YARN-6169 message on empty configuration file improved ( #4952 )
2022-10-03 23:31:06 -04:00
42d883937d
YARN-11310. [Federation] Refactoring Yarn Router's Federation Web Page. ( #4924 )
2022-09-27 13:31:52 -07:00
bfd6415827
YARN-11290. Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster. ( #4846 )
2022-09-27 13:28:52 -07:00
0e65f4cc04
YARN-11316. Fix Yarn federation.md table format. ( #4935 )
2022-09-26 15:47:08 -07:00
aeba204fa2
YARN-11306. [Federation] Refactor NM#FederationInterceptor#recover Code. ( #4897 )
2022-09-26 15:46:06 -07:00
603e9bd745
YARN-11271. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-common ( #4774 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:30:58 +09:00
fd0415c44a
YARN-11270. Upgrade JUnit from 4 to 5 in hadoop-yarn-server-timelineservice-hbase-client ( #4773 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-09-25 23:26:06 +09:00
e526f48fa4
YARN-11296. [Federation] Fix SQLFederationStateStore#Sql script bug. ( #4858 )
2022-09-22 16:40:47 -07:00
917aef75fc
YARN-11255. Support loading alternative docker client config from system environment ( #4884 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
2022-09-21 16:54:02 +05:30
4c5a7cc6fc
YARN-11307. Fix Yarn Router Broken Link. ( #4905 )
2022-09-20 23:10:09 -04:00
slfan1989
susheel-gupta
Steve Loughran
Viraj Jasani
jokercurry
Masatake Iwasaki
Szilard Nemeth
Nikita Eshkeev
Szilard Nemeth
Riya Khandelwal
Chris Nauroth
curie71
Akira Ajisaka
Bence Kosztolnik
K0K0V0K
Gautham B A
Ashutosh Gupta
Steve Vaughan
PJ Fanning
Szilard Nemeth