ad024999d8
HADOOP-18487. change scope so that protobuf explicitly exported again
...
rename protobuf2.scope to transient.protobuf2.scope as it is
only used in transient exports
Change-Id: Icfa54ad440834314ee69ec9bf3d585f981e20fef
2023-03-06 10:20:46 +00:00
f198ebc0cf
HADOOP-18487. protobuf -style and imports
...
Change-Id: I6e33572b42ebd79ed1b8f81f20cc0e17794b0c4a
2023-03-06 10:20:46 +00:00
6c2197b645
HADOOP-18487. making protobuf 2.5 optional
...
spotbugs and cleanup of uses of getRemoteException()
Change-Id: I5140094acc422a4df872ffa06fd944b7ea55ce9c
2023-03-06 10:20:45 +00:00
5f86dd58bd
HADOOP-18487. making protobuf 2.5 optional
...
* new org.apache.hadoop.ipc.internal for internal only classes
* with a ShadedProtobufHelper in there which has shaded protobuf refs
only, so guaranteed not to need protobuf-2.5 on the CP
* findbugs, protobuf source patch etc
* can spec new export policies for the protobuf jar in hadoop-common
and other places it is referenced. hadoop-common back @compile
Change-Id: I61a99d2fd673259ab50d000f28a29e8a38aaf1b1
2023-03-06 10:20:45 +00:00
ce2b9d98cb
HADOOP-18487. protobuf 2.5 marked as provided.
...
The methods which were overloaded with shaded/unshaded protobuf paramters
now have @Private ones with different names. the existing overloaded
methods are still there but deprecated, in case things use them.
* removed the extra imports needed before this change went in
* protobuf2.scope is the name for the scope var
+added a test for exception extraction, so yetus stops complaining
I am thinking of merging this in to 3.3.5 with the scope set to "compile",
as we get today. That way
* the overloading changes are in
* anyone who wants to cut their own release without protobuf2.5 can do it
Change-Id: I3423720c7047c63f7c9797d6e386774bff10b21a
2023-03-06 10:20:45 +00:00
91acb8da23
HADOOP-18487. protobuf 2.5 marked as provided.
...
The option protobuf.scope defines whether the protobuf 2.5.0
dependency is marked as provided or not.
* all declarations except those in yarn-csi are updated
* those modules which don't compile without their own explicit
import (hadoop-hdfs-client, hadoop-hdfs-rbf)
It's actually interesting to see where/how that compile fails
hadoop-hdfs-client: ClientNamenodeProtocolTranslatorPB
hadoop-hdfs-rbf:RouterAdminProtocolTranslatorPB
both with "class file for com.google.protobuf.ServiceException not found",
even though *neither class uses it*
what they do have is references to ProtobufHelper.getRemoteException(),
which is overloaded to both the shaded ServiceException and the original one
Hypothesis: the javac overload resolution needs to look at the entire
class hierarchy before it can decide which one to use.
Proposed: add a new method
ioe extractException(org.apache.hadoop.thirdparty.protobuf.ServiceException)
and move our own code to it. Without the overloading the classes should not
be needed
Change-Id: I70354abfe3f1fdc03c418dac88e60f8cc4929a33
2023-03-06 10:20:45 +00:00
2a0dc2ab2f
YARN-11383. Workflow priority mappings is case sensitive ( #5171 )
...
Contributed by Aparajita Choudhary
2023-03-05 21:25:16 +05:30
bcc51ce2c5
YARN-11375. [Federation] Support refreshAdminAcls、refreshServiceAcls API's for Federation. ( #5312 )
2023-02-28 14:44:00 -08:00
8798b94ee1
YARN-11221. [Federation] Add replaceLabelsOnNodes, replaceLabelsOnNode REST APIs for Router. ( #5302 )
2023-02-27 09:34:39 -08:00
25ebd0b8b1
YARN-11222. [Federation] Add addToClusterNodeLabels, removeFromClusterNodeLabels REST APIs for Router. ( #5328 )
2023-02-24 10:52:57 -08:00
27a54955f9
YARN-5604. [Federation] Add versioning for FederationStateStore. ( #5394 )
2023-02-24 10:51:19 -08:00
2e997d818d
YARN-11370. [Federation] Refactor MemoryFederationStateStore code. ( #5126 )
2023-02-22 12:37:35 -08:00
49b8ac19f2
YARN-11408. Add a check of autoQueueCreation is disabled for emitDefaultUserLimitFactor method ( #5278 )
...
* added a check of autoQueueCreationV2Disabled
Change-Id: If1e36c5969d270c1b81a4bbd2e883fa819c81f20
* added check of AutoCreateChildQueueDisabled
Change-Id: Ia011b4393ccd8d4d419a2e46b06a5237d050851c
* removed auto-create-child-queue-enabled check and emit
Change-Id: I7a154124519ecbd81379b46a238707c16db1e82a
2023-02-22 09:46:42 +01:00
4e6e2f318c
YARN-11394. Fix hadoop-yarn-server-resourcemanager module Java Doc Errors. ( #5288 )
...
Contributed by Shilun Fan
2023-02-21 14:39:32 +00:00
7e486038ea
YARN-11439. Fix Typo of hadoop-yarn-ui README.md. ( #5405 )
...
Co-authored-by: Shilun Fan <slfan1989@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-18 13:14:01 +08:00
10e7ca481c
YARN-11441. Revert YARN-10495.
...
This reverts commit 7d3c8ef606
2023-02-17 15:05:06 +00:00
a5f48eacca
YARN-11425. [Federation] Router Supports SubClusterCleaner. ( #5326 )
2023-02-15 14:40:34 -08:00
c3706597a3
YARN-11349. [Federation] Router Support DelegationToken With SQL. ( #5244 )
2023-02-15 14:38:41 -08:00
90de1ff151
HADOOP-18206 Cleanup the commons-logging references and restrict its usage in future ( #5315 )
2023-02-14 03:24:06 +08:00
4fcceff535
HADOOP-18620 Avoid using grizzly-http-* APIs ( #5356 )
2023-02-09 10:45:07 +08:00
af20841fb1
YARN-11217. [Federation] Add dumpSchedulerLogs REST APIs for Router. ( #5272 )
2023-02-08 11:48:38 -08:00
a6a9fe17e0
YARN-3657. Federation maintenance mechanisms (simple CLI and command propagation). ( #5348 )
2023-02-06 11:47:07 -08:00
dad73b76c0
YARN-11419. Remove redundant exception capture in NMClientAsyncImpl and improve readability in ContainerShellWebSocket, etc ( #5309 )
...
Co-authored-by: smallzhongfeng <982458633@qq.com>
Reviewed-by: Shilun Fan <slfan1989@apache.org>
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2023-02-04 10:29:19 +08:00
a70f84098f
HADOOP-18601. Fix build failure with docs profile. ( #5331 )
...
Reviewed-by: Steve Loughran <stevel@apache.org>
2023-01-31 19:44:19 +09:00
468135a4d9
YARN-11218. [Federation] Add getActivities, getBulkActivities REST APIs for Router. ( #5284 )
2023-01-26 11:14:05 -08:00
cf1b3711cb
YARN-10965. Centralize queue resource calculation based on CapacityVectors. Contributed by Andras Gyori
2023-01-26 19:45:54 +01:00
815cde9810
YARN-6971. Clean up different ways to create resources. Contributed by Riya Khandelwal
2023-01-25 17:28:29 +01:00
29f2230cb6
YARN-5607. Document TestContainerResourceUsage#waitForContainerCompletion. Contributed by Susheel Gupta
2023-01-25 15:13:24 +01:00
3f767a61b1
YARN-8900. [Follow Up] Fix FederationInterceptorREST#invokeConcurrent Inaccurate Order of Subclusters. ( #5260 )
2023-01-19 17:13:55 -08:00
442a5fb285
YARN-11320. [Federation] Add getSchedulerInfo REST APIs for Router. ( #5217 )
2023-01-17 09:36:19 -08:00
4de31123ce
Fix "the the" and friends typos ( #5267 )
...
Signed-off-by: Nikita Eshkeev <neshkeev@yandex.ru>
2023-01-17 03:33:59 +08:00
168fa07801
YARN-11409. Fix Typo of ResourceManager#webapp moudle. ( #5285 )
2023-01-13 11:53:13 +08:00
4520448327
YARN-11374. [Federation] Support refreshSuperUserGroupsConfiguration、refreshUserToGroupsMappings API's for Federation. ( #5193 )
2023-01-12 15:04:09 -08:00
3d21cff263
YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412. ( #5289 )
...
* YARN-11413. Fix Junit Test ERROR Introduced By YARN-6412.
* YARN-11413. Fix CheckStyle.
* YARN-11413. Fix CheckStyle.
Co-authored-by: slfan1989 <louj1988@@>
2023-01-12 14:29:05 +01:00
7f6cc196f8
YARN-11190. CS Mapping rule bug: User matcher does not work correctly for usernames with dot ( #4471 )
2023-01-11 13:23:04 +01:00
dd49077aed
YARN-6412 aux-services classpath not documented ( #5242 )
2023-01-09 15:30:24 +01:00
0926fa5a2c
YARN-11225. [Federation] Add postDelegationToken postDelegationTokenExpiration cancelDelegationToken REST APIs for Router. ( #5185 )
2023-01-03 02:14:02 -08:00
c44c9f984b
YARN-11393. Fs2cs could be extended to set ULF to -1 upon conversion ( #5201 )
2023-01-02 15:35:16 +01:00
6b67373d10
YARN-11388: Prevent resource leaks in TestClientRMService. ( #5187 )
...
Signed-off-by: Shilun Fan <slfan1989@apache.org>
2022-12-28 11:00:27 -08:00
9668a85d40
YARN-11392 Audit Log missing in ClientRMService ( #5250 ). Contributed by Beibei Zhao.
...
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-12-27 15:58:53 -08:00
049d1762bd
MAPREDUCE-7428. Fix failing MapReduce tests due to the JUnit upgrades in WebServicesTestUtils ( #5243 )
...
Removed JUnit APIs from WebServicesTestUtils and TestContainerLogsUtils.
They are used by MapReduce modules as well as YARN modules, so the
APIs need to be removed to upgrade the JUnit version on a per-module basis.
Also, this effectively reverts the prior fix in #5209 because it didn't actually
fix the issue.
2022-12-24 04:33:35 +09:00
bf8ab83cd0
YARN-11395. RM UI, RMAttemptBlock can not render FINAL_SAVING. Contributed by Bence Kosztolnik
...
- In the YARN-1345 remove of FINAL_SAVING was missed from RMAttemptBlock
- Same issue was present after YARN-1345 in YARN-4411
- YARN-4411 logic was applied in this commit for FINAL_SAVING
2022-12-23 17:15:36 +01:00
17035da46e
YARN-11226. [Federation] Add createNewReservation, submitReservation, updateReservation, deleteReservation REST APIs for Router. ( #5175 )
2022-12-22 11:25:09 -08:00
e6056d128a
YARN-10879. Incorrect WARN text in ACL check for application tag based placement ( #5231 )
...
Change-Id: Id892e38fe4c834b1743a0df2f0a40146d3d5a878
2022-12-22 17:20:53 +01:00
6172c3192d
YARN-11358. [Federation] Add FederationInterceptor#allow-partial-result config. ( #5056 )
2022-12-14 14:37:56 -08:00
63b9a6a2b6
YARN-11350. [Federation] Router Support DelegationToken With ZK. ( #5131 )
2022-12-14 09:09:38 -08:00
a71aaef9a9
YARN-11385. Fix hadoop-yarn-server-common module Java Doc Errors. ( #5182 )
2022-12-10 15:03:49 -08:00
ee7d1787cd
YARN-11390. TestResourceTrackerService.testNodeRemovalNormally: Shutdown nodes should be 0 now expected: <1> but was: <0> ( #5190 )
...
Reviewed-by: Peter Szucs
Signed-off-by: Chris Nauroth <cnauroth@apache.org>
2022-12-08 09:52:19 -08:00
f71fd885be
YARN-11373. [Federation] Support refreshQueues refreshNodes API's for Federation. ( #5146 )
2022-12-06 08:17:05 -08:00
dadd3d9138
YARN-11386. Fix issue with classpath resolution ( #5183 )
...
* This PR ensures that all the special notations such as
<CPS> are resolved before getting added to classpath.
2022-12-06 16:32:26 +05:30
Steve Loughran
Varun Saxena
slfan1989
susheel-gupta
Viraj Jasani
jokercurry
Masatake Iwasaki
Szilard Nemeth
Nikita Eshkeev
Szilard Nemeth
Riya Khandelwal
Chris Nauroth
curie71
Akira Ajisaka
Bence Kosztolnik
K0K0V0K
Gautham B A