2e32cc68fc
YARN-10985. Add some tests to verify ACL behaviour in CapacitySchedulerConfiguration ( #3570 ). Contributed by Szilard Nemeth
2021-10-28 15:19:32 +02:00
d598904046
YARN-10904. Investigate: Remove unnecessary fields from AbstractCSQueue ( #3551 ) contributed by Szilard Nemeth
2021-10-27 19:03:45 +02:00
66ac476b48
YARN-10924. Clean up CapacityScheduler#initScheduler ( #3581 ) Contributed by Szilard Nemeth
2021-10-27 17:13:49 +02:00
9cfd8d0a83
YARN-10909. AbstractCSQueue: Annotate all methods with VisibleForTesting that are only used by test code. Contributed by JackWangCS, Szilard Nemeth
2021-10-23 14:47:09 +02:00
32ecaed9c3
YARN-10930. Introduce universal capacity resource vector. Contributed by Andras Gyori
2021-10-22 17:32:33 +02:00
23772d946b
YARN-10948. Rename SchedulerQueue#activeQueue to activateQueue. Contributed by Adam Antal
2021-10-22 16:33:03 +02:00
516f36c6f1
HADOOP-17967. Keep restrict-imports-enforcer-rule for Guava VisibleForTesting in hadoop-main pom ( #3555 )
2021-10-21 16:54:25 +09:00
d286994009
YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne)
2021-10-20 22:18:36 +00:00
20aeb5ecc3
YARN-10916. Investigate and simplify GuaranteedOrZeroCapacityOverTimePolicy#computeQueueManagementChanges. Contributed by Szilard Nemeth
2021-10-20 15:52:37 +02:00
35b8441fd9
YARN-10949. Simplify AbstractCSQueue#updateMaxAppRelatedField and find a more meaningful name for this method. Contributed by Andras Gyori
2021-10-20 12:56:41 +02:00
414d40155c
YARN-10958. Use correct configuration for Group service init in CSMappingPlacementRule ( #3560 )
...
* YARN-10958. Initial commit
* Fix javadoc + behaviour
* Fix review comments
* fix checkstyle + blanks
* fix checkstyle + blanks
* Fix checkstyle + blanks
2021-10-20 10:48:42 +02:00
616cea2e80
YARN-10954. Remove commented code block from CSQueueUtils#loadCapacitiesByLabelsFromConf. Contributed by Andras Gyori
2021-10-19 13:06:45 +02:00
025f97c8c2
YARN-10942. Move AbstractCSQueue fields to separate objects that are tracking usage. Contributed by Szilard Nemeth
2021-10-19 12:24:58 +02:00
ae95caa60e
YARN-10976. Fix resource leak due to Files.walk ( #3552 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-10-18 15:24:15 +09:00
d336227e5c
HADOOP-17963. Replace Guava VisibleForTesting by Hadoop's own annotation in hadoop-yarn-project modules ( #3541 )
2021-10-14 18:03:01 +09:00
1151edf12e
HADOOP-17956. Replace all default Charset usage with UTF-8 ( #3529 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-10-14 13:07:24 +09:00
8c271caa81
YARN-6862. Nodemanager resource usage metrics sometimes are negative. Contributed by Benjamin Teke
2021-10-12 17:35:06 +02:00
35eff54556
YARN-10934. Fix LeafQueue#activateApplication NPE when the user of the pending application is missing from usersManager. Contributed by Benjamin Teke
...
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2021-10-07 20:11:42 +02:00
4b1b6b858a
YARN-10953. Make CapacityScheduler#getOrCreateQueueFromPlacementConte… Contributed by Andras Gyori
2021-10-07 17:09:38 +02:00
5535d66fb5
YARN-10692. Do not extend from CapacitySchedulerTestBase when not needed. Contributed by Tamas Domok
...
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-10-07 16:21:05 +02:00
8071dbb9c6
HADOOP-17950. Provide replacement for deprecated APIs of commons-io IOUtils ( #3515 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-10-07 10:58:29 +09:00
ed8e879320
YARN-10823. Expose all node labels for root without explicit configurations. Contributed by Andras Gyori
2021-10-01 04:20:36 +02:00
3c9e3d5321
YARN-10960. Extract test queues and related methods from TestCapacityScheduler. Contributed by Tamas Domok
...
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-10-01 04:05:52 +02:00
4bd0c36189
YARN-10970. Standby RM should expose prom endpoint ( #3480 )
...
Reviewed-by: Adam Antal <adamantal@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-09-29 15:46:55 +09:00
51bd528bd3
YARN-10957. Using invokeConcurrent Overload with Collection in getClusterMetrics ( #3439 )
2021-09-28 09:51:18 -07:00
8f4456d4a1
YARN-10961. TestCapacityScheduler: reuse appHelper where feasible. Contributed by Tamas Domok
...
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-09-21 16:13:04 +02:00
9f6430c9ed
YARN-10897. Introduce QueuePath class. Contributed by Andras Gyori
2021-09-21 16:08:24 +02:00
4df4389325
YARN-10911. AbstractCSQueue: Create a separate class for usernames and weights that are travelling in a Map. Contributed by Szilard Nemeth
2021-09-20 16:47:46 +02:00
a9b2469a53
YARN-10950. Code cleanup in QueueCapacities ( #3454 )
2021-09-19 14:42:02 +02:00
5dcb23c1f0
YARN-10937. Fix log message arguments in LogAggregationFileController ( #3450 )
2021-09-19 14:35:34 +02:00
aa74a303ed
YARN-10913. AbstractCSQueue: Group preemption methods and fields into a separate class ( #3420 )
2021-09-19 13:11:56 +02:00
f5c76c8e31
YARN-10936. Log typo corrected. ( #3449 )
...
Contributed by: Tibor Kovacs
2021-09-17 08:27:51 +02:00
43f0a34dd4
YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne.
2021-09-15 20:03:45 +00:00
5dc2f7b137
YARN-10915. AbstractCSQueue: Simplify complex logic in methods: deriveCapacityFromAbsoluteConfigurations and updateEffectiveResources ( #3418 )
...
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2021-09-14 18:05:40 +02:00
783d94f5cd
YARN-10917. Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy ( #3403 )
...
* YARN-10917. Investigate and simplify CapacitySchedulerConfigValidator#validateQueueHierarchy.
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-09-14 17:54:25 +02:00
63c892278f
YARN-10912. AbstractCSQueue#updateConfigurableResourceRequirement: Separate validation logic from initialization logic ( #3390 )
...
- capacityConfigType update is extracted to a separate method
- validation logic is extracted to a helper function
- min resource must not be greater than max resource is now checked
after the max resource is updated
Change-Id: I731c2639281721afed32c30854bafcf048d6ee28
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-09-14 17:30:44 +02:00
ad1d40970a
YARN-10928. Support default queue config for minimum-user-limit-percent/user-limit-factor ( #3389 )
...
Contributed by Weihao Zheng
2021-09-13 11:06:53 +08:00
d8026e387e
YARN-10903. Fix the headroom check in ParentQueue and RegularContainerAllocator for DRF ( #3352 )
...
Contributed by Jie Wang <jie.wang@hulu.com>
2021-09-13 10:54:11 +08:00
971f1b8b0a
YARN-10872. Replace getPropsWithPrefix calls in AutoCreatedQueueTemplate ( #3396 )
...
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2021-09-10 17:32:42 +02:00
811fd23f23
YARN-10852. Optimise CSConfiguration getAllUserWeightsForQueue ( #3392 )
2021-09-10 16:59:46 +02:00
b229e5a345
YARN-10910. AbstractCSQueue#setupQueueConfigs: Separate validation logic from initialization logic ( #3407 )
...
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2021-09-10 16:48:58 +02:00
29a6f141d4
YARN-10914. Simplify duplicated code for tracking ResourceUsage in AbstractCSQueue ( #3402 )
...
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-09-10 15:57:46 +02:00
dee6dc2f89
YARN-10893. Adding metrics for getClusterMetrics and getApplications APIs in FederationClientInterceptor ( #3325 )
2021-09-09 09:20:57 -07:00
a186460004
YARN-10829. Follow up: Adding null checks before merging ResourceUsage Report ( #3252 )
2021-09-08 10:36:56 -07:00
2ff3fc50e4
YARN-10870. Missing user filtering check -> yarn.webapp.filter-entity-list-by-user for RM Scheduler page. Contributed by Gergely Pollak
2021-09-08 18:01:39 +02:00
5e166898aa
YARN-10901. Permission checking error on an existing directory in LogAggregationFileController#verifyAndCreateRemoteLogDir ( #3355 )
...
Co-authored-by: Tamas Domok <tdomok@cloudera.com>
2021-09-08 17:04:32 +02:00
4e209a31da
YARN-10919. Remove LeafQueue#scheduler field ( #3382 )
...
Co-authored-by: Jie Wang <jie.wang@hulu.com>
2021-09-08 16:19:29 +02:00
40e639ad07
YARN-10646. TestCapacitySchedulerWeightMode test descriptor comments doesnt reflect the correct scenario ( #3339 )
...
Co-authored-by: Benjamin Teke <bteke@cloudera.com>
2021-09-08 16:11:04 +02:00
3024a47026
Add documentation for YARN-10623 auto refresh queue conf in CS ( #3279 )
2021-09-08 16:03:15 +02:00
d9cb69853b
YARN-10522. Document for Flexible Auto Queue Creation in Capacity Scheduler
2021-09-08 15:43:57 +02:00
Szilard Nemeth
Jack
9uapaw
Adam Antal
Viraj Jasani
Ahmed Hussein
lujiefsi
Szilard Nemeth
Benjamin Teke
Tamas Domok
Neil
Akshat Bordia
TiborKovacsCloudera
Eric Badger
Weihao Zheng
Szilard Nemeth
zhuqi