druid/docs
Lucas Capistrant 58ce2e55d8
Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284)
* dynamic coord config adding more balancing control

add new dynamic coordinator config, maxSegmentsToConsiderPerMove. This
config caps the number of segments that are iterated over when selecting
a segment to move. The default value combined with current balancing
strategies will still iterate over all provided segments. However,
setting this value to something > 0 will cap the number of segments
visited. This could make sense in cases where a cluster has a very large
number of segments and the admins prefer less iterations vs a thorough
consideration of all segments provided.

* fix checkstyle failure

* Make doc more detailed for admin to understand when/why to use new config

* refactor PR to use a % of segments instead of raw number

* update the docs

* remove bad doc line

* fix typo in name of new dynamic config

* update RservoirSegmentSampler to gracefully deal with values > 100%

* add handler for <= 0 in ReservoirSegmentSampler

* fixup CoordinatorDynamicConfigTest naming and argument ordering

* fix items in docs after spellcheck flags

* Fix lgtm flag on missing space in string literal

* improve documentation for new config

* Add default value to config docs and add advice in cluster tuning doc

* Add percentOfSegmentsToConsiderPerMove to web console coord config dialog

* update jest snapshot after console change

* fix spell checker errors

* Improve debug logging in getRandomSegmentBalancerHolder to cover all bad inputs for % of segments to consider

* add new config back to web console module after merge with master

* fix ReservoirSegmentSamplerTest

* fix line breaks in coordinator console dialog

* Add a test that helps ensure not regressions for percentOfSegmentsToConsiderPerMove

* Make improvements based off of feedback in review

* additional cleanup coming from review

* Add a warning log if limit on segments to consider for move can't be calcluated

* remove unused import

* fix tests for CoordinatorDynamicConfig

* remove precondition test that is redundant in CoordinatorDynamicConfig Builder class
2020-12-22 08:27:55 -08:00
..
_bin De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
assets Security overview documentation (#10339) 2020-11-19 15:24:58 -08:00
comparisons Spellcheck docs (#8548) 2019-09-17 12:47:30 -07:00
configuration Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284) 2020-12-22 08:27:55 -08:00
dependencies cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
design cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
development cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
ingestion cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
misc cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
operations Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284) 2020-12-22 08:27:55 -08:00
querying integration test for coordinator and overlord leadership client (#10680) 2020-12-17 22:50:12 -08:00
tutorials cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00