druid/docs/operations
Lucas Capistrant 58ce2e55d8
Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284)
* dynamic coord config adding more balancing control

add new dynamic coordinator config, maxSegmentsToConsiderPerMove. This
config caps the number of segments that are iterated over when selecting
a segment to move. The default value combined with current balancing
strategies will still iterate over all provided segments. However,
setting this value to something > 0 will cap the number of segments
visited. This could make sense in cases where a cluster has a very large
number of segments and the admins prefer less iterations vs a thorough
consideration of all segments provided.

* fix checkstyle failure

* Make doc more detailed for admin to understand when/why to use new config

* refactor PR to use a % of segments instead of raw number

* update the docs

* remove bad doc line

* fix typo in name of new dynamic config

* update RservoirSegmentSampler to gracefully deal with values > 100%

* add handler for <= 0 in ReservoirSegmentSampler

* fixup CoordinatorDynamicConfigTest naming and argument ordering

* fix items in docs after spellcheck flags

* Fix lgtm flag on missing space in string literal

* improve documentation for new config

* Add default value to config docs and add advice in cluster tuning doc

* Add percentOfSegmentsToConsiderPerMove to web console coord config dialog

* update jest snapshot after console change

* fix spell checker errors

* Improve debug logging in getRandomSegmentBalancerHolder to cover all bad inputs for % of segments to consider

* add new config back to web console module after merge with master

* fix ReservoirSegmentSamplerTest

* fix line breaks in coordinator console dialog

* Add a test that helps ensure not regressions for percentOfSegmentsToConsiderPerMove

* Make improvements based off of feedback in review

* additional cleanup coming from review

* Add a warning log if limit on segments to consider for move can't be calcluated

* remove unused import

* fix tests for CoordinatorDynamicConfig

* remove precondition test that is redundant in CoordinatorDynamicConfig Builder class
2020-12-22 08:27:55 -08:00
..
alerts.md Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
api-reference.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
auth-ldap.md Security overview documentation (#10339) 2020-11-19 15:24:58 -08:00
basic-cluster-tuning.md Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284) 2020-12-22 08:27:55 -08:00
deep-storage-migration.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
druid-console.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
dump-segment.md roaring bitmaps by default (#9548) 2020-03-23 18:15:57 -07:00
dynamic-config-provider.md document DynamicConfigProvider for kafka consumer properties (#10658) 2020-12-10 08:24:33 -08:00
export-metadata.md Update Kafka loading docs to use the streaming data loader (#8544) 2019-09-22 15:00:52 -07:00
getting-started.md Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
high-availability.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
http-compression.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
insert-segment-to-db.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
kubernetes.md kubernetes based discovery druid extension to run Druid on K8S without Zookeeper (#10544) 2020-12-14 21:10:31 -08:00
management-uis.md Druid user permissions (#10047) 2020-06-23 17:39:48 -07:00
metadata-migration.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
metrics.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
other-hadoop.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
password-provider.md Security overview documentation (#10339) 2020-11-19 15:24:58 -08:00
pull-deps.md Fix typo (#10385) 2020-09-11 16:31:36 -07:00
reset-cluster.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
rolling-updates.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
rule-configuration.md Fix broadcast rule drop and docs (#10019) 2020-06-12 02:33:28 -07:00
security-overview.md Security overview documentation (#10339) 2020-11-19 15:24:58 -08:00
security-user-auth.md Security overview documentation (#10339) 2020-11-19 15:24:58 -08:00
segment-optimization.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
single-server.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
tls-support.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
use_sbt_to_build_fat_jar.md Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00