opensearch-docs-cn/_ml-commons-plugin/cluster-settings.md

156 lines
3.9 KiB
Markdown
Raw Normal View History

---
layout: default
title: ML Commons cluster settings
has_children: false
nav_order: 10
---
# ML Commons cluster settings
To enhance and customize your OpenSearch cluster for machine learning (ML), you can add and modify several configuration settings for the ML commons plugin in your 'opensearch.yml' file.
## Run tasks and models on ML nodes only
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
If `true`, ML Commons tasks and models run machine learning (ML) tasks on ML nodes only. If `false`, tasks and models run on ML nodes first. If no ML nodes exist, tasks and models run on data nodes. We recommend that you do not set this value to "false" on production clusters.
### Setting
```
Add GPU acceleration documentation (#2384) * Add GPU acceleration documentation Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Address tech feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Address technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Adjust model size sentence Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add optional to neuron step Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add Jeff's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add copy and customize for Inferntia examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/gpu-acceleration.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/gpu-acceleration.md Co-authored-by: Nate Bower <nbower@amazon.com> * Apply suggestions from code review Co-authored-by: Nate Bower <nbower@amazon.com> * Fix link Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com> * Update _ml-commons-plugin/gpu-acceleration.md Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com> * Update _ml-commons-plugin/gpu-acceleration.md Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com> * Update _ml-commons-plugin/gpu-acceleration.md Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com> * Fix numbering in final section Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add final tech feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * A couple more suggestion Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Co-authored-by: Yaliang Wu <ylwu@amazon.com> * Fix Neural Search link Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add experimental warning Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/gpu-acceleration.md Co-authored-by: Yaliang Wu <ylwu@amazon.com> * Final tech feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Move OpenSearch to step 2. Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Nate Bower <nbower@amazon.com> Co-authored-by: Caroline <113052567+carolxob@users.noreply.github.com> Co-authored-by: Yaliang Wu <ylwu@amazon.com>
2023-01-18 14:31:52 -05:00
plugins.ml_commons.only_run_on_ml_node: true
```
### Values
- Default value: `true`
- Value range: `true` or `false`
## Dispatch tasks to ML node
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
`round_robin` dispatches ML tasks to ML nodes using round robin routing. `least_load` gathers runtime information from all ML nodes, like JVM heap memory usage and running tasks, and then dispatches the tasks to the ML node with the lowest load.
### Setting
```
plugins.ml_commons.task_dispatch_policy: round_robin
```
### Values
- Dafault value: `round_robin`
- Value range: `round_robin` or `least_load`
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
## Set sync job intervals
When returning runtime information with the [profile API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api#profile), ML Commons will run a regular job to sync newly loaded or unloaded models on each node. When set to `0`, ML Commons immediately stops sync up jobs.
When returning runtime information with the [profile API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api#profile), ML Commons will run a regular sync up job to sync up newly loaded or unloaded models on each node. When set to `0`, ML Commons immediately stops sync up jobs.
### Setting
```
plugins.ml_commons.sync_up_job_interval_in_seconds: 10
```
### Values
- Default value: `10`
- Value range: [0, 86,400]
## Predict monitoring requests
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
Controls how many upload model tasks can run in parallel on one node. If set to `0`, you cannot upload models to any node.
### Setting
```
plugins.ml_commons.monitoring_request_count: 100
```
### Value range
- Default value: `100`
- Value range: [0, 100,000,000]
## Upload model tasks per node
Controls how many upload model tasks can run in parallel on one node. If set to `0`, you cannot upload models to any node.
### Setting
```
plugins.ml_commons.max_upload_model_tasks_per_node: 10
```
### Values
- Default value: `10`
- Value range: [0, 10]
## Load model tasks per node
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
Controls how many load model tasks can run in parallel on one node. If set to 0, you cannot load models to any node.
### Setting
```
plugins.ml_commons.max_load_model_tasks_per_node: 10
```
### Values
- Default value: `10`
- Value range: [0, 10]
## Add trusted URL
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
The default value allows you to upload a model file from any http/https/ftp/local file. You can change this value to restrict trusted model URLs.
### Setting
```
plugins.ml_commons.trusted_url_regex: ^(https?\|ftp\|file)://[-a-zA-Z0-9+&@#/%?=~_\|!:,.;]*[-a-zA-Z0-9+&@#/%=~_\|]
```
### Values
- Default value: `^(https?\|ftp\|file)://[-a-zA-Z0-9+&@#/%?=~_\|!:,.;]*[-a-zA-Z0-9+&@#/%=~_\|]`
- Value range: Java regular expression (regex) string
Add cluster awareness and decommission docs (#2438) * Add cluster awareness and decommission docs Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> * Edit technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add new cluster awareness examples Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add technical feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> * Add Caroline's feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add one more tweak Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _ml-commons-plugin/cluster-settings.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-awareness.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update _api-reference/cluster-decommission.md Co-authored-by: Nate Bower <nbower@amazon.com> * Add editoiral feedback Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typos Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Final editorial note Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: Alice Williams <88908598+alicejw-aws@users.noreply.github.com> Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM> Co-authored-by: Nate Bower <nbower@amazon.com>
2023-01-23 18:13:07 -05:00
## Assign task timeout
Assigns how long in seconds an ML task will live. After the timeout, the task will fail.
### Setting
```
plugins.ml_commons.ml_task_timeout_in_seconds: 600
```
### Values
- Default value: 600
- Value range: [1, 86400]
## Set native memory threshold
Sets a circuit breaker that checks all system memory usage before running an ML task. If the native memory exceeds the threshold, OpenSearch throws an exception and stops running any ML task.
Values are based on the percentage of memory available. When set to `0`, no ML tasks will run. When set to `100`, the circuit breaker closes and no threshold exists.
### Setting
```
plugins.ml_commons.native_memory_threshold: 90
```
### Values
- Default value: 90
- Value range: [0, 100]