Add ML node settings and dynamic node description (#787)

* Add ML node instructions

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Change cluster settings from ml to dynamic

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Quick typo fix

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Adjust ML docs to remove dymanic mention. Adjust CAT example to be general

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add editorial feedback.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Fix columns in example

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
This commit is contained in:
Naarcha-AWS 2022-07-21 15:57:45 -05:00 committed by GitHub
parent 0b01755beb
commit 6336b83b14
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 12 additions and 5 deletions

View File

@ -23,3 +23,12 @@ There are two reserved user roles that can use of the ML commons plugin.
- `ml_full_access`: Full access to all ML features, including starting new ML tasks and reading or deleting models.
- `ml_readonly_access`: Can only read ML tasks, trained models and statistics relevant to the model's cluster. Cannot start nor delete ML tasks or models.
## ML node
To prevent your cluster from failing when running ML tasks, you configure a node with the `ml` node role. When configuring without the `data` node role, ML nodes will not store any shards and will calculate resource requirements at runtime. To use an ML node, create a node in your `opensearch.yml` file. Give your node a custom name and define the node role as `ml`:
```yml
node.name: ml-node
node.roles: [ ml ]
```

View File

@ -30,6 +30,7 @@ Cluster manager eligible | Elects one node among them as the cluster manager nod
Data | Stores and searches data. Performs all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of your cluster and need more disk space than any other node type. | As you add data nodes, keep them balanced between zones. For example, if you have three zones, add data nodes in multiples of three, one for each zone. We recommend using storage and RAM-heavy nodes.
Ingest | Pre-processes data before storing it in the cluster. Runs an ingest pipeline that transforms your data before adding it to an index. | If you plan to ingest a lot of data and run complex ingest pipelines, we recommend you use dedicated ingest nodes. You can also optionally offload your indexing from the data nodes so that your data nodes are used exclusively for searching and aggregating.
Coordinating | Delegates client requests to the shards on the data nodes, collects and aggregates the results into one final result, and sends this result back to the client. | A couple of dedicated coordinating-only nodes is appropriate to prevent bottlenecks for search-heavy workloads. We recommend using CPUs with as many cores as you can.
Dynamic | Delegates a specific node for custom work, such as machine learning (ML) tasks, preventing the consumption of resources from data nodes and therefore not affecting any OpenSearch functionality.
By default, each node is a cluster-manager-eligible, data, ingest, and coordinating node. Deciding on the number of nodes, assigning node types, and choosing the hardware for each node type depends on your use case. You must take into account factors like the amount of time you want to hold on to your data, the average size of your documents, your typical workload (indexing, searches, aggregations), your expected price-performance ratio, your risk tolerance, and so on.

View File

@ -46,9 +46,6 @@ include_unloaded_segments | Boolean | Whether to include information from segmen
## Response
```json
ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | cluster_manager | name
172.18.0.3 | 31 | 97 | 3 | 0.03 | 0.10 | 0.14 dimr | * | opensearch-node2
172.18.0.4 | 45 | 97 | 3 | 0.19 | 0.14 | 0.15 dimr | - | opensearch-node1
ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | node.roles | master | name
10.11.1.225 | 31 | 32 | 0 | 0.00 | 0.00 | di | data,ingest,ml | - | data-e5b89ad7
```