Add ML node settings and dynamic node description (#787)
* Add ML node instructions Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Change cluster settings from ml to dynamic Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Quick typo fix Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Adjust ML docs to remove dymanic mention. Adjust CAT example to be general Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add editorial feedback. Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix columns in example Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
This commit is contained in:
parent
0b01755beb
commit
6336b83b14
|
@ -23,3 +23,12 @@ There are two reserved user roles that can use of the ML commons plugin.
|
|||
|
||||
- `ml_full_access`: Full access to all ML features, including starting new ML tasks and reading or deleting models.
|
||||
- `ml_readonly_access`: Can only read ML tasks, trained models and statistics relevant to the model's cluster. Cannot start nor delete ML tasks or models.
|
||||
|
||||
## ML node
|
||||
|
||||
To prevent your cluster from failing when running ML tasks, you configure a node with the `ml` node role. When configuring without the `data` node role, ML nodes will not store any shards and will calculate resource requirements at runtime. To use an ML node, create a node in your `opensearch.yml` file. Give your node a custom name and define the node role as `ml`:
|
||||
|
||||
```yml
|
||||
node.name: ml-node
|
||||
node.roles: [ ml ]
|
||||
```
|
||||
|
|
|
@ -30,6 +30,7 @@ Cluster manager eligible | Elects one node among them as the cluster manager nod
|
|||
Data | Stores and searches data. Performs all data-related operations (indexing, searching, aggregating) on local shards. These are the worker nodes of your cluster and need more disk space than any other node type. | As you add data nodes, keep them balanced between zones. For example, if you have three zones, add data nodes in multiples of three, one for each zone. We recommend using storage and RAM-heavy nodes.
|
||||
Ingest | Pre-processes data before storing it in the cluster. Runs an ingest pipeline that transforms your data before adding it to an index. | If you plan to ingest a lot of data and run complex ingest pipelines, we recommend you use dedicated ingest nodes. You can also optionally offload your indexing from the data nodes so that your data nodes are used exclusively for searching and aggregating.
|
||||
Coordinating | Delegates client requests to the shards on the data nodes, collects and aggregates the results into one final result, and sends this result back to the client. | A couple of dedicated coordinating-only nodes is appropriate to prevent bottlenecks for search-heavy workloads. We recommend using CPUs with as many cores as you can.
|
||||
Dynamic | Delegates a specific node for custom work, such as machine learning (ML) tasks, preventing the consumption of resources from data nodes and therefore not affecting any OpenSearch functionality.
|
||||
|
||||
By default, each node is a cluster-manager-eligible, data, ingest, and coordinating node. Deciding on the number of nodes, assigning node types, and choosing the hardware for each node type depends on your use case. You must take into account factors like the amount of time you want to hold on to your data, the average size of your documents, your typical workload (indexing, searches, aggregations), your expected price-performance ratio, your risk tolerance, and so on.
|
||||
|
||||
|
|
|
@ -46,9 +46,6 @@ include_unloaded_segments | Boolean | Whether to include information from segmen
|
|||
## Response
|
||||
|
||||
```json
|
||||
|
||||
ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | cluster_manager | name
|
||||
|
||||
172.18.0.3 | 31 | 97 | 3 | 0.03 | 0.10 | 0.14 dimr | * | opensearch-node2
|
||||
172.18.0.4 | 45 | 97 | 3 | 0.19 | 0.14 | 0.15 dimr | - | opensearch-node1
|
||||
ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | node.roles | master | name
|
||||
10.11.1.225 | 31 | 32 | 0 | 0.00 | 0.00 | di | data,ingest,ml | - | data-e5b89ad7
|
||||
```
|
||||
|
|
Loading…
Reference in New Issue