* Add agent framework documentation Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add hidden model and API updates Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Vale error Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Updated field names Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add updating credentials Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Added tools table Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add OpenSearch forum thread for OS Assistant Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add tech review for conv search Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Fix links Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add tools Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add links to tools Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * More info about tools Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Tool parameters Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Update cat-index-tool.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Parameter clarification Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Tech review feedback Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Typo fix Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * More tech review feedback: RAG tool Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Tech review feedback: memory APis Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Update _ml-commons-plugin/agents-tools/index.md Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/opensearch-assistant.md Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/ppl-tool.md Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Separated search and get APIs and add conversational flow agent Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * More parameters for PPL tool Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Added more parameters Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Tech review feedback: PPL tool Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Rename to automating configurations Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Editorial comments on the new text Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add parameter to PPl tool Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Changed link to configurations Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Rate limiter feedback and added warning Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> --------- Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Melissa Vagi <vagimeli@amazon.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
78 lines
2.1 KiB
Markdown
78 lines
2.1 KiB
Markdown
---
|
|
layout: default
|
|
title: Deploy model
|
|
parent: Model APIs
|
|
grand_parent: ML Commons APIs
|
|
nav_order: 20
|
|
---
|
|
|
|
# Deploy a model
|
|
|
|
The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`.
|
|
|
|
For information about user access for this API, see [Model access control considerations]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/model-apis/index/#model-access-control-considerations).
|
|
|
|
## Path and HTTP methods
|
|
|
|
```json
|
|
POST /_plugins/_ml/models/<model_id>/_deploy
|
|
```
|
|
|
|
#### Example request: Deploying to all available ML nodes
|
|
|
|
In this example request, OpenSearch deploys the model to any available OpenSearch ML node:
|
|
|
|
```json
|
|
POST /_plugins/_ml/models/WWQI44MBbzI2oUKAvNUt/_deploy
|
|
```
|
|
{% include copy-curl.html %}
|
|
|
|
#### Example request: Deploying to a specific node
|
|
|
|
If you want to reserve the memory of other ML nodes within your cluster, you can deploy your model to a specific node(s) by specifying the `node_ids` in the request body:
|
|
|
|
```json
|
|
POST /_plugins/_ml/models/WWQI44MBbzI2oUKAvNUt/_deploy
|
|
{
|
|
"node_ids": ["4PLK7KJWReyX0oWKnBA8nA"]
|
|
}
|
|
```
|
|
{% include copy-curl.html %}
|
|
|
|
#### Example response
|
|
|
|
```json
|
|
{
|
|
"task_id" : "hA8P44MBhyWuIwnfvTKP",
|
|
"status" : "DEPLOYING"
|
|
}
|
|
```
|
|
|
|
## Check the status of model deployment
|
|
|
|
To see the status of your model deployment and retrieve the model ID created for the new model version, pass the `task_id` as a path parameter to the Tasks API:
|
|
|
|
```json
|
|
GET /_plugins/_ml/tasks/hA8P44MBhyWuIwnfvTKP
|
|
```
|
|
{% include copy-curl.html %}
|
|
|
|
The response contains the model ID of the model version:
|
|
|
|
```json
|
|
{
|
|
"model_id": "Qr1YbogBYOqeeqR7sI9L",
|
|
"task_type": "DEPLOY_MODEL",
|
|
"function_name": "TEXT_EMBEDDING",
|
|
"state": "COMPLETED",
|
|
"worker_node": [
|
|
"N77RInqjTSq_UaLh1k0BUg"
|
|
],
|
|
"create_time": 1685478486057,
|
|
"last_update_time": 1685478491090,
|
|
"is_async": true
|
|
}
|
|
```
|
|
|
|
If a cluster or node is restarted, then you need to redeploy the model. To learn how to set up automatic redeployment, see [Enable auto redeploy]({{site.url}}{{site.baseurl}}/ml-commons-plugin/cluster-settings/#enable-auto-redeploy).
|
|
{: .tip} |