iSharkFly-Docs
/
opensearch-docs-cn
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update connector documentation (#5723) 

* Connector rewrites

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add a table of supported connectors

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change next steps

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
This commit is contained in:
kolchfa-aws 2023-12-04 07:56:12 -05:00 committed by GitHub
parent 47204cf4e8
commit bee7322c93
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 72 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
_ml-commons-plugin/remote-models/blueprints.md
View File

@ -3,7 +3,7 @@ layout: default
title: Connector blueprints
has_children: false
nav_order: 65
parent: Connecting to remote models
parent: Connecting to externally hosted models
grand_parent: Integrating ML models
redirect_from:
- ml-commons-plugin/extensibility/blueprints/
@ -13,12 +13,11 @@ redirect_from:
**Introduced 2.9**
{: .label .label-purple }
All connectors consist of a JSON blueprint created by machine learning (ML) developers. The blueprint allows administrators and data scientists to make connections between OpenSearch and an AI service or model-serving technology.
Every [connector]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/connectors/) is specified by a _connector blueprint_. The blueprint defines all the parameters you need to provide when creating a connector.
The following example shows a blueprint of an Amazon SageMaker connector:
For example, the following blueprint is a specification for an Amazon SageMaker connector:
```json
POST /_plugins/_ml/connectors/_create
{
"name": "<YOUR CONNECTOR NAME>",
"description": "<YOUR CONNECTOR DESCRIPTION>",
@ -46,15 +45,17 @@ POST /_plugins/_ml/connectors/_create
]
}
```
{% include copy-curl.html %}
{% include copy-curl.html %}
## Example blueprints
## OpenSearch-provided connector blueprints
You can find blueprints for each connector in the [ML Commons repository](https://github.com/opensearch-project/ml-commons/tree/2.x/docs/remote_inference_blueprints).
OpenSearch provides connector blueprints for several machine learning (ML) platforms and models. For a list of all connector blueprints provided by OpenSearch, see [Supported connectors]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/connectors/#supported-connectors).
## Configuration options
As an ML developer, you can build connector blueprints for other platforms. Using those blueprints, administrators and data scientists can create connectors for models hosted on those platforms.
The following configuration options are **required** in order to build a connector blueprint. These settings can be used for both standalone and internal connectors.
## Configuration parameters
The following configuration parameters are **required** in order to build a connector blueprint.
| Field | Data type | Description |
| :--- | :--- | :--- |
@ -85,8 +86,8 @@ The `action` parameter supports the following options.
Call the built-in pre- and post-processing functions instead of writing a custom Painless script when connecting to the following text embedding models or your own text embedding models deployed on a remote server (for example, Amazon SageMaker):
- [OpenAI remote models](https://platform.openai.com/docs/api-reference/embeddings)
- [Cohere remote models](https://docs.cohere.com/reference/embed)
- [OpenAI models](https://platform.openai.com/docs/api-reference/embeddings)
- [Cohere models](https://docs.cohere.com/reference/embed)
OpenSearch provides the following pre- and post-processing functions:
@ -96,7 +97,7 @@ OpenSearch provides the following pre- and post-processing functions:
### Default pre- and post-processing functions
When you perform vector search using [neural search]({{site.url}}{{site.baseurl}}/search-plugins/neural-search/), the neural search request is routed first to ML Commons and then to the model. If the model is one of the [pretrained models provided by OpenSearch]({{site.url}}{{site.baseurl}}/ml-commons-plugin/pretrained-models/), it can parse the ML Commons request and return the response in the format that ML Commons expects. However, for a remote model, the expected format may be different from the ML Commons format. The default pre- and post-processing functions translate between the format that the model expects and the format that neural search expects.
When you perform vector search using [neural search]({{site.url}}{{site.baseurl}}/search-plugins/neural-search/), the neural search request is routed first to ML Commons and then to the model. If the model is one of the [pretrained models provided by OpenSearch]({{site.url}}{{site.baseurl}}/ml-commons-plugin/pretrained-models/), it can parse the ML Commons request and return the response in the format that ML Commons expects. However, for a model hosted on an external platform, the expected format may be different from the ML Commons format. The default pre- and post-processing functions translate between the format that the model expects and the format that neural search expects.
#### Example request
@ -216,6 +217,7 @@ POST /_plugins/_ml/connectors/_create
```
{% include copy-curl.html %}
## Next step
## Next steps
For examples of creating various connectors, see [Connectors]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/connectors/).
- To learn more about connecting to external models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index/).
- For connector examples, see [Connectors]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/connectors/).

75
_ml-commons-plugin/remote-models/connectors.md
View File

@ -4,7 +4,7 @@ title: Connectors
has_children: false
has_toc: false
nav_order: 61
parent: Connecting to remote models
parent: Connecting to externally hosted models
grand_parent: Integrating ML models
redirect_from:
- ml-commons-plugin/extensibility/connectors/
@ -14,31 +14,45 @@ redirect_from:
**Introduced 2.9**
{: .label .label-purple }
Connectors facilitate access to remote models hosted on third-party platforms.
Connectors facilitate access to models hosted on third-party machine learning (ML) platforms.
You can provision connectors in two ways:
1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused and shared by multiple remote models but requires access to both the model and connector in OpenSearch and the third-party platform, such as OpenAI or Amazon SageMaker, that the connector is accessing. Standalone connectors are saved in a connector index.
2. [Create a connector for a specific remote model](#creating-a-connector-for-a-specific-model): Alternatively, you can create a connector that can only be used with the remote model for which it was created. To access such a connector, you only need access to the model itself because the connection is established inside the model. These connectors are saved in the model index.
## Supported connectors
As of OpenSearch 2.9, connectors have been tested for the following ML services, though it is possible to create connectors for other platforms not listed here:
OpenSearch provides connectors for several platforms, for example:
- [Amazon SageMaker](https://aws.amazon.com/sagemaker/) allows you to host and manage the lifecycle of text embedding models, powering semantic search queries in OpenSearch. When connected, Amazon SageMaker hosts your models and OpenSearch is used to query inferences. This benefits Amazon SageMaker users who value its functionality, such as model monitoring, serverless hosting, and workflow automation for continuous training and deployment.
- [OpenAI ChatGPT](https://openai.com/blog/chatgpt) enables you to invoke an OpenAI chat model from inside an OpenSearch cluster.
- [Cohere](https://cohere.com/) allows you to use data from OpenSearch to power the Cohere large language models.
- The [Bedrock Titan Embeddings](https://aws.amazon.com/bedrock/titan/) model can drive semantic search and retrieval-augmented generation in OpenSearch.
- [Amazon Bedrock](https://aws.amazon.com/bedrock/) supports models like [Bedrock Titan Embeddings](https://aws.amazon.com/bedrock/titan/), which can drive semantic search and retrieval-augmented generation in OpenSearch.
All connectors consist of a JSON blueprint created by machine learning (ML) developers. The blueprint allows administrators and data scientists to make connections between OpenSearch and an AI service or model-serving technology.
## Connector blueprints
You can find blueprints for each connector in the [ML Commons repository](https://github.com/opensearch-project/ml-commons/tree/2.x/docs/remote_inference_blueprints).
A _connector blueprint_ defines the set of parameters (the request body) you need to provide when sending an API request to create a specific connector. Connector blueprints may differ based on the platform and the model that you are accessing.
For more information about blueprint parameters, see [Connector blueprints]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/).
OpenSearch provides connector blueprints for several ML platforms and models. For a full list of connector blueprints provided by OpenSearch, see [Supported connectors](#supported-connectors).
Admins are only required to enter their `credential` settings, such as `"openAI_key"`, for the service they are connecting to. All other parameters are defined within the [blueprint]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/).
{: .note}
As an ML developer, you can also create connector blueprints for other platforms and models. Data scientists and administrators can then use the blueprint to create connectors. They are only required to enter their `credential` settings, such as `openAI_key`, for the service to which they are connecting. For information about creating connector blueprints, including descriptions of all parameters, see [Connector blueprints]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/).
## Supported connectors
The following table lists all connector blueprints provided by OpenSearch. Follow the links to each connector blueprint for an example request that you can use to create the connector, including all parameters, and an example Predict API request.
Platform | Model | Connector blueprint
:--- | :--- | :---
[Amazon Bedrock](https://aws.amazon.com/bedrock/) | [A21 Labs Jurassic-2 Mid](https://aws.amazon.com/bedrock/jurassic/) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/bedrock_connector_ai21labs_jurassic_blueprint.md)
[Amazon Bedrock](https://aws.amazon.com/bedrock/) | [Anthropic Claude v2](https://aws.amazon.com/bedrock/claude/) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/bedrock_connector_anthropic_claude_blueprint.md)
[Amazon Bedrock](https://aws.amazon.com/bedrock/) | [Titan Text Embeddings](https://aws.amazon.com/bedrock/titan/) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/bedrock_connector_titan_embedding_blueprint.md)
[Amazon SageMaker](https://aws.amazon.com/sagemaker/) | Text embedding models | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/sagemaker_connector_blueprint.md)
[Cohere](https://cohere.com/) | Text embedding models (for example, `embed-english-v2.0`) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/cohere_connector_embedding_blueprint.md)
[OpenAI](https://openai.com/) | Chat models (for example, `gpt-3.5-turbo`) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/open_ai_connector_chat_blueprint.md)
[OpenAI](https://openai.com/) | Completion models (for example, `text-davinci-003`) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/open_ai_connector_completion_blueprint.md)
[OpenAI](https://openai.com/) | Text embedding models (for example, `text-embedding-ada-002`) | [Blueprint](https://github.com/opensearch-project/ml-commons/blob/2.x/docs/remote_inference_blueprints/openai_connector_embedding_blueprint.md)
## Creating a connector
You can provision connectors in two ways:
1. [Create a standalone connector](#creating-a-standalone-connector): A standalone connector can be reused and shared by multiple models but requires access to both the model and connector in OpenSearch and the third-party platform, such as OpenAI or Amazon SageMaker, that the connector is accessing. Standalone connectors are saved in a connector index.
2. [Create a connector for a specific externally hosted model](#creating-a-connector-for-a-specific-model): Alternatively, you can create a connector that can only be used with the model for which it was created. To access such a connector, you only need access to the model itself because the connection is established inside the model. These connectors are saved in the model index.
## Creating a standalone connector
@ -114,9 +128,13 @@ POST /_plugins/_ml/models/_register
```
{% include copy-curl.html %}
## OpenAI chat connector
## Connector examples
The following example shows how to create a standalone OpenAI chat connector:
The following sections contain examples of connectors for popular ML platforms. For a full list of supported connectors, see [Supported connectors](#supported-connectors).
### OpenAI chat connector
You can use the following example request to create a standalone OpenAI chat connector:
```json
POST /_plugins/_ml/connectors/_create
@ -145,13 +163,11 @@ POST /_plugins/_ml/connectors/_create
]
}
```
{% include copy-curl.html %}
After creating the connector, you can retrieve the `task_id` and `connector_id` to register and deploy the model and then use the Predict API, similarly to a standalone connector.
### Amazon SageMaker connector
## Amazon SageMaker connector
The following example shows how to create a standalone Amazon SageMaker connector:
You can use the following example request to create a standalone Amazon SageMaker connector:
```json
POST /_plugins/_ml/connectors/_create
@ -182,6 +198,7 @@ POST /_plugins/_ml/connectors/_create
]
}
```
{% include copy-curl.html %}
The `credential` parameter contains the following options reserved for `aws_sigv4` authentication:
@ -194,9 +211,9 @@ The `parameters` section requires the following options when using `aws_sigv4` a
- `region`: The AWS Region in which the AWS instance is located.
- `service_name`: The name of the AWS service for the connector.
## Cohere connector
### Cohere connector
The following example request creates a standalone Cohere connector:
You can use the following example request to create a standalone Cohere connector:
```json
POST /_plugins/_ml/connectors/_create
@ -229,9 +246,9 @@ POST /_plugins/_ml/connectors/_create
```
{% include copy-curl.html %}
## Amazon Bedrock connector
### Amazon Bedrock connector
The following example request creates a standalone Amazon Bedrock connector:
You can use the following example request to create a standalone Amazon Bedrock connector:
```json
POST /_plugins/_ml/connectors/_create
@ -269,5 +286,5 @@ POST /_plugins/_ml/connectors/_create
## Next steps
- To learn more about using models in OpenSearch, see [Using ML models within OpenSearch]({{site.url}}{{site.baseurl}}/ml-commons-plugin/using-ml-models/).
- To learn more about connecting to external models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index/).
- To learn more about model access control and model groups, see [Model access control]({{site.url}}{{site.baseurl}}/ml-commons-plugin/model-access-control/).

20
_ml-commons-plugin/remote-models/index.md
View File

@ -1,6 +1,6 @@
---
layout: default
title: Connecting to remote models
title: Connecting to externally hosted models
parent: Integrating ML models
has_children: true
has_toc: false
@ -9,13 +9,13 @@ redirect_from:
- ml-commons-plugin/extensibility/index/
---
# Connecting to remote models
# Connecting to externally hosted models
**Introduced 2.9**
{: .label .label-purple }
Machine learning (ML) remote models enable ML developers to create integrations with other ML services, such as Amazon SageMaker or OpenAI. These integrations allow system administrators and data scientists to run ML workloads outside of their OpenSearch cluster.
Integrations with machine learning (ML) models hosted on third-party platforms allow system administrators and data scientists to run ML workloads outside of their OpenSearch cluster. Connecting to externally hosted models enables ML developers to create integrations with other ML services, such as Amazon SageMaker or OpenAI.
To get started with ML remote models, choose from the following options:
To integrate a model hosted on a third-party platform, choose from the following options:
- If you're an ML developer wanting to create integrations with your specific ML services, see [Connector blueprints]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/).
- If you're a system administrator or data scientist wanting to create a connection to an ML service, see [Connectors]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/connectors/).
@ -82,7 +82,7 @@ When access control is enabled, you can install the [Security plugin]({{site.url
### Node settings
Remote models based on external connectors consume fewer resources. Therefore, you can deploy any model from a standalone connector using data nodes. To make sure that your standalone connection uses data nodes, set `plugins.ml_commons.only_run_on_ml_node` to `false`:
Externally hosted models that are deployed using connectors consume fewer resources. Therefore, you can deploy such models model on data nodes. To make sure that your model connection uses data nodes, set `plugins.ml_commons.only_run_on_ml_node` to `false`:
```json
PUT /_cluster/settings
@ -108,7 +108,7 @@ To register a model group, send the following request:
POST /_plugins/_ml/model_groups/_register
{
"name": "remote_model_group",
"description": "A model group for remote models"
"description": "A model group for external models"
}
```
{% include copy-curl.html %}
@ -167,9 +167,9 @@ The response contains the connector ID for the newly created connector:
}
```
## Step 3: Register a remote model
## Step 3: Register an externally hosted model
To register a remote model to the model group created in step 1, provide the model group ID from step 1 and the connector ID from step 2 in the following request:
To register an externally hosted model to the model group created in step 1, provide the model group ID from step 1 and the connector ID from step 2 in the following request. You must specify the `function_name` as `remote`:
```json
POST /_plugins/_ml/models/_register
@ -218,7 +218,7 @@ When the operation is complete, the state changes to `COMPLETED`:
Take note of the returned `model_id` because you’ll need it to deploy the model.
## Step 4: Deploy the remote model
## Step 4: Deploy the model
To deploy the registered model, provide its model ID from step 3 in the following request:
@ -260,7 +260,7 @@ When the operation is complete, the state changes to `COMPLETED`:
}
```
## Step 5 (Optional): Test the remote model
## Step 5 (Optional): Test the model
Use the [Predict API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/train-predict/predict/) to test the model: