iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update model access control documentation (#4552)

Browse Source 
* Update model access control documentation

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Remove duplicated info

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Reword registering model

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Reword registering model

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Clarification for model size

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Update _ml-commons-plugin/api.md

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/model-access-control.md

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/api.md

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Add tech review comment

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Update _ml-commons-plugin/api.md

Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Reworded for clarity

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Heather Halter <HDHALTER@AMAZON.COM>
Co-authored-by: Nathan Bower <nbower@amazon.com>
This commit is contained in:
kolchfa-aws 2023-07-18 10:55:58 -04:00 committed by GitHub
parent d90ec83738
commit 06665364fd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 48 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
_ml-commons-plugin/api.md
View File

@ -156,13 +156,22 @@ The response contains the following model information:
## Registering a model
Before you register a model, you must [register a model group]({{site.url}}{{site.baseurl}}/ml-commons-plugin/model-access-control#registering-a-model-group) for the model.
{: .important}
All versions of a particular model are held in a model group. You can either [register a model group]({{site.url}}{{site.baseurl}}/ml-commons-plugin/model-access-control#registering-a-model-group) before registering a model to the group or register a first version of a model, thereby creating the group. Each model group name in the cluster must be globally unique.
All versions of a particular model are held in a model group. After you register a model group, you can register a model to the model group. ML Commons splits the model into smaller chunks and saves those chunks in the model's index.
If you are registering the first version of a model without first registering the model group, a new model group is created automatically with the following name and access level:
- Name: The new model group will have the same name as the model. Because the model group name must be unique, ensure that your model name does not have the same name as any model groups in the cluster.
- Access level: The access level for the new model group is determined using the `access_mode`, `backend_roles`, and `add_all_backend_roles` parameters that you pass in the request. If you provide none of the three parameters, the new model group will be `private` if model access control is enabled on your cluster and `public` if model access control is disabled. The newly registered model is the first model version assigned to that model group.
Once a model group is created, provide its `model_group_id` to register a new model version to the model group. In this case, the model name does not need to be unique.
If you're using [pretrained models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/pretrained-models#supported-pretrained-models) provided by OpenSearch, we recommend that you first register a model group with a unique name for these models. Then register the pretrained models as versions to that model group. This ensures that every model group has a globally unique model group name.
{: .tip}
For information about user access for this API, see [Model access control considerations](#model-access-control-considerations).
If the model is more than 10 MB in size, ML Commons splits it into smaller chunks and saves those chunks in the model's index.
### Path and HTTP methods
```json
@ -179,7 +188,7 @@ Field | Data type | Description
`name`| String | The model's name. |
`version` | Integer | The model's version number. |
`model_format` | String | The portable format of the model file. Currently only supports `TORCH_SCRIPT`. |
`model_group_id` | String | The model group ID for the model.
`model_group_id` | String | The model group ID of the model group to register this model to.
`model_content_hash_value` | String | The model content hash generated using the SHA-256 hashing algorithm.
`model_config` | JSON object | The model's configuration, including the `model_type`, `embedding_dimension`, and `framework_type`. `all_config` is an optional JSON string that contains all model configurations. |
`url` | String | The URL that contains the model. |
@ -477,6 +486,9 @@ POST /_plugins/_ml/models/_search
Deletes a model based on the `model_id`.
When you delete the last model version in a model group, that model group is automatically deleted from the index.
{: .important}
For information about user access for this API, see [Model access control considerations](#model-access-control-considerations).
### Path and HTTP methods

46
_ml-commons-plugin/model-access-control.md
View File

@ -30,7 +30,7 @@ Before using model access control, you must satisfy the following prerequisites:
1. Enable the Security plugin on your cluster. For more information, see [Security in OpenSearch]({{site.url}}{{site.baseurl}}/security/).
2. For `restricted` model groups, ensure that an admin has [assigned backend roles to users](#assigning-backend-roles-to-users).
3. [Enable model access control](#enabling-model-access-control) on your cluster. You can enable model access control dynamically by setting `plugins.ml_commons.model_access_control_enabled` to `true`.
3. [Enable model access control](#enabling-model-access-control) on your cluster.
If any of the prerequisites are not met, all models in the cluster are `public` and can be accessed by any user who has access to the cluster.
{: .note}
@ -102,7 +102,10 @@ PUT _cluster/settings
## Registering a model group
Use the `_register` endpoint to register a model group. You can register a model group with a `public`, `private`, or `restricted` access mode.
To register a model group, send a `POST` request to the `_register` endpoint. You can register a model group in `public`, `private`, or `restricted` access mode.
Each model group name in the cluster must be globally unique.
{: .important}
### Path and HTTP method
@ -118,8 +121,8 @@ Field |Data type | Description
:--- | :--- | :---
`name` | String | The model group name. Required.
`description` | String | The model group description. Optional.
`model_access_mode` | String | The access mode for this model. Valid values are `public`, `private`, and `restricted`. When this parameter is set to `restricted`, you must specify either `backend_roles` or `add_all_backend_roles`, but not both. Optional. Default is `restricted`.
`backend_roles` | Array | A list of the model owner's backend roles to add to the model. Can be specified only if the `model_access_mode` is `restricted`. Cannot be specified at the same time as `add_all_backend_roles`. Optional.
`access_mode` | String | The access mode for this model. Valid values are `public`, `private`, and `restricted`. When this parameter is set to `restricted`, you must specify either `backend_roles` or `add_all_backend_roles`, but not both. Optional. If you specify none of the security parameters (`access_mode`, `backend_roles`, and `add_all_backend_roles`), the default `access_mode` is `private`.
`backend_roles` | Array | A list of the model owner's backend roles to add to the model. Can be specified only if `access_mode` is `restricted`. Cannot be specified at the same time as `add_all_backend_roles`. Optional.
`add_all_backend_roles` | Boolean | If `true`, all backend roles of the model owner are added to the model group. Default is `false`. Cannot be specified at the same time as `backend_roles`. Admin users cannot set this parameter to `true`. Optional.
#### Example request
@ -129,7 +132,7 @@ POST /_plugins/_ml/model_groups/_register
{
"name": "test_model_group_public",
"description": "This is a public model group",
"model_access_mode": "public"
"access_mode": "public"
}
```
{% include copy-curl.html %}
@ -161,7 +164,7 @@ POST /_plugins/_ml/model_groups/_register
{
"name": "test_model_group_public",
"description": "This is a public model group",
"model_access_mode": "public"
"access_mode": "public"
}
```
{% include copy-curl.html %}
@ -188,7 +191,7 @@ POST /_plugins/_ml/model_groups/_register
{
"name": "model_group_test",
"description": "This is an example description",
"model_access_mode": "restricted",
"access_mode": "restricted",
"backend_roles" : ["IT"]
}
```
@ -203,7 +206,7 @@ POST /_plugins/_ml/model_groups/_register
{
"name": "model_group_test",
"description": "This is an example description",
"model_access_mode": "restricted",
"access_mode": "restricted",
"add_all_backend_roles": "true"
}
```
@ -218,7 +221,18 @@ POST /_plugins/_ml/model_groups/_register
{
"name": "model_group_test",
"description": "This is an example description",
"model_access_mode": "private"
"access_mode": "private"
}
```
{% include copy-curl.html %}
If you don't specify any of the `access_mode`, `backend_roles`, or `add_all_backend_roles`, the model will have a `private` access mode:
```json
POST /_plugins/_ml/model_groups/_register
{
"name": "model_group_test",
"description": "This is an example description"
}
```
{% include copy-curl.html %}
@ -229,17 +243,18 @@ If model access control is disabled on your cluster (one of the [prerequisites](
## Updating a model group
To update a model group, send a request to the `<model_group_id>_update` endpoint.
To update a model group, send a `PUT` request to the `model_groups` endpoint and provide the ID of the model group you want to update.
When updating a model group, the following restrictions apply:
- The model owner or an admin user can update all fields. Any user who shares one or more backend roles with the model group can update the `name` and `description` fields only.
- When updating the `model_access_mode` to `restricted`, you must specify one but not both `backend_roles` or `add_all_backend_roles`.
- When updating the `access_mode` to `restricted`, you must specify either `backend_roles` or `add_all_backend_roles` but not both.
- When updating the `name`, ensure the name is globally unique in the cluster.
### Path and HTTP method
```json
PUT /_plugins/_ml/model_groups/<model_group_id>_update
PUT /_plugins/_ml/model_groups/<model_group_id>
```
### Request fields
@ -249,10 +264,10 @@ Refer to [Request fields](#request-fields-1) for request field descriptions.
#### Example request
```json
PUT /_plugins/_ml/model_groups/<model_group_id>_update
PUT /_plugins/_ml/model_groups/<model_group_id>
{
"name": "model_group_test",
"description": "This is an example description",
"description": "This is the updated description",
"add_all_backend_roles": true
}
```
@ -567,6 +582,9 @@ If model access control is disabled on your cluster, users with the `delete mode
Admin users can delete any model group.
{: .note}
When you delete the last model version in a model group, that model group is automatically deleted from the index.
{: .important}
#### Example request
```json