* add description and saving disk Signed-off-by: xinyual <xinyual@amazon.com> * update Signed-off-by: xinyual <xinyual@amazon.com> * update Signed-off-by: xinyual <xinyual@amazon.com> * Update _ml-commons-plugin/pretrained-models.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com> * Update _query-dsl/specialized/neural-sparse.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com> * Update _search-plugins/neural-sparse-search.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com> * Update _search-plugins/neural-sparse-search.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com> * Update _search-plugins/neural-sparse-search.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com> * Moved recommended model choice options outside the table Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Add a link for more info Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: xinyual <xinyual@amazon.com> Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
2.1 KiB
layout | title | parent | grand_parent | nav_order |
---|---|---|---|---|
default | Neural sparse | Specialized queries | Query DSL | 55 |
Neural sparse query
Introduced 2.11 {: .label .label-purple }
Use the neural_sparse
query for vector field search in sparse neural search.
Request fields
Include the following request fields in the neural_sparse
query:
"neural_sparse": {
"<vector_field>": {
"query_text": "<query_text>",
"model_id": "<model_id>",
"max_token_score": "<max_token_score>"
}
}
The top-level vector_field
specifies the vector field against which to run a search query. The following table lists the other neural_sparse
query fields.
Field | Data type | Required/Optional | Description
:--- | :--- | :---
query_text
| String | Required | The query text from which to generate vector embeddings.
model_id
| String | Required | The ID of the sparse encoding model or tokenizer model that will be used to generate vector embeddings from the query text. The model must be deployed in OpenSearch before it can be used in sparse neural search. For more information, see Using custom models within OpenSearch and Semantic search.
max_token_score
| Float | Optional | The theoretical upper bound of the score for all tokens in the vocabulary (required for performance optimization). For OpenSearch-provided pretrained sparse embedding models, we recommend setting max_token_score
to 2 for amazon/neural-sparse/opensearch-neural-sparse-encoding-doc-v1
and to 3.5 for amazon/neural-sparse/opensearch-neural-sparse-encoding-v1
.
Example request
GET my-nlp-index/_search
{
"query": {
"neural_sparse": {
"passage_embedding": {
"query_text": "Hi world",
"model_id": "aP2Q8ooBpBj3wT4HVS8a",
"max_token_score": 2
}
}
}
}
{% include copy-curl.html %}