Alice Williams 253ae34bd8
Search with k-NN filters (#1814)
* new file for knn filter searches

Signed-off-by: alicejw <alicejw@amazon.com>

* for knn filter queries

Signed-off-by: alicejw <alicejw@amazon.com>

* more details and include graphic

Signed-off-by: alicejw <alicejw@amazon.com>

* add graph of filtered doc set

Signed-off-by: alicejw <alicejw@amazon.com>

* add release label

Signed-off-by: alicejw <alicejw@amazon.com>

* filters are defined by Query DSL

Signed-off-by: alicejw <alicejw@amazon.com>

* more details about how the algorithm works and how to specify lucene as the search engine

Signed-off-by: alicejw <alicejw@amazon.com>

* more refining sentences

Signed-off-by: alicejw <alicejw@amazon.com>

* for response samples

Signed-off-by: alicejw <alicejw@amazon.com>

* reorg heading levels

Signed-off-by: alicejw <alicejw@amazon.com>

* more rewrites for clarity

Signed-off-by: alicejw <alicejw@amazon.com>

* to add the complex filter query

Signed-off-by: alicejw <alicejw@amazon.com>

* update response for complex query

Signed-off-by: alicejw <alicejw@amazon.com>

* for typo

Signed-off-by: alicejw <alicejw@amazon.com>

* for rewrites to overview

Signed-off-by: alicejw <alicejw@amazon.com>

* to add better request/response for the complex filter example

Signed-off-by: alicejw <alicejw@amazon.com>

* for eng review update

Signed-off-by: alicejw <alicejw@amazon.com>

* format fix for example

Signed-off-by: alicejw <alicejw@amazon.com>

* for filter selectiveness use case section

Signed-off-by: alicejw <alicejw@amazon.com>

* for new workflow diagram and description

Signed-off-by: alicejw <alicejw@amazon.com>

* update section headings

Signed-off-by: alicejw <alicejw@amazon.com>

* add image for algorithm workflow diagram

Signed-off-by: alicejw <alicejw@amazon.com>

* reorg sections to make more concise

Signed-off-by: alicejw <alicejw@amazon.com>

* explain selectiveness percentage

Signed-off-by: alicejw <alicejw@amazon.com>

* more rewrites to complex query description

Signed-off-by: alicejw <alicejw@amazon.com>

* define complex query

Signed-off-by: alicejw <alicejw@amazon.com>

* more rewrites

Signed-off-by: alicejw <alicejw@amazon.com>

* for tech review feedback and add new information

Signed-off-by: alicejw <alicejw@amazon.com>

* to blend new Boolean query example into filter approaches section

Signed-off-by: alicejw <alicejw@amazon.com>

* for complex query description clarity

Signed-off-by: alicejw <alicejw@amazon.com>

* more rewrites

Signed-off-by: alicejw <alicejw@amazon.com>

* typo

Signed-off-by: alicejw <alicejw@amazon.com>

* eng review updates

Signed-off-by: alicejw <alicejw@amazon.com>

* nit for grammar

Signed-off-by: alicejw <alicejw@amazon.com>

* to fix incorrect descriptions of restrictive filters

Signed-off-by: alicejw <alicejw@amazon.com>

* to fix incorrect descriptions of restrictive filters

Signed-off-by: alicejw <alicejw@amazon.com>

* for doc review feedback updates

Signed-off-by: alicejw <alicejw@amazon.com>

* minor grammar change

Signed-off-by: alicejw <alicejw@amazon.com>

* removed figure and table titles, per AWS Style Guide

Signed-off-by: alicejw <alicejw@amazon.com>

* remove table title per style guide

Signed-off-by: alicejw <alicejw@amazon.com>

* update nav orders for all pages to give space for new topics in multiples of 5, and add links to other knn topics where appropriate

Signed-off-by: alicejw <alicejw@amazon.com>

* small rewrite

Signed-off-by: alicejw <alicejw@amazon.com>

* for second doc review comments

Signed-off-by: alicejw <alicejw@amazon.com>

* Update _search-plugins/knn/filter-search-knn.md

Co-authored-by: Nate Bower <nbower@amazon.com>

* Update _search-plugins/knn/filter-search-knn.md

Co-authored-by: Nate Bower <nbower@amazon.com>

* for editorial review updates

Signed-off-by: alicejw <alicejw@amazon.com>

* for editorial review updates

Signed-off-by: alicejw <alicejw@amazon.com>

* fix cross-ref link

Signed-off-by: alicejw <alicejw@amazon.com>

* fix undone commit suggestions

Signed-off-by: alicejw <alicejw@amazon.com>

Signed-off-by: alicejw <alicejw@amazon.com>
Co-authored-by: Nate Bower <nbower@amazon.com>
2022-11-11 11:23:45 -08:00

26 lines
1.8 KiB
Markdown

---
layout: default
title: Settings
parent: k-NN
nav_order: 40
---
# k-NN settings
The k-NN plugin adds several new cluster settings.
## Cluster settings
Setting | Default | Description
:--- | :--- | :---
`knn.algo_param.index_thread_qty` | 1 | The number of threads used for native library index creation. Keeping this value low reduces the CPU impact of the k-NN plugin, but also reduces indexing performance.
`knn.cache.item.expiry.enabled` | false | Whether to remove native library indices that have not been accessed for a certain duration from memory.
`knn.cache.item.expiry.minutes` | 3h | If enabled, the idle time before removing a native library index from memory.
`knn.circuit_breaker.unset.percentage` | 75% | The native memory usage threshold for the circuit breaker. Memory usage must be below this percentage of `knn.memory.circuit_breaker.limit` for `knn.circuit_breaker.triggered` to remain false.
`knn.circuit_breaker.triggered` | false | True when memory usage exceeds the `knn.circuit_breaker.unset.percentage` value.
`knn.memory.circuit_breaker.limit` | 50% | The native memory limit for native library indices. At the default value, if a machine has 100 GB of memory and the JVM uses 32 GB, the k-NN plugin uses 50% of the remaining 68 GB (34 GB). If memory usage exceeds this value, k-NN removes the least recently used native library indices.
`knn.memory.circuit_breaker.enabled` | true | Whether to enable the k-NN memory circuit breaker.
`knn.plugin.enabled`| true | Enables or disables the k-NN plugin.
`knn.model.index.number_of_shards`| 1 | Number of shards to use for the model system index, the OpenSearch index that stores the models used for Approximate k-NN Search.
`knn.model.index.number_of_replicas`| 1 | Number of replica shards to use for the model system index. Generally, in a multi-node cluster, this should be at least 1 to increase stability.