From bf886cba59917d5a9a72e6b50d2cea445710e4f0 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Fri, 22 Sep 2023 17:30:34 -0400 Subject: [PATCH] Add a note about zstd incompatibility with k-NN and security analytics (#5014) * Add a note about zstd incompatibility with k-NN and security analytics plugins Signed-off-by: Fanit Kolchina * Capitalization Signed-off-by: Fanit Kolchina * Update _im-plugin/index-codecs.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _im-plugin/index-codecs.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower --- _im-plugin/index-codecs.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/_im-plugin/index-codecs.md b/_im-plugin/index-codecs.md index 8c836456..f880e141 100644 --- a/_im-plugin/index-codecs.md +++ b/_im-plugin/index-codecs.md @@ -24,6 +24,9 @@ It may be challenging to change the codec setting of an existing index (see [Cha * `zstd` (OpenSearch 2.9 and later) -- This codec provides significant compression comparable to the `best_compression` codec with reasonable CPU usage and improved indexing and search performance compared to the `default` codec. * `zstd_no_dict` (OpenSearch 2.9 and later) -- This codec is similar to `zstd` but excludes the dictionary compression feature. It provides faster indexing and search operations compared to `zstd` at the expense of a slightly larger index size. +As of OpenSearch 2.10, the `zstd` and `zstd_no_dict` compression codecs cannot be used for [k-NN]({{site.url}}{{site.baseurl}}/search-plugins/knn/index/) or [Security Analytics]({{site.url}}{{site.baseurl}}/security-analytics/index/) indexes. +{: .warning} + For the `zstd` and `zstd_no_dict` codecs, you can optionally specify a compression level in the `index.codec.compression_level` setting. This setting takes integers in the [1, 6] range. A higher compression level results in a higher compression ratio (smaller storage size) with a tradeoff in speed (slower compression and decompression speeds lead to greater indexing and search latencies). When an index segment is created, it uses the current index codec for compression. If you update the index codec, any segment created after the update will use the new compression algorithm. For specific operation considerations, see [Index codec considerations for index operations](#index-codec-considerations-for-index-operations).