Commit Graph

7 Commits

Author SHA1 Message Date
zhichao-aws 5f486abf83
Deprecate max_token_score in neural sparse search (#6554)
* deprecated max_token_score

Signed-off-by: zhichao-aws <zhichaog@amazon.com>

* Update _query-dsl/specialized/neural-sparse.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

---------

Signed-off-by: zhichao-aws <zhichaog@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
2024-03-01 09:30:03 -05:00
kolchfa-aws f524e6891f
Change terminology from sparse to neural sparse (#5714)
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
2023-11-30 10:43:12 -05:00
kolchfa-aws f999e0a8a8
Add an overview of search methods and pages for each search method (#5636)
* Restructuring TOC

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Resolve merge conflicts

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* More foundational rewrites of ML

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* TOC restructure

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Rename and rewrite search pages and add keyword search

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Small wording change

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Small wording change

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Updated response

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Small rewording

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Move neural search to top of vector search list

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change terminology

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Reorganize search methods list

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Rename links

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* More link renames

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Implemented editorial comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-11-29 15:28:20 -05:00
kolchfa-aws 826e6771ed
Refactor ML section - local and remote models (#5609)
* Refactor ML section - local and remote models

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added command to calculate checksum

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add ONNX format to register API

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add sparse encoding predict example

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add API section

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Refactor the API section

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Typo

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Implemented Vale comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add get connector API

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Reword heading

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Addressed tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
2023-11-17 15:59:27 -05:00
xinyual 242c35db59
add pretrained model description and saving disk description (#5383)
* add description and saving disk

Signed-off-by: xinyual <xinyual@amazon.com>

* update

Signed-off-by: xinyual <xinyual@amazon.com>

* update

Signed-off-by: xinyual <xinyual@amazon.com>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _query-dsl/specialized/neural-sparse.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _search-plugins/neural-sparse-search.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Moved recommended model choice options outside the table

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add a link for more info

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

---------

Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-25 11:59:49 -04:00
xinyual abe43ce3a5
Add pretrained model information for neural sparse (#5355)
* add pretrained model for neural sparse

Signed-off-by: xinyual <xinyual@amazon.com>

* change typo error

Signed-off-by: xinyual <xinyual@amazon.com>

* Apply suggestions from code review

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/ml-framework.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/ml-framework.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _ml-commons-plugin/ml-framework.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _search-plugins/neural-sparse-search.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/ml-framework.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update pretrained-models.md

Change description and name in the sparse encoding doc model example

Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>

* Update _search-plugins/neural-sparse-search.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _ml-commons-plugin/pretrained-models.md

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

---------

Signed-off-by: xinyual <xinyual@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: xinyual <74362153+xinyual@users.noreply.github.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-23 09:34:09 -04:00
kolchfa-aws a97c719591
Add multimodal search/sparse search/pre- and post-processing function documentation (#5168)
* Add multimodal search documentation

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Text image embedding processor

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add prerequisite

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change query text

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added bedrock connector tutorial and renamed ML TOC

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Name changes and rewording

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change connector link

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change link

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Implemented tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Link fix and field name fix

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add default text embedding preprocessing and post-processing functions

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add sparse search documentation

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Fix links

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Pre/post processing function tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Fix link

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Sparse search tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Implemented doc review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add actual test sparse pipeline response

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added tested examples

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added model choice for sparse search

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Remove Bedrock connector

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Implemented tech review feedback

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add that the model must be deployed to neural search

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Link fix

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add session token to sagemaker blueprint

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Formatted bullet points the same way

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Specified both model types in neural sparse query

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added more explanation for default pre/post-processing functions

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Remove framework and extensibility references

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Minor rewording

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-10-16 10:45:35 -04:00