kolchfa-aws f5b8ff79fd
Add score normalization and combination documentation (#4985)
* Add search phase results processor

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add hybrid query

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Normalization processor additions

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add more details

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Continue writing

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add more query then fetch details and diagram

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Small rewording

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Leaner left nav headers

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Tech review feedback

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add semantic search tutorial

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Reworded prerequisites

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Removed comma

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Rewording advanced prerequisites

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Changed searching for ML model to shorter request

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Update task type in register model response

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Changing example

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added huggingface prefix to model names

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change example responses

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added note about huggingface prefix

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Update _ml-commons-plugin/semantic-search.md

Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Implemented doc review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* List weights under parameters

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Remove one-shard warning for normalization processor

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Implemented editorial comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change links

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* More editorial feedback

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Change model-serving framework to ML framework

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Use get model API to check model status

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Implemented tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Added neural search description and diagram

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* More editorial comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add link to profile API

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Addressed more tech review comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Implemented editorial comments on changes

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
2023-09-22 17:29:58 -04:00

3.9 KiB

layout title parent grand_parent nav_order redirect_from
default Boosting Compound queries Query DSL 30
/query-dsl/query-dsl/compound/boosting/

Boosting queries

If you're searching for the word "pitcher", your results may relate to either baseball players or containers for liquids. For a search in the context of baseball, you might want to completely exclude results that contain the words "glass" or "water" by using the must_not clause. However, if you want to keep those results but downgrade them in relevance, you can do so with boosting queries.

A boosting query returns documents that match a positive query. Among those documents, the ones that also match the negative query are scored lower in relevance (their relevance score is multiplied by the negative boosting factor).

Example

Consider an index with two documents that you index as follows:

PUT testindex/_doc/1
{
  "article_name": "The greatest pitcher in baseball history"
}
PUT testindex/_doc/2
{
  "article_name": "The making of a glass pitcher"
}

Use the following match query to search for documents containing the word "pitcher":

GET testindex/_search
{
  "query": {
    "match": {
      "article_name": "pitcher"
    }
  }
}

Both returned documents have the same relevance score:

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 0.18232156,
    "hits": [
      {
        "_index": "testindex",
        "_id": "1",
        "_score": 0.18232156,
        "_source": {
          "article_name": "The greatest pitcher in baseball history"
        }
      },
      {
        "_index": "testindex",
        "_id": "2",
        "_score": 0.18232156,
        "_source": {
          "article_name": "The making of a glass pitcher"
        }
      }
    ]
  }
}

Now use the following boosting query to search for documents containing the word "pitcher" but downgrade the documents that contain the words "glass", "crystal", or "water":

GET testindex/_search
{
  "query": {
    "boosting": {
      "positive": {
        "match": {
          "article_name": "pitcher"
        }
      },
      "negative": {
        "match": {
          "article_name": "glass crystal water"
        }
      },
      "negative_boost": 0.1
    }
  }
}

{% include copy-curl.html %}

Both documents are still returned, but the document with the word "glass" has a relevance score that is 10 times lower than in the previous case:

{
  "took": 13,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "max_score": 0.18232156,
    "hits": [
      {
        "_index": "testindex",
        "_id": "1",
        "_score": 0.18232156,
        "_source": {
          "article_name": "The greatest pitcher in baseball history"
        }
      },
      {
        "_index": "testindex",
        "_id": "2",
        "_score": 0.018232157,
        "_source": {
          "article_name": "The making of a glass pitcher"
        }
      }
    ]
  }
}

Parameters

The following table lists all top-level parameters supported by boosting queries.

Parameter Description
positive The query that a document must match to be returned in the results. Required.
negative If a document in the results matches this query, its relevance score is reduced by multiplying its original relevance score (produced by the positive query) by the negative_boost parameter. Required.
negative_boost A floating-point factor between 0 and 1.0 that the original relevance score is multiplied by in order to reduce the relevance of documents that match the negative query. Required.