opensearch-docs-cn/_api-reference/explain.md

166 lines
6.7 KiB
Markdown
Raw Normal View History

2021-08-16 03:24:18 -04:00
---
layout: default
title: Explain
Make API reference top level (#1637) * Make API reference top level Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix typo on Drag and Drop page (#1633) * Fix typo on Drag and Drop page * Update _dashboards/drag-drop-wizard.md Co-authored-by: Nate Bower <nbower@amazon.com> * Update drag-drop-wizard.md Co-authored-by: Nate Bower <nbower@amazon.com> * Putting all the Docker install material on a single page (#1452) * Putting all the Docker install material on a single page Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Making room for revamp Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Intro added Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Continuing to flesh out the intro section and overview Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Overview finalized Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Introducing docker compose Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Added link to compose Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Continuing docker image commentary Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Sometimes I wonder if anyone reads these Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Adding notes on installing compose with pip Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Adding prereqs Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Magnets - how do they work? Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Almonds and peaches are part of the same plant subgenus, Amygdalus Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * There are 293 ways to make change for a dollar Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * A shark is the only known fish that can blink with both eyes Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * A crocodile cannot stick its tongue out Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * wording Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Reorganizing a couple paragraphs to make it flow better Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Forgot a word Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Add tip about pruning stopped containers Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Cleaning up Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Add blurb about container ls Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Adding the Docker Compose stuff Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Working on compose Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Continuing work on the compose section - it's a lot of info Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Added important settings Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Updates to settings that need configured Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Still working through compose things Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Fixed wording Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Working through compose commands and guidance Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Reordering/rewording Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * More phrasing Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * More wording in steps Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * More wording in steps Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Organizing Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Adding stuff and things Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Continuing to work through the configuration steps Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Fixes Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Fixes Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Still working on the configuration steps Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Changes Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * More work Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Removed perf analyzer - refer to GH issue 1555 Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Fixing things Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Adding guidance on passing settings in compose Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Working through dockerfile materials now Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * wording Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Finalized the sample dev compose file Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Continuing work with configuration Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Finished - ready for reviews Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Fixed a link I forgot to change before Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Changes from first proofread Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Changed heading Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Addressed reviewer comments and made some changes Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Forgot to incorporate one change. Fixed. Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Final editorial changes Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * fix#1584-custom_attr_allowlist (#1636) Signed-off-by: cwillum <cwmmoore@amazon.com> Signed-off-by: cwillum <cwmmoore@amazon.com> * Update TERMS.md with definition for Setting (#1632) * fix#1631-Terms-setting Signed-off-by: cwillum <cwmmoore@amazon.com> * fix#1631-Terms-setting Signed-off-by: cwillum <cwmmoore@amazon.com> Signed-off-by: cwillum <cwmmoore@amazon.com> * Add disclaimer about remote fs usage and an example of setting env var (#1644) * Add disclaimer about remote fs usage and an example of setting env var Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * Enhanced wording a little bit Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> * [DOC] New documentation: Self-host maps server (#1625) * Add new page self-host maps server Signed-off-by: vagimeli <vagimeli@amazon.com> * Added new content Signed-off-by: vagimeli <vagimeli@amazon.com> * Copy edit Signed-off-by: vagimeli <vagimeli@amazon.com> * Tech review edits Signed-off-by: vagimeli <vagimeli@amazon.com> * Doc review edits Signed-off-by: vagimeli <vagimeli@amazon.com> * Editorial review changes Signed-off-by: vagimeli <vagimeli@amazon.com> * Final edits Signed-off-by: vagimeli <vagimeli@amazon.com> Signed-off-by: vagimeli <vagimeli@amazon.com> * Add feedback. Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Fix links Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: JeffH-AWS <jeffhuss@amazon.com> Signed-off-by: cwillum <cwmmoore@amazon.com> Signed-off-by: vagimeli <vagimeli@amazon.com> Co-authored-by: Nate Bower <nbower@amazon.com> Co-authored-by: Jeff Huss <jeffhuss@amazon.com> Co-authored-by: Chris Moore <107723039+cwillum@users.noreply.github.com> Co-authored-by: Melissa Vagi <105296784+vagimeli@users.noreply.github.com>
2022-10-27 12:50:39 -04:00
nav_order: 30
redirect_from:
- /opensearch/rest-api/explain/
2021-08-16 03:24:18 -04:00
---
# Explain
Introduced 1.0
{: .label .label-purple }
2021-08-16 14:00:48 -04:00
Wondering why a specific document ranks higher (or lower) for a query? You can use the explain API for an explanation of how the relevance score (`_score`) is calculated for every result.
2021-08-16 03:24:18 -04:00
OpenSearch uses a probabilistic ranking framework called [Okapi BM25](https://en.wikipedia.org/wiki/Okapi_BM25) to calculate relevance scores. Okapi BM25 is based on the original [TF/IDF](https://lucene.apache.org/core/{{site.lucene_version}}/core/org/apache/lucene/search/package-summary.html#scoring) framework used by Apache Lucene.
2021-08-16 03:24:18 -04:00
2021-08-16 15:01:53 -04:00
The explain API is an expensive operation in terms of both resources and time. On production clusters, we recommend using it sparingly for the purpose of troubleshooting.
2021-08-16 03:24:18 -04:00
{: .warning }
## Example
To see the explain output for all results, set the `explain` flag to `true` either in the URL or in the body of the request:
```json
POST opensearch_dashboards_sample_data_ecommerce/_search?explain=true
2021-08-16 03:24:18 -04:00
{
"query": {
"match": {
"customer_first_name": "Mary"
}
}
}
```
{% include copy-curl.html %}
2021-08-16 03:24:18 -04:00
2021-08-16 15:01:53 -04:00
More often, you want the output for a single document. In that case, specify the document ID in the URL:
2021-08-16 03:24:18 -04:00
```json
POST opensearch_dashboards_sample_data_ecommerce/_explain/EVz1Q3sBgg5eWQP6RSte
2021-08-16 03:24:18 -04:00
{
"query": {
"match": {
"customer_first_name": "Mary"
}
}
}
```
{% include copy-curl.html %}
2021-08-16 03:24:18 -04:00
## Path and HTTP methods
```
GET <target>/_explain/<id>
POST <target>/_explain/<id>
```
## URL parameters
You must specify the index and document ID. All other URL parameters are optional.
Parameter | Type | Description | Required
:--- | :--- | :--- | :---
`<index>` | String | Name of the index. You can only specify a single index. | Yes
`<_id>` | String | A unique identifier to attach to the document. | Yes
`analyzer` | String | The analyzer to use in the query string. | No
`analyze_wildcard` | Boolean | Specifies whether to analyze wildcard and prefix queries. Default is false. | No
`default_operator` | String | Indicates whether the default operator for a string query should be AND or OR. Default is OR. | No
`df` | String | The default field in case a field prefix is not provided in the query string. | No
`lenient` | Boolean | Specifies whether OpenSearch should ignore format-based query failures (for example, querying a text field for an integer). Default is false. | No
2021-08-16 16:21:29 -04:00
`preference` | String | Specifies a preference of which shard to retrieve results from. Available options are `_local`, which tells the operation to retrieve results from a locally allocated shard replica, and a custom string value assigned to a specific shard replica. By default, OpenSearch executes the explain operation on random shards. | No
2021-08-16 03:24:18 -04:00
`q` | String | Query in the Lucene query string syntax. | No
`stored_fields` | Boolean | If true, the operation retrieves document fields stored in the index rather than the documents `_source`. Default is false. | No
`routing` | String | Value used to route the operation to a specific shard. | No
`_source` | String | Whether to include the `_source` field in the response body. Default is true. | No
`_source_excludes` | String | A comma-separated list of source fields to exclude in the query response. | No
`_source_includes` | String | A comma-separated list of source fields to include in the query response. | No
## Response
```json
{
"_index" : "kibana_sample_data_ecommerce",
"_id" : "EVz1Q3sBgg5eWQP6RSte",
"matched" : true,
"explanation" : {
"value" : 3.5671005,
"description" : "weight(customer_first_name:mary in 1) [PerFieldSimilarity], result of:",
"details" : [
{
"value" : 3.5671005,
"description" : "score(freq=1.0), computed as boost * idf * tf from:",
"details" : [
{
"value" : 2.2,
"description" : "boost",
"details" : [ ]
},
{
"value" : 3.4100041,
"description" : "idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from:",
"details" : [
{
"value" : 154,
"description" : "n, number of documents containing term",
"details" : [ ]
},
{
"value" : 4675,
"description" : "N, total number of documents with field",
"details" : [ ]
}
]
},
{
"value" : 0.47548598,
"description" : "tf, computed as freq / (freq + k1 * (1 - b + b * dl / avgdl)) from:",
"details" : [
{
"value" : 1.0,
"description" : "freq, occurrences of term within document",
"details" : [ ]
},
{
"value" : 1.2,
"description" : "k1, term saturation parameter",
"details" : [ ]
},
{
"value" : 0.75,
"description" : "b, length normalization parameter",
"details" : [ ]
},
{
"value" : 1.0,
"description" : "dl, length of field",
"details" : [ ]
},
{
"value" : 1.1206417,
"description" : "avgdl, average length of field",
"details" : [ ]
}
]
}
]
}
]
}
}
```
## Response body fields
Field | Description
:--- | :---
`matched` | Indicates if the document is a match for the query.
`explanation` | The `explanation` object has three properties: `value`, `description`, and `details`. The `value` shows the result of the calculation, the `description` explains what type of calculation is performed, and the `details` shows any subcalculations performed.
Term frequency (`tf`) | How many times the term appears in a field for a given document. The more times the term occurs the higher is the relevance score.
2021-08-16 15:04:23 -04:00
Inverse document frequency (`idf`) | How often the term appears within the index (across all the documents). The more often the term appears the lower is the relevance score.
2021-08-16 03:24:18 -04:00
Field normalization factor (`fieldNorm`) | The length of the field. OpenSearch assigns a higher relevance score to a term appearing in a relatively short field.
The `tf`, `idf`, and `fieldNorm` values are calculated and stored at index time when a document is added or updated. The values might have some (typically small) inaccuracies as its based on summing the samples returned from each shard.
2021-08-16 03:24:18 -04:00
Individual queries include other factors for calculating the relevance score, such as term proximity, fuzziness, and so on.