OpenSearch/docs/reference/search/request/inner-hits.asciidoc

[[search-request-inner-hits]]
=== Inner hits

experimental[]

The <<mapping-parent-field, parent/child>> and <<nested, nested>> features allow the return of documents that
have matches in a different scope. In the parent/child case, parent document are returned based on matches in child
documents or child document are returned based on matches in parent documents. In the nested case, documents are returned
based on matches in nested inner objects.

In both cases, the actual matches in the different scopes that caused a document to be returned is hidden. In many cases,
it's very useful to know which inner nested objects (in the case of nested or children or parent documents), or (in the case
of parent/child) caused certain information to be returned. The inner hits feature can be used for this. This feature
returns per search hit in the search response additional nested hits that caused a search hit to match in a different scope.

Inner hits can be used by defining a `inner_hits` definition on a `nested`, `has_child` or `has_parent` query and filter.
The structure looks like this:

[source,js]
--------------------------------------------------
"<query>" : {
    "inner_hits" : {
        <inner_hits_options>
    }
}
--------------------------------------------------

If `_inner_hits` is defined on a query that supports it then each search hit will contain a `inner_hits` json object with the following structure:

[source,js]
--------------------------------------------------
"hits": [
     {
        "_index": ...,
        "_type": ...,
        "_id": ...,
        "inner_hits": {
           "<inner_hits_name>": {
              "hits": {
                 "total": ...,
                 "hits": [
                    {
                       "_type": ...,
                       "_id": ...,
                       ...
                    },
                    ...
                 ]
              }
           }
        },
        ...
     },
     ...
]
--------------------------------------------------

==== Options

Inner hits support the following options:

[horizontal]
`from`:: The offset from where the first hit to fetch for each `inner_hits` in the returned regular search hits.
`size`:: The maximum number of hits to return per `inner_hits`. By default the top three matching hits are returned.
`sort`:: How the inner hits should be sorted per `inner_hits`. By default the hits are sorted by the score.
`name`:: The name to be used for the particular inner hit definition in the response. Useful when multiple inner hits
         have been defined in a single search request. The default depends in which query the inner hit is defined.
         For `has_child` query and filter this is the child type, `has_parent` query and filter this is the parent type
         and the nested query and filter this is the nested path.

Inner hits also supports the following per document features:

* <<search-request-highlighting,Highlighting>>
* <<search-request-explain,Explain>>
* <<search-request-source-filtering,Source filtering>>
* <<search-request-script-fields,Script fields>>
* <<search-request-fielddata-fields,Fielddata fields>>
* <<search-request-version,Include versions>>

[[nested-inner-hits]]
==== Nested inner hits

The nested `inner_hits` can be used to include nested inner objects as inner hits to a search hit.

The example below assumes that there is a nested object field defined with the name `comments`:

[source,js]
--------------------------------------------------
{
    "query" : {
        "nested" : {
            "path" : "comments",
            "query" : {
                "match" : {"comments.message" : "[actual query]"}
            },
            "inner_hits" : {} <1>
        }
    }
}
--------------------------------------------------

<1> The inner hit definition in the nested query. No other options need to be defined.

An example of a response snippet that could be generated from the above search request:

[source,js]
--------------------------------------------------
...
"hits": {
  ...
  "hits": [
     {
        "_index": "my-index",
        "_type": "question",
        "_id": "1",
        "_source": ...,
        "inner_hits": {
           "comments": { <1>
              "hits": {
                 "total": ...,
                 "hits": [
                    {
                       "_type": "question",
                       "_id": "1",
                       "_nested": {
                          "field": "comments",
                          "offset": 2
                       },
                       "_source": ...
                    },
                    ...
                 ]
              }
           }
        }
     },
     ...
--------------------------------------------------

<1> The name used in the inner hit definition in the search request. A custom key can be used via the `name` option.

The `_nested` metadata is crucial in the above example, because it defines from what inner nested object this inner hit
came from. The `field` defines the object array field the nested hit is from and the `offset` relative to its location
in the `_source`. Due to sorting and scoring the actual location of the hit objects in the `inner_hits` is usually
different than the location a nested inner object was defined.

By default the `_source` is returned also for the hit objects in `inner_hits`, but this can be changed. Either via
`_source` filtering feature part of the source can be returned or be disabled. If stored fields are defined on the
nested level these can also be returned via the `fields` feature.

An important default is that the `_source` returned in hits inside `inner_hits` is relative to the `_nested` metadata.
So in the above example only the comment part is returned per nested hit and not the entire source of the top level
document that contained the the comment.

[[hierarchical-nested-inner-hits]]
==== Hierarchical levels of nested object fields and inner hits.

If a mapping has multiple levels of hierarchical nested object fields each level can be accessed via dot notated path.
For example if there is a `comments` nested field that contains a `votes` nested field and votes should directly be returned
with the the root hits then the following path can be defined:

[source,js]
--------------------------------------------------
{
   "query" : {
      "nested" : {
         "path" : "comments.votes",
         "query" : { ... },
         "inner_hits" : {}
      }
    }
}
--------------------------------------------------

This indirect referencing is only supported for nested inner hits.

[[parent-child-inner-hits]]
==== Parent/child inner hits

The parent/child `inner_hits` can be used to include parent or child

The examples below assumes that there is a `_parent` field mapping in the `comment` type:

[source,js]
--------------------------------------------------
{
    "query" : {
        "has_child" : {
            "type" : "comment",
            "query" : {
                "match" : {"message" : "[actual query]"}
            },
            "inner_hits" : {} <1>
        }
    }
}
--------------------------------------------------

<1> The inner hit definition like in the nested example.

An example of a response snippet that could be generated from the above search request:

[source,js]
--------------------------------------------------
...
"hits": {
  ...
  "hits": [
     {
        "_index": "my-index",
        "_type": "question",
        "_id": "1",
        "_source": ...,
        "inner_hits": {
           "comment": {
              "hits": {
                 "total": ...,
                 "hits": [
                    {
                       "_type": "comment",
                       "_id": "5",
                       "_source": ...
                    },
                    ...
                 ]
              }
           }
        }
     },
     ...
--------------------------------------------------

[[top-level-inner-hits]]
==== top level inner hits

Besides defining inner hits on query and filters, inner hits can also be defined as a top level construct alongside the
`query` and `aggregations` definition. The main reason for using the top level inner hits definition is to let the
inner hits return documents that don't match with the main query. Also inner hits definitions can be nested via the
top level notation. Other then that the inner hit definition inside the query should be used, because that is the most
compact way for defining inner hits.

The following snippet explains the basic structure of inner hits defined at the top level of the search request body:

[source,js]
--------------------------------------------------
"inner_hits" : {
    "<inner_hits_name>" : {
        "<path|type>" : {
            "<path-to-nested-object-field|child-or-parent-type>" : {
                <inner_hits_body>
                [,"inner_hits" : { [<sub_inner_hits>]+ } ]?
            }
        }
    }
    [,"<inner_hits_name_2>" : { ... } ]*
}
--------------------------------------------------

Inside the `inner_hits` definition, first the name if the inner hit is defined then whether the inner_hit
is a nested by defining `path` or a parent/child based definition by defining `type`. The next object layer contains
the name of the nested object field if the inner_hits is nested or the parent or child type if the inner_hit definition
is parent/child based.

Multiple inner hit definitions can be defined in a single request. In the `<inner_hits_body>` any option for features
that `inner_hits` support can be defined. Optionally another `inner_hits` definition can be defined in the `<inner_hits_body>`.

An example that shows the use of nested inner hits via the top level notation:

[source,js]
--------------------------------------------------
{
    "query" : {
        "nested" : {
            "path" : "comments",
            "query" : {
                "match" : {"comments.message" : "[actual query]"}
            }
        }
    },
    "inner_hits" : {
        "comment" : {
            "path" : { <1>
                "comments" : { <2>
                    "query" : {
                        "match" : {"comments.message" : "[different query]"} <3>
                    }
                }
            }
        }
    }
}
--------------------------------------------------

<1> The inner hit definition is nested and requires the `path` option.
<2> The path option refers to the nested object field `comments`
<3> A query that runs to collect the nested inner documents for each search hit returned. If no query is defined all nested
    inner documents will be included belonging to a search hit. This shows that it only make sense to the top level
    inner hit definition if no query or a different query is specified.

Additional options that are only available when using the top level inner hits notation:

[horizontal]
`path`:: Defines the nested scope where hits will be collected from.
`type`:: Defines the parent or child type score where hits will be collected from.
`query`:: Defines the query that will run in the defined nested, parent or child scope to collect and score hits. By default all document in the scope will be matched.

Either `path` or `type` must be defined. The `path` or `type` defines the scope from where hits are fetched and
used as inner hits.