OpenSearch/docs/reference/search/search-fields.asciidoc

[[search-fields]]
== Retrieve selected fields from a search
++++
<titleabbrev>Retrieve selected fields</titleabbrev>
++++

By default, each hit in the search response includes the document
<<mapping-source-field,`_source`>>, which is the entire JSON object that was
provided when indexing the document. To retrieve specific fields in the search
response, you can use the `fields` parameter:

[source,console]
----
POST my-index-000001/_search
{
  "query": {
    "match": {
      "message": "foo"
    }
  },
  "fields": ["user.id", "@timestamp"],
  "_source": false
}
----
// TEST[setup:my_index]

The `fields` parameter consults both a document's `_source` and the index
mappings to load and return values. Because it makes use of the mappings,
`fields` has some advantages over referencing the `_source` directly: it
accepts <<multi-fields, multi-fields>> and <<alias, field aliases>>, and
also formats field values like dates in a consistent way.

A document's `_source` is stored as a single field in Lucene. So the whole
`_source` object must be loaded and parsed even if only a small number of
fields are requested. To avoid this limitation, you can try another option for
loading fields:

* Use the <<docvalue-fields, `docvalue_fields`>>
parameter to get values for selected fields. This can be a good
choice when returning a fairly small number of fields that support doc values,
such as keywords and dates.
* Use the <<request-body-search-stored-fields, `stored_fields`>> parameter to
get the values for specific stored fields (fields that use the
<<mapping-store,`store`>> mapping option).

If needed, you can use the <<script-fields,`script_field`>> parameter to
transform field values in the response using a script. However, scripts can’t
make use of {es}'s index structures or related optimizations. This can sometimes
result in slower search speeds.

You can find more detailed information on each of these methods in the
following sections:

* <<search-fields-param>>
* <<docvalue-fields>>
* <<stored-fields>>
* <<source-filtering>>
* <<script-fields>>

[discrete]
[[search-fields-param]]
=== Fields

The `fields` parameter allows for retrieving a list of document fields in
the search response. It consults both the document `_source` and the index
mappings to return each value in a standardized way that matches its mapping
type. By default, date fields are formatted according to the
<<mapping-date-format,date format>> parameter in their mappings.

The following search request uses the `fields` parameter to retrieve values
for the `user.id` field, all fields starting with `http.response.`, and the
`@timestamp` field:

[source,console]
----
POST my-index-000001/_search
{
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  },
  "fields": [
    "user.id",
    "http.response.*", <1>
    {
      "field": "@timestamp",
      "format": "epoch_millis" <2>
    }
  ],
  "_source": false
}
----
// TEST[setup:my_index]

<1> Both full field names and wildcard patterns are accepted.
<2> Using object notation, you can pass a `format` parameter to apply a custom
    format for the field's values. The date fields
    <<date,`date`>> and <<date_nanos, `date_nanos`>> accept a
    <<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
    accept either `geojson` for http://www.geojson.org[GeoJSON] (the default)
    or `wkt` for
    {wikipedia}/Well-known_text_representation_of_geometry[Well Known Text].
    Other field types do not support the `format` parameter.

The values are returned as a flat list in the `fields` section in each hit:

[source,console-result]
----
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "my-index-000001",
        "_id" : "0",
        "_score" : 1.0,
        "_type" : "_doc",
        "fields" : {
          "user.id" : [
            "kimchy"
          ],
          "@timestamp" : [
            "4098435132000"
          ],
          "http.response.bytes": [
            1070000
          ],
          "http.response.status_code": [
            200
          ]
        }
      }
    ]
  }
}
----
// TESTRESPONSE[s/"took" : 2/"took": $body.took/]
// TESTRESPONSE[s/"max_score" : 1.0/"max_score" : $body.hits.max_score/]
// TESTRESPONSE[s/"_score" : 1.0/"_score" : $body.hits.hits.0._score/]

Only leaf fields are returned -- `fields` does not allow for fetching entire
objects.

The `fields` parameter handles field types like <<alias, field aliases>> and
<<constant-keyword, `constant_keyword`>> whose values aren't always present in
the `_source`. Other mapping options are also respected, including
<<ignore-above, `ignore_above`>>, <<ignore-malformed, `ignore_malformed`>> and
<<null-value, `null_value`>>.

NOTE: The `fields` response always returns an array of values for each field,
even when there is a single value in the `_source`. This is because {es} has
no dedicated array type, and any field could contain multiple values. The
`fields` parameter also does not guarantee that array values are returned in
a specific order. See the mapping documentation on <<array, arrays>> for more
background.



[discrete]
[[docvalue-fields]]
=== Doc value fields

You can use the <<docvalue-fields,`docvalue_fields`>> parameter to return
<<doc-values,doc values>> for one or more fields in the search response.

Doc values store the same values as the `_source` but in an on-disk,
column-based structure that's optimized for sorting and aggregations. Since each
field is stored separately, {es} only reads the field values that were requested
and can avoid loading the whole document `_source`.

Doc values are stored for supported fields by default. However, doc values are
not supported for <<text,`text`>> or
{plugins}/mapper-annotated-text-usage.html[`text_annotated`] fields.

The following search request uses the `docvalue_fields` parameter to retrieve
doc values for the `user.id` field, all fields starting with `http.response.`, and the
`@timestamp` field:

[source,console]
----
GET my-index-000001/_search
{
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  },
  "docvalue_fields": [
    "user.id",
    "http.response.*", <1>
    {
      "field": "date",
      "format": "epoch_millis" <2>
    }
  ]
}
----
// TEST[setup:my_index]

<1> Both full field names and wildcard patterns are accepted.
<2> Using object notation, you can pass a `format` parameter to apply a custom
    format for the field's doc values. <<date,Date fields>> support a
    <<mapping-date-format,date `format`>>. <<number,Numeric fields>> support a
    https://docs.oracle.com/javase/8/docs/api/java/text/DecimalFormat.html[DecimalFormat
    pattern]. Other field datatypes do not support the `format` parameter.

TIP: You cannot use the `docvalue_fields` parameter to retrieve doc values for
nested objects. If you specify a nested object, the search returns an empty
array (`[ ]`) for the field. To access nested fields, use the
<<inner-hits, `inner_hits`>> parameter's `docvalue_fields`
property.

[discrete]
[[stored-fields]]
=== Stored fields

It's also possible to store an individual field's values by using the
<<mapping-store,`store`>> mapping option. You can use the
`stored_fields` parameter to include these stored values in the search response.

include::request/stored-fields.asciidoc[]

[discrete]
[[source-filtering]]
=== Source filtering

You can use the `_source` parameter to select what fields of the source are
returned. This is called _source filtering_.

The following search API request sets the `_source` request body parameter to
`false`. The document source is not included in the response.

[source,console]
----
GET /_search
{
  "_source": false,
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  }
}
----

To return only a subset of source fields, specify a wildcard (`*`) pattern in
the `_source` parameter. The following search API request returns the source for
only the `obj` field and its properties.

[source,console]
----
GET /_search
{
  "_source": "obj.*",
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  }
}
----

You can also specify an array of wildcard patterns in the `_source` field. The
following search API request returns the source for only the `obj1` and
`obj2` fields and their properties.

[source,console]
----
GET /_search
{
  "_source": [ "obj1.*", "obj2.*" ],
  "query": {
    "match": {
      "user.id": "kimchy"
    }
  }
}
----

For finer control, you can specify an object containing arrays of `includes` and
`excludes` patterns in the `_source` parameter.

If the `includes` property is specified, only source fields that match one of
its patterns are returned. You can exclude fields from this subset using the
`excludes` property.

If the `includes` property is not specified, the entire document source is
returned, excluding any fields that match a pattern in the `excludes` property.

The following search API request returns the source for only the `obj1` and
`obj2` fields and their properties, excluding any child `description` fields.

[source,console]
----
GET /_search
{
  "_source": {
    "includes": [ "obj1.*", "obj2.*" ],
    "excludes": [ "*.description" ]
  },
  "query": {
    "term": {
      "user.id": "kimchy"
    }
  }
}
----

include::request/script-fields.asciidoc[]