322 lines
9.3 KiB
Plaintext
322 lines
9.3 KiB
Plaintext
[[search-fields]]
|
||
== Retrieve selected fields from a search
|
||
++++
|
||
<titleabbrev>Retrieve selected fields</titleabbrev>
|
||
++++
|
||
|
||
By default, each hit in the search response includes the document
|
||
<<mapping-source-field,`_source`>>, which is the entire JSON object that was
|
||
provided when indexing the document. To retrieve specific fields in the search
|
||
response, you can use the `fields` parameter:
|
||
|
||
[source,console]
|
||
----
|
||
POST my-index-000001/_search
|
||
{
|
||
"query": {
|
||
"match": {
|
||
"message": "foo"
|
||
}
|
||
},
|
||
"fields": ["user.id", "@timestamp"],
|
||
"_source": false
|
||
}
|
||
----
|
||
// TEST[setup:my_index]
|
||
|
||
The `fields` parameter consults both a document's `_source` and the index
|
||
mappings to load and return values. Because it makes use of the mappings,
|
||
`fields` has some advantages over referencing the `_source` directly: it
|
||
accepts <<multi-fields, multi-fields>> and <<alias, field aliases>>, and
|
||
also formats field values like dates in a consistent way.
|
||
|
||
A document's `_source` is stored as a single field in Lucene. So the whole
|
||
`_source` object must be loaded and parsed even if only a small number of
|
||
fields are requested. To avoid this limitation, you can try another option for
|
||
loading fields:
|
||
|
||
* Use the <<docvalue-fields, `docvalue_fields`>>
|
||
parameter to get values for selected fields. This can be a good
|
||
choice when returning a fairly small number of fields that support doc values,
|
||
such as keywords and dates.
|
||
* Use the <<request-body-search-stored-fields, `stored_fields`>> parameter to
|
||
get the values for specific stored fields (fields that use the
|
||
<<mapping-store,`store`>> mapping option).
|
||
|
||
If needed, you can use the <<script-fields,`script_field`>> parameter to
|
||
transform field values in the response using a script. However, scripts can’t
|
||
make use of {es}'s index structures or related optimizations. This can sometimes
|
||
result in slower search speeds.
|
||
|
||
You can find more detailed information on each of these methods in the
|
||
following sections:
|
||
|
||
* <<search-fields-param>>
|
||
* <<docvalue-fields>>
|
||
* <<stored-fields>>
|
||
* <<source-filtering>>
|
||
* <<script-fields>>
|
||
|
||
[discrete]
|
||
[[search-fields-param]]
|
||
=== Fields
|
||
|
||
The `fields` parameter allows for retrieving a list of document fields in
|
||
the search response. It consults both the document `_source` and the index
|
||
mappings to return each value in a standardized way that matches its mapping
|
||
type. By default, date fields are formatted according to the
|
||
<<mapping-date-format,date format>> parameter in their mappings.
|
||
|
||
The following search request uses the `fields` parameter to retrieve values
|
||
for the `user.id` field, all fields starting with `http.response.`, and the
|
||
`@timestamp` field:
|
||
|
||
[source,console]
|
||
----
|
||
POST my-index-000001/_search
|
||
{
|
||
"query": {
|
||
"match": {
|
||
"user.id": "kimchy"
|
||
}
|
||
},
|
||
"fields": [
|
||
"user.id",
|
||
"http.response.*", <1>
|
||
{
|
||
"field": "@timestamp",
|
||
"format": "epoch_millis" <2>
|
||
}
|
||
],
|
||
"_source": false
|
||
}
|
||
----
|
||
// TEST[setup:my_index]
|
||
|
||
<1> Both full field names and wildcard patterns are accepted.
|
||
<2> Using object notation, you can pass a `format` parameter to apply a custom
|
||
format for the field's values. The date fields
|
||
<<date,`date`>> and <<date_nanos, `date_nanos`>> accept a
|
||
<<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
|
||
accept either `geojson` for http://www.geojson.org[GeoJSON] (the default)
|
||
or `wkt` for
|
||
https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry[Well Known Text].
|
||
Other field types do not support the `format` parameter.
|
||
|
||
The values are returned as a flat list in the `fields` section in each hit:
|
||
|
||
[source,console-result]
|
||
----
|
||
{
|
||
"took" : 2,
|
||
"timed_out" : false,
|
||
"_shards" : {
|
||
"total" : 1,
|
||
"successful" : 1,
|
||
"skipped" : 0,
|
||
"failed" : 0
|
||
},
|
||
"hits" : {
|
||
"total" : {
|
||
"value" : 1,
|
||
"relation" : "eq"
|
||
},
|
||
"max_score" : 1.0,
|
||
"hits" : [
|
||
{
|
||
"_index" : "my-index-000001",
|
||
"_id" : "0",
|
||
"_score" : 1.0,
|
||
"_type" : "_doc",
|
||
"fields" : {
|
||
"user.id" : [
|
||
"kimchy"
|
||
],
|
||
"@timestamp" : [
|
||
"4098435132000"
|
||
],
|
||
"http.response.bytes": [
|
||
1070000
|
||
],
|
||
"http.response.status_code": [
|
||
200
|
||
]
|
||
}
|
||
}
|
||
]
|
||
}
|
||
}
|
||
----
|
||
// TESTRESPONSE[s/"took" : 2/"took": $body.took/]
|
||
// TESTRESPONSE[s/"max_score" : 1.0/"max_score" : $body.hits.max_score/]
|
||
// TESTRESPONSE[s/"_score" : 1.0/"_score" : $body.hits.hits.0._score/]
|
||
|
||
Only leaf fields are returned -- `fields` does not allow for fetching entire
|
||
objects.
|
||
|
||
The `fields` parameter handles field types like <<alias, field aliases>> and
|
||
<<constant-keyword, `constant_keyword`>> whose values aren't always present in
|
||
the `_source`. Other mapping options are also respected, including
|
||
<<ignore-above, `ignore_above`>>, <<ignore-malformed, `ignore_malformed`>> and
|
||
<<null-value, `null_value`>>.
|
||
|
||
NOTE: The `fields` response always returns an array of values for each field,
|
||
even when there is a single value in the `_source`. This is because {es} has
|
||
no dedicated array type, and any field could contain multiple values. The
|
||
`fields` parameter also does not guarantee that array values are returned in
|
||
a specific order. See the mapping documentation on <<array, arrays>> for more
|
||
background.
|
||
|
||
|
||
|
||
[discrete]
|
||
[[docvalue-fields]]
|
||
=== Doc value fields
|
||
|
||
You can use the <<docvalue-fields,`docvalue_fields`>> parameter to return
|
||
<<doc-values,doc values>> for one or more fields in the search response.
|
||
|
||
Doc values store the same values as the `_source` but in an on-disk,
|
||
column-based structure that's optimized for sorting and aggregations. Since each
|
||
field is stored separately, {es} only reads the field values that were requested
|
||
and can avoid loading the whole document `_source`.
|
||
|
||
Doc values are stored for supported fields by default. However, doc values are
|
||
not supported for <<text,`text`>> or
|
||
{plugins}/mapper-annotated-text-usage.html[`text_annotated`] fields.
|
||
|
||
The following search request uses the `docvalue_fields` parameter to retrieve
|
||
doc values for the `user.id` field, all fields starting with `http.response.`, and the
|
||
`@timestamp` field:
|
||
|
||
[source,console]
|
||
----
|
||
GET my-index-000001/_search
|
||
{
|
||
"query": {
|
||
"match": {
|
||
"user.id": "kimchy"
|
||
}
|
||
},
|
||
"docvalue_fields": [
|
||
"user.id",
|
||
"http.response.*", <1>
|
||
{
|
||
"field": "date",
|
||
"format": "epoch_millis" <2>
|
||
}
|
||
]
|
||
}
|
||
----
|
||
// TEST[setup:my_index]
|
||
|
||
<1> Both full field names and wildcard patterns are accepted.
|
||
<2> Using object notation, you can pass a `format` parameter to apply a custom
|
||
format for the field's doc values. <<date,Date fields>> support a
|
||
<<mapping-date-format,date `format`>>. <<number,Numeric fields>> support a
|
||
https://docs.oracle.com/javase/8/docs/api/java/text/DecimalFormat.html[DecimalFormat
|
||
pattern]. Other field datatypes do not support the `format` parameter.
|
||
|
||
TIP: You cannot use the `docvalue_fields` parameter to retrieve doc values for
|
||
nested objects. If you specify a nested object, the search returns an empty
|
||
array (`[ ]`) for the field. To access nested fields, use the
|
||
<<inner-hits, `inner_hits`>> parameter's `docvalue_fields`
|
||
property.
|
||
|
||
[discrete]
|
||
[[stored-fields]]
|
||
=== Stored fields
|
||
|
||
It's also possible to store an individual field's values by using the
|
||
<<mapping-store,`store`>> mapping option. You can use the
|
||
`stored_fields` parameter to include these stored values in the search response.
|
||
|
||
include::request/stored-fields.asciidoc[]
|
||
|
||
[discrete]
|
||
[[source-filtering]]
|
||
=== Source filtering
|
||
|
||
You can use the `_source` parameter to select what fields of the source are
|
||
returned. This is called _source filtering_.
|
||
|
||
The following search API request sets the `_source` request body parameter to
|
||
`false`. The document source is not included in the response.
|
||
|
||
[source,console]
|
||
----
|
||
GET /_search
|
||
{
|
||
"_source": false,
|
||
"query": {
|
||
"match": {
|
||
"user.id": "kimchy"
|
||
}
|
||
}
|
||
}
|
||
----
|
||
|
||
To return only a subset of source fields, specify a wildcard (`*`) pattern in
|
||
the `_source` parameter. The following search API request returns the source for
|
||
only the `obj` field and its properties.
|
||
|
||
[source,console]
|
||
----
|
||
GET /_search
|
||
{
|
||
"_source": "obj.*",
|
||
"query": {
|
||
"match": {
|
||
"user.id": "kimchy"
|
||
}
|
||
}
|
||
}
|
||
----
|
||
|
||
You can also specify an array of wildcard patterns in the `_source` field. The
|
||
following search API request returns the source for only the `obj1` and
|
||
`obj2` fields and their properties.
|
||
|
||
[source,console]
|
||
----
|
||
GET /_search
|
||
{
|
||
"_source": [ "obj1.*", "obj2.*" ],
|
||
"query": {
|
||
"match": {
|
||
"user.id": "kimchy"
|
||
}
|
||
}
|
||
}
|
||
----
|
||
|
||
For finer control, you can specify an object containing arrays of `includes` and
|
||
`excludes` patterns in the `_source` parameter.
|
||
|
||
If the `includes` property is specified, only source fields that match one of
|
||
its patterns are returned. You can exclude fields from this subset using the
|
||
`excludes` property.
|
||
|
||
If the `includes` property is not specified, the entire document source is
|
||
returned, excluding any fields that match a pattern in the `excludes` property.
|
||
|
||
The following search API request returns the source for only the `obj1` and
|
||
`obj2` fields and their properties, excluding any child `description` fields.
|
||
|
||
[source,console]
|
||
----
|
||
GET /_search
|
||
{
|
||
"_source": {
|
||
"includes": [ "obj1.*", "obj2.*" ],
|
||
"excludes": [ "*.description" ]
|
||
},
|
||
"query": {
|
||
"term": {
|
||
"user.id": "kimchy"
|
||
}
|
||
}
|
||
}
|
||
----
|
||
|
||
include::request/script-fields.asciidoc[] |