312 lines
8.9 KiB
Plaintext
312 lines
8.9 KiB
Plaintext
[discrete]
|
|
[[search-fields]]
|
|
=== Retrieve selected fields
|
|
|
|
By default, each hit in the search response includes the document
|
|
<<mapping-source-field,`_source`>>, which is the entire JSON object that was
|
|
provided when indexing the document. To retrieve specific fields in the search
|
|
response, you can use the `fields` parameter:
|
|
|
|
[source,console]
|
|
----
|
|
POST my-index-000001/_search
|
|
{
|
|
"query": {
|
|
"match": {
|
|
"message": "foo"
|
|
}
|
|
},
|
|
"fields": ["user.id", "@timestamp"],
|
|
"_source": false
|
|
}
|
|
----
|
|
// TEST[setup:my_index]
|
|
|
|
The `fields` parameter consults both a document's `_source` and the index
|
|
mappings to load and return values. Because it makes use of the mappings,
|
|
`fields` has some advantages over referencing the `_source` directly: it
|
|
accepts <<multi-fields, multi-fields>> and <<alias, field aliases>>, and
|
|
also formats field values like dates in a consistent way.
|
|
|
|
A document's `_source` is stored as a single field in Lucene. So the whole
|
|
`_source` object must be loaded and parsed even if only a small number of
|
|
fields are requested. To avoid this limitation, you can try another option for
|
|
loading fields:
|
|
|
|
* Use the <<docvalue-fields, `docvalue_fields`>>
|
|
parameter to get values for selected fields. This can be a good
|
|
choice when returning a fairly small number of fields that support doc values,
|
|
such as keywords and dates.
|
|
* Use the <<request-body-search-stored-fields, `stored_fields`>> parameter to
|
|
get the values for specific stored fields (fields that use the
|
|
<<mapping-store,`store`>> mapping option).
|
|
|
|
You can find more detailed information on each of these methods in the
|
|
following sections:
|
|
|
|
* <<search-fields-param>>
|
|
* <<docvalue-fields>>
|
|
* <<stored-fields>>
|
|
* <<source-filtering>>
|
|
|
|
[discrete]
|
|
[[search-fields-param]]
|
|
=== Fields
|
|
|
|
The `fields` parameter allows for retrieving a list of document fields in
|
|
the search response. It consults both the document `_source` and the index
|
|
mappings to return each value in a standardized way that matches its mapping
|
|
type. By default, date fields are formatted according to the
|
|
<<mapping-date-format,date format>> parameter in their mappings.
|
|
|
|
The following search request uses the `fields` parameter to retrieve values
|
|
for the `user.id` field, all fields starting with `http.response.`, and the
|
|
`@timestamp` field:
|
|
|
|
[source,console]
|
|
----
|
|
POST my-index-000001/_search
|
|
{
|
|
"query": {
|
|
"match": {
|
|
"user.id": "kimchy"
|
|
}
|
|
},
|
|
"fields": [
|
|
"user.id",
|
|
"http.response.*", <1>
|
|
{
|
|
"field": "@timestamp",
|
|
"format": "epoch_millis" <2>
|
|
}
|
|
],
|
|
"_source": false
|
|
}
|
|
----
|
|
// TEST[setup:my_index]
|
|
|
|
<1> Both full field names and wildcard patterns are accepted.
|
|
<2> Using object notation, you can pass a `format` parameter to apply a custom
|
|
format for the field's values. The date fields
|
|
<<date,`date`>> and <<date_nanos, `date_nanos`>> accept a
|
|
<<mapping-date-format,date format>>. <<spatial_datatypes, Spatial fields>>
|
|
accept either `geojson` for http://www.geojson.org[GeoJSON] (the default)
|
|
or `wkt` for
|
|
https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry[Well Known Text].
|
|
Other field types do not support the `format` parameter.
|
|
|
|
The values are returned as a flat list in the `fields` section in each hit:
|
|
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"took" : 2,
|
|
"timed_out" : false,
|
|
"_shards" : {
|
|
"total" : 1,
|
|
"successful" : 1,
|
|
"skipped" : 0,
|
|
"failed" : 0
|
|
},
|
|
"hits" : {
|
|
"total" : {
|
|
"value" : 1,
|
|
"relation" : "eq"
|
|
},
|
|
"max_score" : 1.0,
|
|
"hits" : [
|
|
{
|
|
"_index" : "my-index-000001",
|
|
"_id" : "0",
|
|
"_score" : 1.0,
|
|
"_type" : "_doc",
|
|
"fields" : {
|
|
"user.id" : [
|
|
"kimchy"
|
|
],
|
|
"@timestamp" : [
|
|
"4098435132000"
|
|
],
|
|
"http.response.bytes": [
|
|
1070000
|
|
],
|
|
"http.response.status_code": [
|
|
200
|
|
]
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
----
|
|
// TESTRESPONSE[s/"took" : 2/"took": $body.took/]
|
|
// TESTRESPONSE[s/"max_score" : 1.0/"max_score" : $body.hits.max_score/]
|
|
// TESTRESPONSE[s/"_score" : 1.0/"_score" : $body.hits.hits.0._score/]
|
|
|
|
Only leaf fields are returned -- `fields` does not allow for fetching entire
|
|
objects.
|
|
|
|
The `fields` parameter handles field types like <<alias, field aliases>> and
|
|
<<constant-keyword, `constant_keyword`>> whose values aren't always present in
|
|
the `_source`. Other mapping options are also respected, including
|
|
<<ignore-above, `ignore_above`>>, <<ignore-malformed, `ignore_malformed`>> and
|
|
<<null-value, `null_value`>>.
|
|
|
|
NOTE: The `fields` response always returns an array of values for each field,
|
|
even when there is a single value in the `_source`. This is because {es} has
|
|
no dedicated array type, and any field could contain multiple values. The
|
|
`fields` parameter also does not guarantee that array values are returned in
|
|
a specific order. See the mapping documentation on <<array, arrays>> for more
|
|
background.
|
|
|
|
|
|
|
|
[discrete]
|
|
[[docvalue-fields]]
|
|
=== Doc value fields
|
|
|
|
You can use the <<docvalue-fields,`docvalue_fields`>> parameter to return
|
|
<<doc-values,doc values>> for one or more fields in the search response.
|
|
|
|
Doc values store the same values as the `_source` but in an on-disk,
|
|
column-based structure that's optimized for sorting and aggregations. Since each
|
|
field is stored separately, {es} only reads the field values that were requested
|
|
and can avoid loading the whole document `_source`.
|
|
|
|
Doc values are stored for supported fields by default. However, doc values are
|
|
not supported for <<text,`text`>> or
|
|
{plugins}/mapper-annotated-text-usage.html[`text_annotated`] fields.
|
|
|
|
The following search request uses the `docvalue_fields` parameter to retrieve
|
|
doc values for the `user.id` field, all fields starting with `http.response.`, and the
|
|
`@timestamp` field:
|
|
|
|
[source,console]
|
|
----
|
|
GET my-index-000001/_search
|
|
{
|
|
"query": {
|
|
"match": {
|
|
"user.id": "kimchy"
|
|
}
|
|
},
|
|
"docvalue_fields": [
|
|
"user.id",
|
|
"http.response.*", <1>
|
|
{
|
|
"field": "date",
|
|
"format": "epoch_millis" <2>
|
|
}
|
|
]
|
|
}
|
|
----
|
|
// TEST[setup:my_index]
|
|
|
|
<1> Both full field names and wildcard patterns are accepted.
|
|
<2> Using object notation, you can pass a `format` parameter to apply a custom
|
|
format for the field's doc values. <<date,Date fields>> support a
|
|
<<mapping-date-format,date `format`>>. <<number,Numeric fields>> support a
|
|
https://docs.oracle.com/javase/8/docs/api/java/text/DecimalFormat.html[DecimalFormat
|
|
pattern]. Other field datatypes do not support the `format` parameter.
|
|
|
|
TIP: You cannot use the `docvalue_fields` parameter to retrieve doc values for
|
|
nested objects. If you specify a nested object, the search returns an empty
|
|
array (`[ ]`) for the field. To access nested fields, use the
|
|
<<request-body-search-inner-hits, `inner_hits`>> parameter's `docvalue_fields`
|
|
property.
|
|
|
|
[discrete]
|
|
[[stored-fields]]
|
|
=== Stored fields
|
|
|
|
It's also possible to store an individual field's values by using the
|
|
<<mapping-store,`store`>> mapping option. You can use the
|
|
<<request-body-search-stored-fields, `stored_fields`>> parameter to include
|
|
these stored values in the search response.
|
|
|
|
[discrete]
|
|
[[source-filtering]]
|
|
=== Source filtering
|
|
|
|
You can use the `_source` parameter to select what fields of the source are
|
|
returned. This is called _source filtering_.
|
|
|
|
The following search API request sets the `_source` request body parameter to
|
|
`false`. The document source is not included in the response.
|
|
|
|
[source,console]
|
|
----
|
|
GET /_search
|
|
{
|
|
"_source": false,
|
|
"query": {
|
|
"match": {
|
|
"user.id": "kimchy"
|
|
}
|
|
}
|
|
}
|
|
----
|
|
|
|
To return only a subset of source fields, specify a wildcard (`*`) pattern in
|
|
the `_source` parameter. The following search API request returns the source for
|
|
only the `obj` field and its properties.
|
|
|
|
[source,console]
|
|
----
|
|
GET /_search
|
|
{
|
|
"_source": "obj.*",
|
|
"query": {
|
|
"match": {
|
|
"user.id": "kimchy"
|
|
}
|
|
}
|
|
}
|
|
----
|
|
|
|
You can also specify an array of wildcard patterns in the `_source` field. The
|
|
following search API request returns the source for only the `obj1` and
|
|
`obj2` fields and their properties.
|
|
|
|
[source,console]
|
|
----
|
|
GET /_search
|
|
{
|
|
"_source": [ "obj1.*", "obj2.*" ],
|
|
"query": {
|
|
"match": {
|
|
"user.id": "kimchy"
|
|
}
|
|
}
|
|
}
|
|
----
|
|
|
|
For finer control, you can specify an object containing arrays of `includes` and
|
|
`excludes` patterns in the `_source` parameter.
|
|
|
|
If the `includes` property is specified, only source fields that match one of
|
|
its patterns are returned. You can exclude fields from this subset using the
|
|
`excludes` property.
|
|
|
|
If the `includes` property is not specified, the entire document source is
|
|
returned, excluding any fields that match a pattern in the `excludes` property.
|
|
|
|
The following search API request returns the source for only the `obj1` and
|
|
`obj2` fields and their properties, excluding any child `description` fields.
|
|
|
|
[source,console]
|
|
----
|
|
GET /_search
|
|
{
|
|
"_source": {
|
|
"includes": [ "obj1.*", "obj2.*" ],
|
|
"excludes": [ "*.description" ]
|
|
},
|
|
"query": {
|
|
"term": {
|
|
"user.id": "kimchy"
|
|
}
|
|
}
|
|
}
|
|
----
|