2020-07-30 16:45:18 -04:00
|
|
|
[[search-your-data]]
|
|
|
|
= Search your data
|
|
|
|
|
|
|
|
[[search-query]]
|
|
|
|
A _search query_, or _query_, is a request for information about data in
|
|
|
|
{es} data streams or indices.
|
|
|
|
|
|
|
|
You can think of a query as a question, written in a way {es} understands.
|
|
|
|
Depending on your data, you can use a query to get answers to questions like:
|
|
|
|
|
|
|
|
* What processes on my server take longer than 500 milliseconds to respond?
|
|
|
|
* What users on my network ran `regsvr32.exe` within the last week?
|
|
|
|
* How many of my products have a price greater than $20?
|
|
|
|
* What pages on my website contain a specific word or phrase?
|
|
|
|
|
|
|
|
A _search_ consists of one or more queries that are combined and sent to {es}.
|
|
|
|
Documents that match a search's queries are returned in the _hits_, or
|
|
|
|
_search results_, of the response.
|
|
|
|
|
|
|
|
A search may also contain additional information used to better process its
|
|
|
|
queries. For example, a search may be limited to a specific index or only return
|
|
|
|
a specific number of results.
|
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[run-an-es-search]]
|
2020-06-01 14:55:26 -04:00
|
|
|
== Run a search
|
|
|
|
|
|
|
|
You can use the <<search-search,search API>> to search data stored in
|
2020-06-18 08:59:00 -04:00
|
|
|
{es} data streams or indices.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-06-18 08:59:00 -04:00
|
|
|
The API can run two types of searches, depending on how you provide
|
2020-07-30 16:45:18 -04:00
|
|
|
queries:
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
<<run-uri-search,URI searches>>::
|
|
|
|
Queries are provided through a query parameter. URI searches tend to be
|
|
|
|
simpler and best suited for testing.
|
|
|
|
|
|
|
|
<<run-request-body-search,Request body searches>>::
|
|
|
|
Queries are provided through the JSON body of the API request. These queries
|
|
|
|
are written in <<query-dsl,Query DSL>>. We recommend using request body
|
|
|
|
searches in most production use cases.
|
|
|
|
|
|
|
|
[WARNING]
|
|
|
|
====
|
|
|
|
If you specify a query in both the URI and request body, the search API request
|
|
|
|
runs only the URI query.
|
|
|
|
====
|
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[run-uri-search]]
|
|
|
|
=== Run a URI search
|
|
|
|
|
|
|
|
You can use the search API's <<search-api-query-params-q,`q` query string
|
|
|
|
parameter>> to run a search in the request's URI. The `q` parameter only accepts
|
|
|
|
queries written in Lucene's <<query-string-syntax,query string syntax>>.
|
|
|
|
|
2020-07-21 16:14:44 -04:00
|
|
|
To get started, ingest or add some data to an {es} data stream or index.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-07-21 16:14:44 -04:00
|
|
|
The following <<docs-bulk,bulk API>> request adds some example server access log
|
|
|
|
data to the `my-index-000001` index.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
PUT /my-index-000001/_bulk?refresh
|
|
|
|
{ "index":{ } }
|
|
|
|
{ "@timestamp": "2099-11-15T14:12:12", "http": { "request": { "method": "get" }, "response": { "bytes": 1070000, "status_code": 200 }, "version": "1.1" }, "message": "GET /search HTTP/1.1 200 1070000", "source": { "ip": "127.0.0.1" }, "user": { "id": "kimchy" } }
|
|
|
|
{ "index":{ } }
|
|
|
|
{ "@timestamp": "2099-11-15T14:12:12", "http": { "request": { "method": "get" }, "response": { "bytes": 1070000, "status_code": 200 }, "version": "1.1" }, "message": "GET /search HTTP/1.1 200 1070000", "source": { "ip": "10.42.42.42" }, "user": { "id": "elkbee" } }
|
|
|
|
{ "index":{ } }
|
|
|
|
{ "@timestamp": "2099-11-15T14:12:12", "http": { "request": { "method": "get" }, "response": { "bytes": 1070000, "status_code": 200 }, "version": "1.1" }, "message": "GET /search HTTP/1.1 200 1070000", "source": { "ip": "10.42.42.42" }, "user": { "id": "elkbee" } }
|
2020-06-01 14:55:26 -04:00
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
// TESTSETUP
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
You can now use the search API to run a URI search on this index.
|
|
|
|
|
2020-07-21 16:14:44 -04:00
|
|
|
The following URI search matches documents with a `user.id` value of `kimchy`.
|
2020-06-01 14:55:26 -04:00
|
|
|
Note the query is specified using the `q` query string parameter.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
GET /my-index-000001/_search?q=user.id:kimchy
|
2020-06-01 14:55:26 -04:00
|
|
|
----
|
|
|
|
|
|
|
|
The API returns the following response. Note the `hits.hits` property contains
|
|
|
|
the document that matched the query.
|
|
|
|
|
|
|
|
[source,console-result]
|
|
|
|
----
|
|
|
|
{
|
2020-07-21 16:14:44 -04:00
|
|
|
"took": 5,
|
2020-06-01 14:55:26 -04:00
|
|
|
"timed_out": false,
|
|
|
|
"_shards": {
|
|
|
|
"total": 1,
|
|
|
|
"successful": 1,
|
|
|
|
"skipped": 0,
|
|
|
|
"failed": 0
|
|
|
|
},
|
|
|
|
"hits": {
|
|
|
|
"total": {
|
|
|
|
"value": 1,
|
|
|
|
"relation": "eq"
|
|
|
|
},
|
|
|
|
"max_score": 0.9808291,
|
|
|
|
"hits": [
|
|
|
|
{
|
2020-07-21 16:14:44 -04:00
|
|
|
"_index": "my-index-000001",
|
2020-06-01 14:55:26 -04:00
|
|
|
"_type": "_doc",
|
2020-07-21 16:14:44 -04:00
|
|
|
"_id": "kxWFcnMByiguvud1Z8vC",
|
2020-06-01 14:55:26 -04:00
|
|
|
"_score": 0.9808291,
|
|
|
|
"_source": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"@timestamp": "2099-11-15T14:12:12",
|
|
|
|
"http": {
|
|
|
|
"request": {
|
|
|
|
"method": "get"
|
|
|
|
},
|
|
|
|
"response": {
|
|
|
|
"bytes": 1070000,
|
|
|
|
"status_code": 200
|
|
|
|
},
|
|
|
|
"version": "1.1"
|
|
|
|
},
|
|
|
|
"message": "GET /search HTTP/1.1 200 1070000",
|
|
|
|
"source": {
|
|
|
|
"ip": "127.0.0.1"
|
2020-06-01 14:55:26 -04:00
|
|
|
},
|
2020-07-21 16:14:44 -04:00
|
|
|
"user": {
|
|
|
|
"id": "kimchy"
|
|
|
|
}
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
]
|
|
|
|
}
|
|
|
|
}
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
// TESTRESPONSE[s/"took": 5/"took": "$body.took"/]
|
|
|
|
// TESTRESPONSE[s/"_id": "kxWFcnMByiguvud1Z8vC"/"_id": "$body.hits.hits.0._id"/]
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[run-request-body-search]]
|
|
|
|
=== Run a request body search
|
|
|
|
|
|
|
|
You can use the search API's <<request-body-search-query,`query` request
|
|
|
|
body parameter>> to provide a query as a JSON object, written in
|
|
|
|
<<query-dsl,Query DSL>>.
|
|
|
|
|
|
|
|
The following request body search uses the <<query-dsl-match-query,`match`>>
|
2020-07-21 16:14:44 -04:00
|
|
|
query to match documents with a `user.id` value of `kimchy`. Note the
|
2020-06-01 14:55:26 -04:00
|
|
|
`match` query is specified as a JSON object in the `query` parameter.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
GET /my-index-000001/_search
|
2020-06-01 14:55:26 -04:00
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"match": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"user.id": "kimchy"
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
----
|
|
|
|
|
|
|
|
The API returns the following response.
|
|
|
|
|
|
|
|
The `hits.hits` property contains matching documents. By default, the response
|
|
|
|
sorts these matching documents by `_score`, a <<relevance-scores,relevance
|
|
|
|
score>> that measures how well each document matches the query.
|
|
|
|
|
|
|
|
[source,console-result]
|
|
|
|
----
|
|
|
|
{
|
2020-07-21 16:14:44 -04:00
|
|
|
"took": 5,
|
2020-06-01 14:55:26 -04:00
|
|
|
"timed_out": false,
|
|
|
|
"_shards": {
|
|
|
|
"total": 1,
|
|
|
|
"successful": 1,
|
|
|
|
"skipped": 0,
|
|
|
|
"failed": 0
|
|
|
|
},
|
|
|
|
"hits": {
|
|
|
|
"total": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"value": 1,
|
2020-06-01 14:55:26 -04:00
|
|
|
"relation": "eq"
|
|
|
|
},
|
2020-07-21 16:14:44 -04:00
|
|
|
"max_score": 0.9808291,
|
2020-06-01 14:55:26 -04:00
|
|
|
"hits": [
|
|
|
|
{
|
2020-07-21 16:14:44 -04:00
|
|
|
"_index": "my-index-000001",
|
2020-06-01 14:55:26 -04:00
|
|
|
"_type": "_doc",
|
2020-07-21 16:14:44 -04:00
|
|
|
"_id": "kxWFcnMByiguvud1Z8vC",
|
|
|
|
"_score": 0.9808291,
|
2020-06-01 14:55:26 -04:00
|
|
|
"_source": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"@timestamp": "2099-11-15T14:12:12",
|
|
|
|
"http": {
|
|
|
|
"request": {
|
|
|
|
"method": "get"
|
|
|
|
},
|
|
|
|
"response": {
|
|
|
|
"bytes": 1070000,
|
|
|
|
"status_code": 200
|
|
|
|
},
|
|
|
|
"version": "1.1"
|
2020-06-01 14:55:26 -04:00
|
|
|
},
|
2020-07-21 16:14:44 -04:00
|
|
|
"message": "GET /search HTTP/1.1 200 1070000",
|
|
|
|
"source": {
|
|
|
|
"ip": "127.0.0.1"
|
2020-06-01 14:55:26 -04:00
|
|
|
},
|
|
|
|
"user": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"id": "kimchy"
|
|
|
|
}
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
]
|
|
|
|
}
|
|
|
|
}
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
// TESTRESPONSE[s/"took": 5/"took": "$body.took"/]
|
|
|
|
// TESTRESPONSE[s/"_id": "kxWFcnMByiguvud1Z8vC"/"_id": "$body.hits.hits.0._id"/]
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
[discrete]
|
|
|
|
[[search-multiple-indices]]
|
2020-06-18 08:59:00 -04:00
|
|
|
=== Search multiple data streams and indices
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-06-18 08:59:00 -04:00
|
|
|
To search multiple data streams and indices, add them as comma-separated values
|
|
|
|
in the search API request path.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-07-21 16:14:44 -04:00
|
|
|
The following request searches the `my-index-000001` and `my-index-000002`
|
2020-06-01 14:55:26 -04:00
|
|
|
indices.
|
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
GET /my-index-000001,my-index-000002/_search
|
2020-06-01 14:55:26 -04:00
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"match": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"user.id": "kimchy"
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
----
|
2020-07-21 16:14:44 -04:00
|
|
|
// TEST[s/^/PUT my-index-000002\n/]
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-06-18 08:59:00 -04:00
|
|
|
You can also search multiple data streams and indices using a wildcard (`*`)
|
|
|
|
pattern.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-06-18 08:59:00 -04:00
|
|
|
The following request targets the wildcard pattern `user_logs*`. The request
|
|
|
|
searches any data streams or indices in the cluster that start with `user_logs`.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET /user_logs*/_search
|
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"match": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"user.id": "kimchy"
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
----
|
|
|
|
|
2020-06-18 08:59:00 -04:00
|
|
|
To search all data streams and indices in a cluster, omit the target from the
|
|
|
|
request path. Alternatively, you can use `_all` or `*`.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
2020-06-18 08:59:00 -04:00
|
|
|
The following requests are equivalent and search all data streams and indices in the cluster.
|
2020-06-01 14:55:26 -04:00
|
|
|
|
|
|
|
[source,console]
|
|
|
|
----
|
|
|
|
GET /_search
|
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"match": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"user.id": "kimchy"
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
GET /_all/_search
|
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"match": {
|
2020-07-21 16:14:44 -04:00
|
|
|
"user.id": "kimchy"
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
GET /*/_search
|
|
|
|
{
|
2020-07-21 16:14:44 -04:00
|
|
|
"query": {
|
|
|
|
"match": {
|
|
|
|
"user.id": "kimchy"
|
|
|
|
}
|
2020-07-21 15:49:58 -04:00
|
|
|
}
|
2020-06-01 14:55:26 -04:00
|
|
|
}
|
|
|
|
----
|
2020-06-01 16:43:06 -04:00
|
|
|
|
|
|
|
include::request/from-size.asciidoc[]
|
2020-06-03 12:33:26 -04:00
|
|
|
include::search-fields.asciidoc[]
|
2020-07-30 09:19:05 -04:00
|
|
|
include::request/collapse.asciidoc[]
|
2020-07-17 10:57:00 -04:00
|
|
|
include::request/highlighting.asciidoc[]
|
2020-07-23 13:44:47 -04:00
|
|
|
include::request/sort.asciidoc[]
|
2020-07-30 16:45:18 -04:00
|
|
|
include::{es-repo-dir}/async-search.asciidoc[]
|
|
|
|
include::{es-repo-dir}/modules/cross-cluster-search.asciidoc[]
|
|
|
|
include::{es-repo-dir}/search/near-real-time.asciidoc[]
|