OpenSearch/docs/reference/search/count.asciidoc

110 lines
3.2 KiB
Plaintext
Raw Normal View History

[[search-count]]
== Count API
The count API allows to easily execute a query and get the number of
matches for that query. It can be executed across one or more indices
and across one or more types. The query can either be provided using a
simple query string as a parameter, or using the
<<query-dsl,Query DSL>> defined within the request
body. Here is an example:
[source,js]
--------------------------------------------------
2016-05-18 07:36:19 -04:00
PUT /twitter/tweet/1?refresh
{
"user": "kimchy"
}
2016-05-18 07:36:19 -04:00
GET /twitter/tweet/_count?q=user:kimchy
GET /twitter/tweet/_count
{
"query" : {
"term" : { "user" : "kimchy" }
}
2016-05-18 07:36:19 -04:00
}
--------------------------------------------------
2016-05-18 07:36:19 -04:00
//CONSOLE
NOTE: The query being sent in the body must be nested in a `query` key, same as
the <<search-search,search api>> works
Both examples above do the same thing, which is count the number of
tweets from the twitter index for a certain user. The result is:
[source,js]
--------------------------------------------------
{
"count" : 1,
"_shards" : {
"total" : 5,
"successful" : 5,
Add a shard filter search phase to pre-filter shards based on query rewriting (#25658) Today if we search across a large amount of shards we hit every shard. Yet, it's quite common to search across an index pattern for time based indices but filtering will exclude all results outside a certain time range ie. `now-3d`. While the search can potentially hit hundreds of shards the majority of the shards might yield 0 results since there is not document that is within this date range. Kibana for instance does this regularly but used `_field_stats` to optimize the indexes they need to query. Now with the deprecation of `_field_stats` and it's upcoming removal a single dashboard in kibana can potentially turn into searches hitting hundreds or thousands of shards and that can easily cause search rejections even though the most of the requests are very likely super cheap and only need a query rewriting to early terminate with 0 results. This change adds a pre-filter phase for searches that can, if the number of shards are higher than a the `pre_filter_shard_size` threshold (defaults to 128 shards), fan out to the shards and check if the query can potentially match any documents at all. While false positives are possible, a negative response means that no matches are possible. These requests are not subject to rejection and can greatly reduce the number of shards a request needs to hit. The approach here is preferable to the kibana approach with field stats since it correctly handles aliases and uses the correct threadpools to execute these requests. Further it's completely transparent to the user and improves scalability of elasticsearch in general on large clusters.
2017-07-12 16:19:20 -04:00
"skipped" : 0,
"failed" : 0
}
}
--------------------------------------------------
2016-05-18 07:36:19 -04:00
// TESTRESPONSE
The query is optional, and when not provided, it will use `match_all` to
count all the docs.
[float]
=== Multi index, Multi type
The count API can be applied to <<search-multi-index-type,multiple types in multiple indices>>.
[float]
=== Request Parameters
When executing count using the query parameter `q`, the query passed is
a query string using Lucene query parser. There are additional
parameters that can be passed:
[cols="<,<",options="header",]
|=======================================================================
|Name |Description
|`df` |The default field to use when no field prefix is defined within the
query.
|`analyzer` |The analyzer name to be used when analyzing the query string.
|`default_operator` |The default operator to be used, can be `AND` or
`OR`. Defaults to `OR`.
|`lenient` |If set to true will cause format based failures (like
providing text to a numeric field) to be ignored. Defaults to false.
|`analyze_wildcard` |Should wildcard and prefix queries be analyzed or
not. Defaults to `false`.
|`terminate_after` |The maximum count for each shard, upon
reaching which the query execution will terminate early.
If set, the response will have a boolean field `terminated_early` to
indicate whether the query execution has actually terminated_early.
Defaults to no terminate_after.
|=======================================================================
[float]
=== Request Body
The count can use the <<query-dsl,Query DSL>> within
its body in order to express the query that should be executed. The body
content can also be passed as a REST parameter named `source`.
Both HTTP GET and HTTP POST can be used to execute count with body.
Since not all clients support GET with body, POST is allowed as well.
[float]
=== Distributed
The count operation is broadcast across all shards. For each shard id
group, a replica is chosen and executed against it. This means that
replicas increase the scalability of count.
[float]
=== Routing
The routing value (a comma separated list of the routing values) can be
specified to control which shards the count request will be executed on.