mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-26 06:46:10 +00:00
98c91a3bd0
This is a protection mechanism to prevent a single search request from hitting a large number of shards in the cluster concurrently. If a search is executed against all indices in the cluster this can easily overload the cluster causing rejections etc. which is not necessarily desirable. Instead this PR adds a per request limit of `max_concurrent_shard_requests` that throttles the number of concurrent initial phase requests to `256` by default. This limit can be increased per request and protects single search requests from overloading the cluster. Subsequent PRs can introduces addiontional improvemetns ie. limiting this on a `_msearch` level, making defaults a factor of the number of nodes or sort shards iters such that we gain the best concurrency across nodes.
76 lines
2.8 KiB
Plaintext
76 lines
2.8 KiB
Plaintext
[[search-search]]
|
|
== Search
|
|
|
|
The search API allows you to execute a search query and get back search hits
|
|
that match the query. The query can either be provided using a simple
|
|
<<search-uri-request,query string as a parameter>>, or using a
|
|
<<search-request-body,request body>>.
|
|
|
|
["float",id="search-multi-index-type"]
|
|
=== Multi-Index, Multi-Type
|
|
|
|
All search APIs can be applied across multiple types within an index, and
|
|
across multiple indices with support for the
|
|
<<multi-index,multi index syntax>>. For
|
|
example, we can search on all documents across all types within the
|
|
twitter index:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /twitter/_search?q=user:kimchy
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:twitter]
|
|
|
|
We can also search within specific types:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /twitter/tweet,user/_search?q=user:kimchy
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:twitter]
|
|
|
|
We can also search all tweets with a certain tag across several indices
|
|
(for example, when each user has his own index):
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /kimchy,elasticsearch/tweet/_search?q=tag:wow
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[s/^/PUT kimchy\nPUT elasticsearch\n/]
|
|
|
|
Or we can search all tweets across all available indices using `_all`
|
|
placeholder:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_all/tweet/_search?q=tag:wow
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:twitter]
|
|
|
|
Or even search across all indices and all types:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_search?q=tag:wow
|
|
--------------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:twitter]
|
|
|
|
By default elasticsearch doesn't reject any search requests based on the number
|
|
of shards the request hits. While elasticsearch will optimize the search execution
|
|
on the coordinating node a large number of shards can have a significant impact
|
|
CPU and memory wise. It is usually a better idea to organize data in such a way
|
|
that there are fewer larger shards. In case you would like to configure a soft
|
|
limit, you can update the `action.search.shard_count.limit` cluster setting in order
|
|
to reject search requests that hit too many shards.
|
|
|
|
The search's `max_concurrent_shard_requests` request parameter can be used to control
|
|
the maximum number of concurrent shard requests the search API will execute for this request.
|
|
This parameter should be used to protect a singe request from overloading a cluster ie. a default
|
|
request will hit all indices in a cluster which could cause shard request rejections if the
|
|
number of shards per node is high. This default is based on the number of data nodes in
|
|
the cluster but at most `256`. |