honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[DOCS] Tighten async EQL copy (#64106) (#64108)

This commit is contained in:
James Rodewig 2020-10-24 14:28:43 -04:00 committed by GitHub
parent d6143bb49d
commit f19f170811
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 32 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

105
docs/reference/eql/eql.asciidoc
View File

@ -437,39 +437,14 @@ GET /my-index-000001/_eql/search
[[eql-search-async]] [[eql-search-async]]
=== Run an async EQL search === Run an async EQL search
EQL searches are designed to run on large volumes of data quickly, often By default, EQL search requests are synchronous and wait for complete results
returning results in milliseconds. For this reason, EQL searches are before returning a response. However, complete results can take longer for
_synchronous_ by default. The search request waits for complete results before searches across <<frozen-indices,frozen indices>> or
returning a response. <<modules-cross-cluster-search,multiple clusters>>.
However, complete results can take longer for searches across: To avoid long waits, run an async EQL search. Set the
`wait_for_completion_timeout` parameter to a duration you'd like to wait for
* <<frozen-indices,Frozen indices>> synchronous results.
* <<modules-cross-cluster-search,Multiple clusters>>
* Many shards
To avoid long waits, use the `wait_for_completion_timeout` parameter to run an
_asynchronous_, or _async_, EQL search.
Set `wait_for_completion_timeout` to a duration you'd like to wait
for complete search results. If the search request does not finish within this
period, the search becomes async and returns a response that includes:
* A search ID, which can be used to monitor the progress of the async search.
* An `is_partial` value of `true`, meaning the response does not contain
complete search results.
* An `is_running` value of `true`, meaning the search is async and ongoing.
The async search continues to run in the background without blocking
other requests.
The following request searches the `frozen-my-index-000001` index, which has been
<<frozen-indices,frozen>> for storage and is rarely searched.
Because searches on frozen indices are expected to take longer to complete, the
request contains a `wait_for_completion_timeout` parameter value of `2s` (two
seconds). If the request does not return complete results in two seconds, the
search becomes async and returns a search ID.
[source,console] [source,console]
---- ----
@ -484,8 +459,16 @@ GET /frozen-my-index-000001/_eql/search
// TEST[setup:sec_logs] // TEST[setup:sec_logs]
// TEST[s/frozen-my-index-000001/my-index-000001/] // TEST[s/frozen-my-index-000001/my-index-000001/]
After two seconds, the request returns the following response. Note `is_partial` If the request doesn't finish within the timeout period, the search becomes async
and `is_running` properties are `true`, indicating an async search. and returns a response that includes:
* A search ID
* An `is_partial` value of `true`, indicating the search results are
incomplete
* An `is_running` value of `true`, indicating the search is ongoing
The async search continues to run in the background without blocking other
requests.
[source,console-result] [source,console-result]
---- ----
@ -504,16 +487,9 @@ and `is_running` properties are `true`, indicating an async search.
// TESTRESPONSE[s/"took": 2000/"took": $body.took/] // TESTRESPONSE[s/"took": 2000/"took": $body.took/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/] // TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
Use the <<get-async-eql-search-api,get async EQL search API>> to check the progress To check the progress of an async search, use the <<get-async-eql-search-api,get
of an async search. async EQL search API>> with the search ID. Specify how long you'd like for
complete results in the `wait_for_completion_timeout` parameter.
The get async EQL search API also accepts a `wait_for_completion_timeout`
parameter. If an ongoing search does not complete during this period, the response
returns an `is_partial` value of `true` and no search results.
The following get async EQL search API request checks the progress of the
previous async EQL search. The request specifies a `wait_for_completion_timeout`
query parameter value of `2s` (two seconds).
[source,console] [source,console]
---- ----
@ -521,9 +497,9 @@ GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTo
---- ----
// TEST[skip: no access to search ID] // TEST[skip: no access to search ID]
The request returns the following response. Note `is_partial` and `is_running` If the response's `is_running` value is `false`, the async search has finished.
are `false`, indicating the async search has finished and the search results If the `is_partial` value is `false`, the returned search results are
in the `hits` property are complete. complete.
[source,console-result] [source,console-result]
---- ----
@ -546,12 +522,7 @@ in the `hits` property are complete.
By default, the EQL search API stores async searches for five days. After this By default, the EQL search API stores async searches for five days. After this
period, any searches and their results are deleted. Use the `keep_alive` period, any searches and their results are deleted. Use the `keep_alive`
parameter to change this retention period. parameter to change this retention period:
In the following EQL search request, the `keep_alive` parameter is `2d` (two
days). If the search becomes async, its results
are stored on the cluster for two days. After two days, the async
search and its results are deleted, even if it's still ongoing.
[source,console] [source,console]
---- ----
@ -570,10 +541,6 @@ You can use the <<get-async-eql-search-api,get async EQL search API>>'s
`keep_alive` parameter to later change the retention period. The new retention `keep_alive` parameter to later change the retention period. The new retention
period starts after the get request runs. period starts after the get request runs.
The following request sets the `keep_alive` query parameter to `5d` (five days).
The async search and its results are deleted five days after the get request
executes.
[source,console] [source,console]
---- ----
GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d
@ -584,8 +551,6 @@ Use the <<delete-async-eql-search-api,delete async EQL search API>> to
manually delete an async EQL search before the `keep_alive` period ends. If the manually delete an async EQL search before the `keep_alive` period ends. If the
search is still ongoing, {es} cancels the search request. search is still ongoing, {es} cancels the search request.
The following request deletes an async EQL search and its results.
[source,console] [source,console]
---- ----
DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d
@ -599,12 +564,8 @@ DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERn
By default, the EQL search API only stores async searches that cannot be By default, the EQL search API only stores async searches that cannot be
completed within the period set by `wait_for_completion_timeout`. completed within the period set by `wait_for_completion_timeout`.
To save the results of searches that complete during this period, set the To save the results for a synchronous search, set `keep_on_completion` to
`keep_on_completion` parameter to `true`. `true`:
In the following search request, `keep_on_completion` is `true`. Search results
are stored on the cluster, even if the search completes within the `2s`
(two-second) period set by the `wait_for_completion_timeout` parameter.
[source,console] [source,console]
---- ----
@ -619,9 +580,8 @@ GET /my-index-000001/_eql/search
---- ----
// TEST[setup:sec_logs] // TEST[setup:sec_logs]
The API returns the following response. A search ID is provided in the `id` The response includes a search ID. `is_partial` and `is_running` are `false`,
property. `is_partial` and `is_running` are `false`, indicating the EQL search indicating the EQL search was synchronous and returned complete results.
was synchronous and returned complete results in `hits`.
[source,console-result] [source,console-result]
---- ----
@ -638,8 +598,8 @@ was synchronous and returned complete results in `hits`.
// TESTRESPONSE[s/"took": 52/"took": $body.took/] // TESTRESPONSE[s/"took": 52/"took": $body.took/]
// TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/] // TESTRESPONSE[s/"hits": \.\.\./"hits": $body.hits/]
Use the search ID and the <<get-async-eql-search-api,get async EQL search API>> Use the <<get-async-eql-search-api,get async EQL search API>> to get the
to retrieve the same results later. same results later:
[source,console] [source,console]
---- ----
@ -647,9 +607,8 @@ GET /_eql/search/FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUTo
---- ----
// TEST[skip: no access to search ID] // TEST[skip: no access to search ID]
Saved synchronous searches are still subject to the retention period set by the Saved synchronous searches are still subject to the `keep_alive` parameter's
`keep_alive` parameter. After this period, the search and its results are retention period. When this period ends, the search and its results are deleted.
deleted.
You can also manually delete saved synchronous searches using the You can also manually delete saved synchronous searches using the
<<delete-async-eql-search-api,delete async EQL search API>>. <<delete-async-eql-search-api,delete async EQL search API>>.