mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-03-25 09:28:27 +00:00
[DOCS] Add throttling based on configuration parameter (#56653)
This commit is contained in:
parent
fb5e6329b7
commit
6a8e10189f
@ -940,15 +940,13 @@ default value is `60s`.
|
||||
end::sync-time-delay[]
|
||||
|
||||
tag::transform-settings[]
|
||||
Defines the settings of the {transform}.
|
||||
Defines optional {transform} settings.
|
||||
end::transform-settings[]
|
||||
|
||||
tag::transform-settings-docs-per-second[]
|
||||
Set `docs-per-second` to throttle the rate transform is issuing search and
|
||||
index requests. Throttling slows down search requests by adding a wait time
|
||||
between searches. For the number of docs, the number of input documents is taken.
|
||||
Throttling can be disabled by setting it to `null`.
|
||||
The default is no throttling, value `null`.
|
||||
Specifies a limit on the number of input documents per second. This setting
|
||||
throttles the {transform} by adding a wait time between search requests. The
|
||||
default value is `null`, which disables throttling.
|
||||
end::transform-settings-docs-per-second[]
|
||||
|
||||
tag::transform-settings-max-page-search-size[]
|
||||
|
@ -128,6 +128,23 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by]
|
||||
====
|
||||
//End pivot
|
||||
|
||||
//Begin settings
|
||||
`settings`::
|
||||
(Optional, object)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
|
||||
+
|
||||
.Properties of `settings`
|
||||
[%collapsible%open]
|
||||
====
|
||||
`docs_per_second`:::
|
||||
(Optional, float)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
|
||||
`max_page_search_size`:::
|
||||
(Optional, integer)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
|
||||
====
|
||||
//End settings
|
||||
|
||||
//Begin source
|
||||
`source`::
|
||||
(Required, object)
|
||||
@ -185,23 +202,6 @@ delays.
|
||||
====
|
||||
//End sync
|
||||
|
||||
//Begin settings
|
||||
`settings`::
|
||||
(Optional, object)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
|
||||
+
|
||||
.Properties of `settings`
|
||||
[%collapsible%open]
|
||||
====
|
||||
`docs_per_second`:::
|
||||
(Optional, float)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
|
||||
`max_page_search_size`:::
|
||||
(Optional, integer)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
|
||||
====
|
||||
//End settings
|
||||
|
||||
|
||||
[[put-transform-example]]
|
||||
==== {api-examples-title}
|
||||
|
@ -102,6 +102,23 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline]
|
||||
(Optional, <<time-units, time units>>)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency]
|
||||
|
||||
//Begin settings
|
||||
`settings`::
|
||||
(Optional, object)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
|
||||
+
|
||||
.Properties of `settings`
|
||||
[%collapsible%open]
|
||||
====
|
||||
`docs_per_second`:::
|
||||
(Optional, float)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
|
||||
`max_page_search_size`:::
|
||||
(Optional, integer)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
|
||||
====
|
||||
//End settings
|
||||
|
||||
//Begin source
|
||||
`source`::
|
||||
(Optional, object)
|
||||
@ -159,23 +176,6 @@ delays.
|
||||
====
|
||||
//End sync
|
||||
|
||||
//Begin settings
|
||||
`settings`::
|
||||
(Optional, object)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
|
||||
+
|
||||
.Properties of `settings`
|
||||
[%collapsible%open]
|
||||
====
|
||||
`docs_per_second`:::
|
||||
(Optional, float)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
|
||||
`max_page_search_size`:::
|
||||
(Optional, integer)
|
||||
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
|
||||
====
|
||||
//End settings
|
||||
|
||||
[[update-transform-example]]
|
||||
==== {api-examples-title}
|
||||
|
||||
|
@ -49,11 +49,7 @@ requires enough resources to support both the composite aggregation search and
|
||||
the indexing of its results.
|
||||
|
||||
TIP: If the cluster experiences unsuitable performance degradation due to the
|
||||
{transform}, stop the {transform}. Consider whether you can apply a
|
||||
source query to the {transform} to reduce the scope of data it
|
||||
processes. Also consider whether the cluster has sufficient resources in place
|
||||
to support both the composite aggregation search and the indexing of its
|
||||
results.
|
||||
{transform}, stop the {transform} and refer to <<transform-performance>>.
|
||||
|
||||
[discrete]
|
||||
[[ml-transform-checkpoint-errors]]
|
||||
|
@ -72,4 +72,19 @@ If your {transform} must process a lot of historic data, it has high resource
|
||||
usage initially--particularly during the first checkpoint.
|
||||
|
||||
For better performance, make sure that your search aggregations and queries are
|
||||
optimized and that your {transform} is processing only necessary data.
|
||||
optimized and that your {transform} is processing only necessary data. Consider
|
||||
whether you can apply a source query to the {transform} to reduce the scope of
|
||||
data it processes. Also consider whether the cluster has sufficient resources in
|
||||
place to support both the composite aggregation search and the indexing of its
|
||||
results.
|
||||
|
||||
If you prefer to spread out the impact on your cluster (at the cost of a slower
|
||||
{transform}), you can throttle the rate at which it performs search and index
|
||||
requests. Set the `docs_per_second` limit when you <<put-transform,create>> or
|
||||
<<update-transform,update>> your {transform}. If you want to calculate the
|
||||
current rate, use the following information from the
|
||||
{ref}/get-transform-stats.html[get {transform} stats API]:
|
||||
```
|
||||
documents_processed / search_time_in_ms * 1000
|
||||
```
|
||||
|
||||
|
Loading…
x
Reference in New Issue
Block a user