106 lines
3.9 KiB
Plaintext
106 lines
3.9 KiB
Plaintext
|
[[dynamic-index-names]]
|
||
|
|
=== Dynamic Index Names
|
||
|
|
|
||
|
|
Several watch constructs deal with indices, including <<actions-index, `index` action>>,
|
||
|
|
the <<transform-search, `search` transform>> and the <<input-search, `search` input>>.
|
||
|
|
When configuring these constructs you can set the index names to static values. In addition
|
||
|
|
to specifying static index names, Watcher enables you to specify indexes using dynamic
|
||
|
|
time-aware templates. These templates resolve to specific index names during the watch
|
||
|
|
execution according to the execution time.
|
||
|
|
|
||
|
|
Dynamic index name resolution enables you to search a range of time-series indices, rather
|
||
|
|
than searching all of your time-series indices and filtering the the results. Limiting the
|
||
|
|
number of indices that are searched reduces the load on the cluster and improves watch
|
||
|
|
execution performance. For example, if you are using a watch to monitor errors in your
|
||
|
|
daily logs, you can use a dynamic index name template to restrict the search to the past
|
||
|
|
two days.
|
||
|
|
|
||
|
|
A dynamic index name takes the following form:
|
||
|
|
|
||
|
|
[source,txt]
|
||
|
|
----------------------------------------------------------------------
|
||
|
|
<static_name{date_math_expr{date_format}}>
|
||
|
|
----------------------------------------------------------------------
|
||
|
|
|
||
|
|
Where:
|
||
|
|
|
||
|
|
* `static_name` is the static text part of the name
|
||
|
|
* `date_math_expr` is a dynamic date math expression that computes the date dynamically
|
||
|
|
* `date_format` is the format in which the computed date should be rendered
|
||
|
|
|
||
|
|
NOTE: You must enclose dynamic index name templates within angle brackets. For example,
|
||
|
|
`<logstash-{now/d-2d}>`
|
||
|
|
|
||
|
|
The following example shows different forms of dynamic index names and the final index names
|
||
|
|
they resolve to given the execution date is 22rd March 2024.
|
||
|
|
|
||
|
|
[options="header"]
|
||
|
|
|======
|
||
|
|
| Expression |Resolves to
|
||
|
|
| `<logstash-{now/d}>` | `logstash-2024.03.22`
|
||
|
|
| `<logstash-{now/M}>` | `logstash-2024.03.01`
|
||
|
|
| `<logstash-{now/M{YYYY.MM}}>` | `logstash-2024.03`
|
||
|
|
| `<logstash-{now/M-1M{YYYY.MM}}>` | `logstash-2024.02`
|
||
|
|
|======
|
||
|
|
|
||
|
|
To use the characters `{` and `}` in the static part of an index name template, escape them
|
||
|
|
with a backslash, `\`:
|
||
|
|
|
||
|
|
* `<elastic\\{ON\\}-{now/M}>` resolves to `elastic{ON}-2024.03.01`
|
||
|
|
|
||
|
|
The following example shows a search input that searches the Logstash indices for the past
|
||
|
|
three days, assuming the indices use the default Logstash index name format,
|
||
|
|
`logstash-YYYY.MM.dd`.
|
||
|
|
|
||
|
|
[source,json]
|
||
|
|
----------------------------------------------------------------------
|
||
|
|
{
|
||
|
|
...
|
||
|
|
"input" : {
|
||
|
|
"search" : {
|
||
|
|
"request" : {
|
||
|
|
"indices" : [
|
||
|
|
"<logstash-{now/d-2d}>",
|
||
|
|
"<logstash-{now/d-1d}>",
|
||
|
|
"<logstash-{now/d}>"
|
||
|
|
],
|
||
|
|
...
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
...
|
||
|
|
}
|
||
|
|
----------------------------------------------------------------------
|
||
|
|
|
||
|
|
[[dynamic-index-name-timezone]]
|
||
|
|
|
||
|
|
By default, the index names are resolved base on `UTC` time zone. You can change this default at
|
||
|
|
multiple levels:
|
||
|
|
|
||
|
|
Configuring the following setting set the default dynamic index name time zone in watcher:
|
||
|
|
|
||
|
|
[source,yaml]
|
||
|
|
--------------------------------------------------
|
||
|
|
watcher.dynamic_indices.time_zone: '+01:00'
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
You can also configure the default time zone separately on each of the construct that make
|
||
|
|
use of it (`search` input/transform and `index` action):
|
||
|
|
|
||
|
|
[source,yaml]
|
||
|
|
--------------------------------------------------
|
||
|
|
watcher.input.search.dynamic_indices.time_zone: '+01:00'
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
[source,yaml]
|
||
|
|
--------------------------------------------------
|
||
|
|
watcher.transform.search.dynamic_indices.time_zone: '+01:00'
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
[source,yaml]
|
||
|
|
--------------------------------------------------
|
||
|
|
watcher.actions.index.dynamic_indices.time_zone: '+01:00'
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
Alternatively, each of these construct can define their own time zone within the watch
|
||
|
|
definition.
|