Rename examples in ILM guide to avoid association with data streams (#54579)
This commit is contained in:
Dan Hermann
GitHub
parent
27f88fcdac
commit
11bfbd8bbf
|
|
@ -8,11 +8,11 @@
|
|||
<titleabbrev>Automate rollover</titleabbrev>
|
||||
++++
|
||||
|
||||
This tutorial demonstrates how to use {ilm} ({ilm-init})
|
||||
to manage indices that contain time-series data.
|
||||
This tutorial demonstrates how to use {ilm} ({ilm-init})
|
||||
to manage indices that contain time-series data.
|
||||
|
||||
When you continuously index timestamped documents into {es} using
|
||||
Filebeat, Logstash, or some other mechanism,
|
||||
When you continuously index timestamped documents into {es} using
|
||||
Filebeat, Logstash, or some other mechanism,
|
||||
you typically use an index alias so you can periodically roll over to a new index.
|
||||
This enables you to implement a hot-warm-cold architecture to meet your performance
|
||||
requirements for your newest data, control costs over time, enforce retention policies,
|
||||
|
|
@ -20,11 +20,11 @@ and still get the most out of your data.
|
|||
|
||||
To automate rollover and management of time-series indices with {ilm-init}, you:
|
||||
|
||||
. <<ilm-gs-create-policy, Create a lifecycle policy>> with the {ilm-init} put policy API.
|
||||
. <<ilm-gs-create-policy, Create a lifecycle policy>> with the {ilm-init} put policy API.
|
||||
. <<ilm-gs-apply-policy, Create an index template>> to apply the policy to each new index.
|
||||
. <<ilm-gs-bootstrap, Bootstrap an index>> as the initial write index.
|
||||
. <<ilm-gs-check-progress, Verify indices are moving through the lifecycle phases>>
|
||||
as expected with the {ilm-init} explain API.
|
||||
. <<ilm-gs-check-progress, Verify indices are moving through the lifecycle phases>>
|
||||
as expected with the {ilm-init} explain API.
|
||||
|
||||
[float]
|
||||
[[ilm-gs-create-policy]]
|
||||
|
|
@ -32,25 +32,25 @@ as expected with the {ilm-init} explain API.
|
|||
|
||||
A lifecycle policy specifies the phases in the index lifecycle
|
||||
and the actions to perform in each phase. A lifecycle can have up to four phases:
|
||||
`hot`, `warm`, `cold`, and `delete`. Policies are defined in JSON
|
||||
and added through the {ilm-init} put policy API.
|
||||
`hot`, `warm`, `cold`, and `delete`. Policies are defined in JSON
|
||||
and added through the {ilm-init} put policy API.
|
||||
|
||||
For example, the following request creates a `datastream_policy` with two phases:
|
||||
|
||||
* The `hot` phase defines a `rollover` action to specify that an index rolls over when it
|
||||
For example, the following request creates a `timeseries_policy` with two phases:
|
||||
|
||||
* The `hot` phase defines a `rollover` action to specify that an index rolls over when it
|
||||
reaches either a `max_size` of 50 gigabytes or a `max_age` of 30 days.
|
||||
* The `delete` phase uses `min_age` to remove the index 90 days after rollover.
|
||||
Note that this value is relative to the rollover time, not the index creation time.
|
||||
Note that this value is relative to the rollover time, not the index creation time.
|
||||
|
||||
[source,console]
|
||||
------------------------
|
||||
PUT _ilm/policy/datastream_policy
|
||||
PUT _ilm/policy/timeseries_policy
|
||||
{
|
||||
"policy": {
|
||||
"policy": {
|
||||
"phases": {
|
||||
"hot": { <1>
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"rollover": {
|
||||
"max_size": "50GB", <2>
|
||||
"max_age": "30d"
|
||||
}
|
||||
|
|
@ -66,7 +66,7 @@ PUT _ilm/policy/datastream_policy
|
|||
}
|
||||
}
|
||||
------------------------
|
||||
<1> The `min_age` defaults to `0ms`, so new indices enter the `hot` phase immediately.
|
||||
<1> The `min_age` defaults to `0ms`, so new indices enter the `hot` phase immediately.
|
||||
<2> Trigger the `rollover` action when either of the conditions are met.
|
||||
<3> Move the index into the `delete` phase 90 days after rollover.
|
||||
<4> Trigger the `delete` action when the index enters the delete phase.
|
||||
|
|
@ -77,43 +77,43 @@ See <<_actions>> for the complete list of actions available in each phase.
|
|||
[[ilm-gs-apply-policy]]
|
||||
=== Create an index template to apply the lifecycle policy
|
||||
|
||||
To automaticaly apply a lifecycle policy to the new write index on rollover,
|
||||
To automaticaly apply a lifecycle policy to the new write index on rollover,
|
||||
specify the policy in the index template used to create new indices.
|
||||
|
||||
For example, the following request creates a `datastream_template` that is applied to new indices
|
||||
whose names match the `datastream-*` index pattern.
|
||||
For example, the following request creates a `timeseries_template` that is applied to new indices
|
||||
whose names match the `timeseries-*` index pattern.
|
||||
The template configures two {ilm-init} settings:
|
||||
|
||||
* `index.lifecycle.name` specifies the name of the lifecycle policy to apply to all new indices that match
|
||||
the index pattern.
|
||||
* `index.lifecycle.rollover_alias` specifies the index alias to be rolled over
|
||||
* `index.lifecycle.name` specifies the name of the lifecycle policy to apply to all new indices that match
|
||||
the index pattern.
|
||||
* `index.lifecycle.rollover_alias` specifies the index alias to be rolled over
|
||||
when the rollover action is triggered for an index.
|
||||
|
||||
[source,console]
|
||||
-----------------------
|
||||
PUT _template/datastream_template
|
||||
PUT _template/timeseries_template
|
||||
{
|
||||
"index_patterns": ["datastream-*"], <1>
|
||||
"index_patterns": ["timeseries-*"], <1>
|
||||
"settings": {
|
||||
"number_of_shards": 1,
|
||||
"number_of_replicas": 1,
|
||||
"index.lifecycle.name": "datastream_policy", <2>
|
||||
"index.lifecycle.rollover_alias": "datastream" <3>
|
||||
"index.lifecycle.name": "timeseries_policy", <2>
|
||||
"index.lifecycle.rollover_alias": "timeseries" <3>
|
||||
}
|
||||
}
|
||||
-----------------------
|
||||
// TEST[continued]
|
||||
|
||||
<1> Apply the template to a new index if its name starts with `datastream-`.
|
||||
<1> Apply the template to a new index if its name starts with `timeseries-`.
|
||||
<2> The name of the lifecycle policy to apply to each new index.
|
||||
<3> The name of the alias used to reference these indices.
|
||||
<3> The name of the alias used to reference these indices.
|
||||
Required for policies that use the rollover action.
|
||||
|
||||
//////////////////////////
|
||||
|
||||
[source,console]
|
||||
--------------------------------------------------
|
||||
DELETE /_template/datastream_template
|
||||
DELETE /_template/timeseries_template
|
||||
--------------------------------------------------
|
||||
// TEST[continued]
|
||||
|
||||
|
|
@ -123,20 +123,20 @@ DELETE /_template/datastream_template
|
|||
[[ilm-gs-bootstrap]]
|
||||
=== Bootstrap the initial time-series index
|
||||
|
||||
To get things started, you need to bootstrap an initial index and
|
||||
designate it as the write index for the rollover alias specified in your index template.
|
||||
The name of this index must match the template's index pattern and end with a number.
|
||||
On rollover, this value is incremented to generate a name for the new index.
|
||||
To get things started, you need to bootstrap an initial index and
|
||||
designate it as the write index for the rollover alias specified in your index template.
|
||||
The name of this index must match the template's index pattern and end with a number.
|
||||
On rollover, this value is incremented to generate a name for the new index.
|
||||
|
||||
For example, the following request creates an index called `datastream-000001`
|
||||
and makes it the write index for the `datastream` alias.
|
||||
For example, the following request creates an index called `timeseries-000001`
|
||||
and makes it the write index for the `timeseries` alias.
|
||||
|
||||
[source,console]
|
||||
-----------------------
|
||||
PUT datastream-000001
|
||||
PUT timeseries-000001
|
||||
{
|
||||
"aliases": {
|
||||
"datastream": {
|
||||
"timeseries": {
|
||||
"is_write_index": true
|
||||
}
|
||||
}
|
||||
|
|
@ -146,13 +146,13 @@ PUT datastream-000001
|
|||
|
||||
When the rollover conditions are met, the `rollover` action:
|
||||
|
||||
* Creates a new index called `datastream-000002`.
|
||||
This matches the `datastream-*` pattern, so the settings from `datastream_template` are applied to the new index.
|
||||
* Creates a new index called `timeseries-000002`.
|
||||
This matches the `timeseries-*` pattern, so the settings from `timeseries_template` are applied to the new index.
|
||||
* Designates the new index as the write index and makes the bootstrap index read-only.
|
||||
|
||||
This process repeats each time rollover conditions are met.
|
||||
You can search across all of the indices managed by the `datastream_policy` with the `datastream` alias.
|
||||
Write operations are routed to the current write index.
|
||||
This process repeats each time rollover conditions are met.
|
||||
You can search across all of the indices managed by the `timeseries_policy` with the `timeseries` alias.
|
||||
Write operations are routed to the current write index.
|
||||
|
||||
For more information about write indices and rollover, see the <<rollover-index-api-desc, rollover API>>.
|
||||
|
||||
|
|
@ -160,43 +160,43 @@ For more information about write indices and rollover, see the <<rollover-index-
|
|||
[[ilm-gs-check-progress]]
|
||||
=== Checking progress
|
||||
|
||||
To get status information for managed indices, you use the {ilm-init} explain API.
|
||||
To get status information for managed indices, you use the {ilm-init} explain API.
|
||||
This lets you find out things like:
|
||||
|
||||
* What phase an index is in and when it entered that phase.
|
||||
* The current action and what step is being performed.
|
||||
* If any errors have occurred or progress is blocked.
|
||||
|
||||
For example, the following request gets information about the `datastream` indices:
|
||||
For example, the following request gets information about the `timeseries` indices:
|
||||
|
||||
[source,console]
|
||||
--------------------------------------------------
|
||||
GET datastream-*/_ilm/explain
|
||||
GET timeseries-*/_ilm/explain
|
||||
--------------------------------------------------
|
||||
// TEST[continued]
|
||||
|
||||
The response below shows that the bootstrap index is waiting in the `hot` phase's `rollover` action.
|
||||
It remains in this state and {ilm-init} continues to call `attempt-rollover`
|
||||
until the rollover conditions are met.
|
||||
It remains in this state and {ilm-init} continues to call `attempt-rollover`
|
||||
until the rollover conditions are met.
|
||||
|
||||
[source,console-result]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"indices": {
|
||||
"datastream-000001": {
|
||||
"index": "datastream-000001",
|
||||
"managed": true,
|
||||
"policy": "datastream_policy", <1>
|
||||
"timeseries-000001": {
|
||||
"index": "timeseries-000001",
|
||||
"managed": true,
|
||||
"policy": "timeseries_policy", <1>
|
||||
"lifecycle_date_millis": 1538475653281,
|
||||
"age": "30s", <2>
|
||||
"phase": "hot",
|
||||
"phase": "hot",
|
||||
"phase_time_millis": 1538475653317,
|
||||
"action": "rollover",
|
||||
"action": "rollover",
|
||||
"action_time_millis": 1538475653317,
|
||||
"step": "attempt-rollover", <3>
|
||||
"step_time_millis": 1538475653317,
|
||||
"phase_execution": {
|
||||
"policy": "datastream_policy",
|
||||
"policy": "timeseries_policy",
|
||||
"phase_definition": { <4>
|
||||
"min_age": "0ms",
|
||||
"actions": {
|
||||
|
|
@ -206,7 +206,7 @@ until the rollover conditions are met.
|
|||
}
|
||||
}
|
||||
},
|
||||
"version": 1,
|
||||
"version": 1,
|
||||
"modified_date_in_millis": 1539609701576
|
||||
}
|
||||
}
|
||||
|
|
@ -215,7 +215,7 @@ until the rollover conditions are met.
|
|||
--------------------------------------------------
|
||||
// TESTRESPONSE[skip:no way to know if we will get this response immediately]
|
||||
|
||||
<1> The policy used to manage the index
|
||||
<1> The policy used to manage the index
|
||||
<2> The age of the index
|
||||
<3> The step {ilm-init} is performing on the index
|
||||
<4> The definition of the current phase (the `hot` phase)
|
||||
|
|
|
|||
Loading…
Reference in New Issue