update updating-policy docs (#34661)
This contains updates to the ILM Docs explaining how updates to policies work and affect policy execution on indices.
This commit is contained in:
Tal Levy
GitHub
parent
810cd46a30
commit
f76ced4e14
|
|
@ -1,5 +1,430 @@
|
|||
[[update-lifecycle-policy]]
|
||||
== Update lifecycle policy
|
||||
|
||||
Discuss how one updates an existing policy (when allowed, when not),
|
||||
as well as all the stuff about switching policies on indices.
|
||||
Updating existing ILM policies is useful to fix mistakes or change
|
||||
strategies for newly created indices. It is possible to update policy definitions
|
||||
and an index's `index.lifecycle.name` settings independently. To prevent the situation
|
||||
that phase definitions are modified while currently being executed on an index, each index
|
||||
will keep the version of the current phase definition it began execution with until it completes.
|
||||
|
||||
There are three scenarios for examining the behavior updating policies and
|
||||
their effects on policy execution on indices.
|
||||
|
||||
=== Updates to policies not managing indices
|
||||
|
||||
Indices not referencing an existing policy that is updated will not be affected.
|
||||
If an index is assigned to the policy, it will be assigned the latest version of that policy
|
||||
|
||||
To show this, let's create a policy `my_policy`.
|
||||
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT _ilm/my_policy
|
||||
{
|
||||
"policy": {
|
||||
"phases": {
|
||||
"hot": {
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_size": "25GB"
|
||||
}
|
||||
}
|
||||
},
|
||||
"delete": {
|
||||
"minimum_age": "30d",
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
|
||||
This newly defined policy will be created and assigned to have a version equal
|
||||
to 1. Since we haven't assigned any indices to this policy, any updates that
|
||||
occur will be reflected completely on indices that are newly set to be managed
|
||||
by this policy.
|
||||
|
||||
Updating the Delete phase's minimum age can be done in an update request.
|
||||
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT _ilm/my_policy
|
||||
{
|
||||
"policy": {
|
||||
"phases": {
|
||||
"hot": {
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_size": "25GB"
|
||||
}
|
||||
}
|
||||
},
|
||||
"delete": {
|
||||
"minimum_age": "10d", <1>
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
<1> update `minimum_age` to 10 days
|
||||
|
||||
//////////
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
GET _ilm
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
//////////
|
||||
|
||||
When we get the policy, we will see it reflect our latest changes, but
|
||||
with its version bumped to 2.
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"my_policy": {
|
||||
"version": 2, <1>
|
||||
"modified_date": 82392349, <2>
|
||||
"policy": {
|
||||
"phases": {
|
||||
"hot": {
|
||||
"minimum_age": "0ms",
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_size": "25gb"
|
||||
}
|
||||
}
|
||||
},
|
||||
"delete": {
|
||||
"minimum_age": "10d",
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TESTRESPONSE[s/"modified_date": 82392349/"modified_date": $body.my_policy.modified_date/]
|
||||
<1> The updated version value
|
||||
<2> The timestamp when this policy was updated last.
|
||||
|
||||
Afterwords, any indices set to `my_policy` will execute against version 2 of
|
||||
the policy.
|
||||
|
||||
=== Updates to executing policies
|
||||
|
||||
Indices preserve the phase definition from the latest policy version that existed
|
||||
at the time that it entered that phase. Changes to the currently-executing phase within policy updates will
|
||||
not be reflected during execution. This means that updates to the `hot` phase, for example, will not affect
|
||||
indices that are currently executing the corresponding `hot` phase.
|
||||
|
||||
Let's say we have an index `my_index` managed by the below `my_executing_policy` definition.
|
||||
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT _ilm/my_executing_policy
|
||||
{
|
||||
"policy": {
|
||||
"phases": {
|
||||
"hot": {
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_docs": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
"delete": {
|
||||
"minimum_age": "10d",
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
|
||||
////
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT my_index
|
||||
{
|
||||
"settings": {
|
||||
"index.lifecycle.name": "my_executing_policy"
|
||||
}
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
////
|
||||
|
||||
The <<ilm-explain,Explain API>> is useful to introspect managed indices to see which phase definition they are currently executing.
|
||||
Using this API, we can find out that `my_index` is currently attempting to be rolled over.
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
GET my_index/_ilm/explain
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"indices": {
|
||||
"my_index": {
|
||||
"index": "my_index",
|
||||
"managed": true,
|
||||
"policy": "my_executing_policy",
|
||||
"lifecycle_date": 1538475653281,
|
||||
"phase": "hot",
|
||||
"phase_time": 1538475653317,
|
||||
"action": "rollover",
|
||||
"action_time": 1538475653317,
|
||||
"step": "attempt_rollover",
|
||||
"step_time": 1538475653317,
|
||||
"phase_execution": {
|
||||
"policy": "my_executing_policy",
|
||||
"modified_date_in_millis": 1538475653317,
|
||||
"version": 1,
|
||||
"phase_definition": {
|
||||
"minimum_age": "0ms",
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_docs": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TESTRESPONSE[s/"lifecycle_date": 1538475653281/"lifecycle_date": $body.indices.my_index.lifecycle_date/]
|
||||
// TESTRESPONSE[s/"phase_time": 1538475653317/"phase_time": $body.indices.my_index.phase_time/]
|
||||
// TESTRESPONSE[s/"action_time": 1538475653317/"action_time": $body.indices.my_index.action_time/]
|
||||
// TESTRESPONSE[s/"step_time": 1538475653317/"step_time": $body.indices.my_index.step_time/]
|
||||
// TESTRESPONSE[s/"modified_date_in_millis": 1538475653317/"modified_date_in_millis": $body.indices.my_index.phase_execution.modified_date_in_millis/]
|
||||
|
||||
Updating `my_executing_policy` to have no rollover action and, instead, go directly into a newly introduced `warm` phase.
|
||||
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT _ilm/my_executing_policy
|
||||
{
|
||||
"policy": {
|
||||
"phases": {
|
||||
"warm": {
|
||||
"minimum_age": "1d",
|
||||
"actions": {
|
||||
"forcemerge": {
|
||||
"max_num_segments": 1
|
||||
}
|
||||
}
|
||||
},
|
||||
"delete": {
|
||||
"minimum_age": "10d",
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
|
||||
Now, version 2 of this policy has no `hot` phase, but if we run the Explain API again, we will see that nothing has changed.
|
||||
The index `my_index` is still executing version 1 of the policy.
|
||||
|
||||
////
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
GET my_index/_ilm/explain
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
////
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"indices": {
|
||||
"my_index": {
|
||||
"index": "my_index",
|
||||
"managed": true,
|
||||
"policy": "my_executing_policy",
|
||||
"lifecycle_date": 1538475653281,
|
||||
"phase": "hot",
|
||||
"phase_time": 1538475653317,
|
||||
"action": "rollover",
|
||||
"action_time": 1538475653317,
|
||||
"step": "attempt_rollover",
|
||||
"step_time": 1538475653317,
|
||||
"phase_execution": {
|
||||
"policy": "my_executing_policy",
|
||||
"modified_date_in_millis": 1538475653317,
|
||||
"version": 1,
|
||||
"phase_definition": {
|
||||
"minimum_age": "0ms",
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_docs": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TESTRESPONSE[s/"lifecycle_date": 1538475653281/"lifecycle_date": $body.indices.my_index.lifecycle_date/]
|
||||
// TESTRESPONSE[s/"phase_time": 1538475653317/"phase_time": $body.indices.my_index.phase_time/]
|
||||
// TESTRESPONSE[s/"action_time": 1538475653317/"action_time": $body.indices.my_index.action_time/]
|
||||
// TESTRESPONSE[s/"step_time": 1538475653317/"step_time": $body.indices.my_index.step_time/]
|
||||
// TESTRESPONSE[s/"modified_date_in_millis": 1538475653317/"modified_date_in_millis": $body.indices.my_index.phase_execution.modified_date_in_millis/]
|
||||
|
||||
After indexing one document into `my_index` so that rollover succeeds and moves onto the next phase, we will notice something new. The
|
||||
index will move into the next phase in the updated version 2 of its policy.
|
||||
|
||||
////
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
PUT my_index/_doc/1
|
||||
{
|
||||
"foo": "bar"
|
||||
}
|
||||
|
||||
GET my_index/_ilm/explain
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
////
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
{
|
||||
"indices": {
|
||||
"my_index": {
|
||||
"index": "my_index",
|
||||
"managed": true,
|
||||
"policy": "my_executing_policy",
|
||||
"lifecycle_date": 1538475653281,
|
||||
"phase": "warm",
|
||||
"phase_time": 1538475653317,
|
||||
"action": "forcemerge",
|
||||
"action_time": 1538475653317,
|
||||
"step": "forcemerge",
|
||||
"step_time": 1538475653317,
|
||||
"phase_execution": {
|
||||
"policy": "my_executing_policy",
|
||||
"modified_date_in_millis": 1538475653317,
|
||||
"version": 2, <1>
|
||||
"phase_definition": {
|
||||
"minimum_age": "1d",
|
||||
"actions": {
|
||||
"forcemerge": {
|
||||
"max_num_segments": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TESTRESPONSE[skip:There is no way to force the index to move to the next step in a timely manner]
|
||||
<1> The index has moved to using version 2 of the policy
|
||||
|
||||
`my_index` will move to the next phase in the latest policy definition, which is the newly added `warm` phase.
|
||||
|
||||
=== Switching policies for an index
|
||||
|
||||
Setting `index.lifecycle.name` to a different policy behaves much like a policy update, but instead of just
|
||||
switching to a different version, it switches to a different policy.
|
||||
|
||||
After setting a policy for an index, we can switch out `my_policy` with
|
||||
`my_other_policy` by just updating the index's `index.lifecycle.name`
|
||||
setting to the new policy. After completing its currently executed phase,
|
||||
it will move on to the next phase in `my_other_policy`. So if it was on the
|
||||
`hot` phase before, it will move to the `delete` phase after the `hot` phase concluded.
|
||||
|
||||
////
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT _ilm/my_policy
|
||||
{
|
||||
"policy": {
|
||||
"phases": {
|
||||
"hot": {
|
||||
"actions": {
|
||||
"rollover": {
|
||||
"max_size": "25GB"
|
||||
}
|
||||
}
|
||||
},
|
||||
"delete": {
|
||||
"minimum_age": "10d",
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
PUT _ilm/my_other_policy
|
||||
{
|
||||
"policy": {
|
||||
"phases": {
|
||||
"delete": {
|
||||
"minimum_age": "1d",
|
||||
"actions": {
|
||||
"delete": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
PUT my_index
|
||||
{
|
||||
"settings": {
|
||||
"index.lifecycle.name": "my_policy"
|
||||
}
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
|
||||
////
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
PUT my_index/_settings
|
||||
{
|
||||
"lifecycle.name": "my_other_policy"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[continued]
|
||||
|
||||
The change to the new policy will not happen immediately. The currently executing phase
|
||||
of the existing policy for `my_index` will continue to execute until it completes. Once
|
||||
completed, `my_index` will move to being managed by the `my_other_policy`.
|
||||
|
|
|
|||
Loading…
Reference in New Issue