158 lines
6.1 KiB
Plaintext
158 lines
6.1 KiB
Plaintext
[role="xpack"]
|
|
[testenv="basic"]
|
|
[[using-policies-rollover]]
|
|
== Using policies to manage index rollover
|
|
|
|
The rollover action enables you to automatically roll over to a new index based
|
|
on the index size, document count, or age. When a rollover is triggered, a new
|
|
index is created, the write alias is updated to point to the new index, and all
|
|
subsequent updates are written to the new index.
|
|
|
|
Rolling over to a new index based on size, document count, or age is preferable
|
|
to time-based rollovers. Rolling over at an arbitrary time often results in
|
|
many small indices, which can have a negative impact on performance and
|
|
resource usage.
|
|
|
|
You control when the rollover action is triggered by specifying one or more
|
|
rollover parameters. The rollover is performed once any of the criteria are
|
|
met. Because the criteria are checked periodically, the index might grow
|
|
slightly beyond the specified threshold. To control how often the criteria are
|
|
checked, specify the `indices.lifecycle.poll_interval` cluster setting.
|
|
|
|
IMPORTANT: New indices created via rollover will not automatically inherit the
|
|
policy used by the old index, and will not use any policy by default. Therefore,
|
|
it is highly recommended to apply the policy via
|
|
<<applying-policy-to-template,index template>>, including a Rollover alias
|
|
setting, for your indices which specifies the policy you wish to use for each
|
|
new index.
|
|
|
|
The rollover action takes the following parameters:
|
|
|
|
[[rollover-action-params]]
|
|
.`rollover` Action Parameters
|
|
[options="header"]
|
|
|===
|
|
|Name |Description
|
|
|max_size |The maximum estimated size the primary shard of the index is allowed
|
|
to grow to. Defaults to `null`. Optional.
|
|
|max_docs |The maximum number of document the index should
|
|
contain. Defaults to `null`. Optional.
|
|
|max_age |The maximum age of the index. Defaults to `null`. Optional.
|
|
|===
|
|
|
|
These parameters are used to determine when the index is considered "full" and
|
|
a rollover should be performed. Where multiple criteria are defined the
|
|
rollover operation will be performed once any of the criteria are met.
|
|
|
|
The following request defines a policy with a rollover action that triggers
|
|
when the index size reaches 25GB. The old index is subsequently deleted after
|
|
30 days.
|
|
|
|
NOTE: Once an index rolls over, {ilm} uses the timestamp of the rollover
|
|
operation rather than the index creation time to evaluate when to move the
|
|
index to the next phase. For indices that have rolled over, the `min_age`
|
|
criteria specified for a phase is relative to the rollover time for indices. In
|
|
this example, that means the index will be deleted 30 days after rollover, not
|
|
30 days from when the index was created.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /_ilm/policy/my_policy
|
|
{
|
|
"policy": {
|
|
"phases": {
|
|
"hot": {
|
|
"actions": {
|
|
"rollover": {
|
|
"max_size": "25GB"
|
|
}
|
|
}
|
|
},
|
|
"delete": {
|
|
"min_age": "30d",
|
|
"actions": {
|
|
"delete": {}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
To use an {ilm} policy, you need to specify it in the index template used to
|
|
create the indices. For example, the following template associates `my_policy`
|
|
with indices created from the template `my_template`.
|
|
|
|
[source,console]
|
|
-----------------------
|
|
PUT _template/my_template
|
|
{
|
|
"index_patterns": ["test-*"], <1>
|
|
"settings": {
|
|
"number_of_shards": 1,
|
|
"number_of_replicas": 1,
|
|
"index.lifecycle.name": "my_policy", <2>
|
|
"index.lifecycle.rollover_alias": "test-alias" <3>
|
|
}
|
|
}
|
|
-----------------------
|
|
|
|
<1> Template applies to all indices with the prefix test-
|
|
<2> Associates my_policy with all indices created with this template
|
|
<3> Rolls over the write alias test when the rollover action is triggered
|
|
|
|
To be able to start using the policy for these `test-*` indexes we need to
|
|
bootstrap the process by creating the first index.
|
|
|
|
[source,console]
|
|
-----------------------
|
|
PUT test-000001 <1>
|
|
{
|
|
"aliases": {
|
|
"test-alias":{
|
|
"is_write_index": true <2>
|
|
}
|
|
}
|
|
}
|
|
-----------------------
|
|
|
|
<1> Creates the index called test-000001. The rollover action increments the
|
|
suffix number for each subsequent index.
|
|
<2> Designates this index as the write index for this alias.
|
|
|
|
When the rollover is performed, the newly-created index is set as the write
|
|
index for the rolled over alias. Documents sent to the alias are indexed into
|
|
the new index, enabling indexing to continue uninterrupted.
|
|
|
|
[[skipping-rollover]]
|
|
=== Skipping Rollover
|
|
|
|
The `index.lifecycle.indexing_complete` setting indicates to {ilm} whether this
|
|
index has already been rolled over. If it is set to `true`, that indicates that
|
|
this index has already been rolled over and does not need to be rolled over
|
|
again. Therefore, {ilm} will skip any Rollover Action configured in the
|
|
associated lifecycle policy for this index. This is useful if you need to make
|
|
an exception to your normal Lifecycle Policy and switching the alias to a
|
|
different index by hand, but do not want to remove the index from {ilm}
|
|
completely.
|
|
|
|
This setting is set to `true` automatically by ILM upon the successful
|
|
completion of a Rollover Action. However, it will be removed if
|
|
<<ilm-remove-policy,the policy is removed>> from the index.
|
|
|
|
IMPORTANT: If `index.lifecycle.indexing_complete` is set to `true` on an index,
|
|
it will not be rolled over by {ilm}, but {ilm} will verify that this index is no
|
|
longer the write index for the alias specified by
|
|
`index.lifecycle.rollover_alias`. If that setting is missing, or if the index is
|
|
still the write index for that alias, this index will be moved to the
|
|
<<index-lifecycle-error-handling,error step>>.
|
|
|
|
For example, if you wish to change the name of new indices while retaining
|
|
previous data in accordance with your configured policy, you can create the
|
|
template for the new index name pattern and the first index with the new name
|
|
manually, change the write index of the alias using the <<indices-aliases, Index
|
|
Aliases API>>, and set `index.lifecycle.indexing_complete` to `true` on the old
|
|
index to indicate that it does not need to be rolled over. This way, {ilm} will
|
|
continue to manage the old index in accordance with its existing policy, as well
|
|
as the new one, with no interruption.
|