264 lines
7.3 KiB
Plaintext
264 lines
7.3 KiB
Plaintext
[role="xpack"]
|
|
[testenv="basic"]
|
|
[[set-up-lifecycle-policy]]
|
|
== Configure a lifecycle policy [[ilm-policy-definition]]
|
|
|
|
For {ilm-init} to manage an index, a valid policy
|
|
must be specified in the `index.lifecycle.name` index setting.
|
|
|
|
To configure a lifecycle policy for <<index-rollover, rolling indices>>,
|
|
you create the policy and add it to the <<indices-templates, index template>>.
|
|
|
|
To use a policy to manage an index that doesn't roll over,
|
|
you can specify a lifecycle policy when you create the index,
|
|
or apply a policy directly to an existing index.
|
|
|
|
{ilm-init} policies are stored in the global cluster state and can be included in snapshots
|
|
by setting `include_global_state` to `true` when you <<snapshots-take-snapshot, take the snapshot>>.
|
|
When the snapshot is restored, all of the policies in the global state are restored and
|
|
any local policies with the same names are overwritten.
|
|
|
|
IMPORTANT: When you enable {ilm} for {beats} or the {ls} {es} output plugin,
|
|
the necessary policies and configuration changes are applied automatically.
|
|
You can modify the default policies, but you do not need to explicitly configure a policy or
|
|
bootstrap an initial index.
|
|
|
|
[discrete]
|
|
[[ilm-create-policy]]
|
|
=== Create lifecycle policy
|
|
|
|
To create lifecycle policies through {kib} Management
|
|
go to Management and click **Index Lifecycle Policies**.
|
|
|
|
[role="screenshot"]
|
|
image:images/ilm/create-policy.png[]
|
|
|
|
You specify the lifecycle phases for the policy and the actions to perform in each phase.
|
|
|
|
The <<ilm-put-lifecycle, put policy>> API is invoked to add the policy to the {es} cluster.
|
|
|
|
.API example
|
|
[%collapsible]
|
|
====
|
|
[source,console]
|
|
------------------------
|
|
PUT _ilm/policy/my_policy
|
|
{
|
|
"policy": {
|
|
"phases": {
|
|
"hot": {
|
|
"actions": {
|
|
"rollover": {
|
|
"max_size": "25GB" <1>
|
|
}
|
|
}
|
|
},
|
|
"delete": {
|
|
"min_age": "30d",
|
|
"actions": {
|
|
"delete": {} <2>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
------------------------
|
|
|
|
<1> Roll over the index when it reaches 25GB in size
|
|
<2> Delete the index 30 days after rollover
|
|
====
|
|
|
|
[discrete]
|
|
[[apply-policy-template]]
|
|
=== Apply lifecycle policy with an index template
|
|
|
|
To use a policy that triggers the rollover action,
|
|
you need to configure the policy in the index template used to create each new index.
|
|
You specify the name of the policy and the alias used to reference the rolling indices.
|
|
|
|
To use the Create template wizard to create a template from {kib} Management,
|
|
go to Management, click **Index Management** and select the **Index Templates** view.
|
|
|
|
[role="screenshot"]
|
|
image:images/ilm/create-template-wizard.png[]
|
|
|
|
The wizard invokes the <<indices-put-template,put template API>> to add templates to a cluster.
|
|
|
|
.API example
|
|
[%collapsible]
|
|
====
|
|
[source,console]
|
|
-----------------------
|
|
PUT _template/my_template
|
|
{
|
|
"index_patterns": ["test-*"], <1>
|
|
"settings": {
|
|
"number_of_shards": 1,
|
|
"number_of_replicas": 1,
|
|
"index.lifecycle.name": "my_policy", <2>
|
|
"index.lifecycle.rollover_alias": "test-alias" <3>
|
|
}
|
|
}
|
|
-----------------------
|
|
|
|
<1> Use this template for all new indices whose names begin with `test-`
|
|
<2> Apply `my_policy` to new indices created with this template
|
|
<3> Define an index alias for referencing indices managed by `my_policy`
|
|
====
|
|
//////////////////////////
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
DELETE /_template/my_template
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
//////////////////////////
|
|
|
|
[discrete]
|
|
[[create-initial-index]]
|
|
==== Create an initial managed index
|
|
|
|
When you set up policies for your own rolling indices, you need to manually create the first index
|
|
managed by a policy and designate it as the write index.
|
|
|
|
IMPORTANT: When you enable {ilm} for {beats} or the {ls} {es} output plugin,
|
|
the necessary policies and configuration changes are applied automatically.
|
|
You can modify the default policies, but you do not need to explicitly configure a policy or
|
|
bootstrap an initial index.
|
|
|
|
The name of the index must match the pattern defined in the index template and end with a number.
|
|
This number is incremented to generate the name of indices created by the rollover action.
|
|
|
|
For example, the following request creates the `test-00001` index.
|
|
Because it matches the index pattern specified in `my_template`,
|
|
{es} automatically applies the settings from that template.
|
|
|
|
[source,console]
|
|
-----------------------
|
|
PUT test-000001
|
|
{
|
|
"aliases": {
|
|
"test-alias":{
|
|
"is_write_index": true <1>
|
|
}
|
|
}
|
|
}
|
|
-----------------------
|
|
|
|
<1> Set this initial index to be the write index for this alias.
|
|
|
|
Now you can start indexing data to the rollover alias specified in the lifecycle policy.
|
|
With the sample `my_policy` policy, the rollover action is triggered once the initial
|
|
index exceeds 25GB.
|
|
{ilm-init} then creates a new index that becomes the write index for the `test-alias`.
|
|
|
|
[discrete]
|
|
[[apply-policy-manually]]
|
|
=== Apply lifecycle policy manually
|
|
|
|
You can specify a policy when you create an index or
|
|
apply a policy to an existing index through {kib} Management or
|
|
the <<indices-update-settings, update settings API>>.
|
|
When you apply a policy, {ilm-init} immediately starts managing the index.
|
|
|
|
IMPORTANT: Do not manually apply a policy that uses the rollover action.
|
|
Policies that use rollover must be applied by the <<apply-policy-template, index template>>.
|
|
Otherwise, the policy is not carried forward when the rollover action creates a new index.
|
|
|
|
The `index.lifecycle.name` setting specifies an index's policy.
|
|
|
|
.API example
|
|
[%collapsible]
|
|
====
|
|
[source,console]
|
|
-----------------------
|
|
PUT test-index
|
|
{
|
|
"settings": {
|
|
"number_of_shards": 1,
|
|
"number_of_replicas": 1,
|
|
"index.lifecycle.name": "my_policy" <1>
|
|
}
|
|
}
|
|
-----------------------
|
|
<1> Sets the lifecycle policy for the index.
|
|
====
|
|
|
|
[discrete]
|
|
[[apply-policy-multiple]]
|
|
==== Apply a policy to multiple indices
|
|
|
|
You can apply the same policy to multiple indices by using wildcards in the index name
|
|
when you call the <<indices-update-settings,update settings>> API.
|
|
|
|
WARNING: Be careful that you don't inadvertently match indices that you don't want to modify.
|
|
|
|
//////////////////////////
|
|
[source,console]
|
|
-----------------------
|
|
PUT _template/mylogs_template
|
|
{
|
|
"index_patterns": [
|
|
"mylogs-*"
|
|
],
|
|
"settings": {
|
|
"number_of_shards": 1,
|
|
"number_of_replicas": 1
|
|
},
|
|
"mappings": {
|
|
"properties": {
|
|
"message": {
|
|
"type": "text"
|
|
},
|
|
"@timestamp": {
|
|
"type": "date"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
-----------------------
|
|
|
|
[source,console]
|
|
-----------------------
|
|
POST mylogs-pre-ilm-2019.06.24/_doc
|
|
{
|
|
"@timestamp": "2019-06-24T10:34:00",
|
|
"message": "this is one log message"
|
|
}
|
|
-----------------------
|
|
// TEST[continued]
|
|
|
|
[source,console]
|
|
-----------------------
|
|
POST mylogs-pre-ilm-2019.06.25/_doc
|
|
{
|
|
"@timestamp": "2019-06-25T17:42:00",
|
|
"message": "this is another log message"
|
|
}
|
|
-----------------------
|
|
// TEST[continued]
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
DELETE _template/mylogs_template
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
//////////////////////////
|
|
|
|
[source,console]
|
|
-----------------------
|
|
PUT mylogs-pre-ilm*/_settings <1>
|
|
{
|
|
"index": {
|
|
"lifecycle": {
|
|
"name": "mylogs_policy_existing"
|
|
}
|
|
}
|
|
}
|
|
-----------------------
|
|
// TEST[continued]
|
|
|
|
<1> Updates all indices with names that start with `mylogs-pre-ilm`
|