2020-02-04 19:45:18 -05:00
|
|
|
[role="xpack"]
|
|
|
|
[testenv="basic"]
|
|
|
|
[[overview-index-lifecycle-management]]
|
2020-04-28 19:38:01 -04:00
|
|
|
== {ilm-init} overview
|
2020-02-04 19:45:18 -05:00
|
|
|
|
|
|
|
++++
|
|
|
|
<titleabbrev>Overview</titleabbrev>
|
|
|
|
++++
|
|
|
|
|
|
|
|
You can create and apply {ilm-cap} ({ilm-init}) policies to automatically manage your indices
|
|
|
|
according to your performance, resiliency, and retention requirements.
|
[DOCS] Combo version of ILM docs. (#57909) (#59029)
* [DOCS] Combo version of ILM docs.
* [DOCS] Moved tutorial from Kibana.
* Adds documentation for index lifecycle policies (#28705)
* [DOCS] Adds documentation for index lifecycle policies
* [DOCS] Updated image for policy options to show all menu items
* Update create-policy.asciidoc
* [DOCS] Incorporated review comments on hot and warm phase
* [DOCS] Additional changes to warm phase
* [DOCS] Removed the word open in the warm phase
* Adds X-Pack icon for ILM (#34178)
* Add ILM tutorial (#59502)
* Add tutorial for ILM with filebeat
* Change screenshots and add additional steps
* Update screenshots, add numbered steps, and other minor edits
* Incorporate feedback: update links, formatting, and minor edits
* Move tip inline with list
* Apply suggestions from code review
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
* Move TIP inline . . . again
* Put TIP inline
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
* Updates for navigation redesign (#68709)
* [DOCS] Updates for navigation redesign
* Getting started
* Set up text
* Discover
* Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools
* Dev Tools, Stack Monitoring, Management
* Management
* Final changes
* [DOCS] Updates for navigation redesign
* [DOCS] Updates CCR monitoring screenshots
* updates SIEM screenshot and Cases overview text
* Added Brandon's APM image
* [DOCS] Refines CCR shard screenshot
* Removed merge conflict image file
Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
* [DOCS] Put API examples in collapsible sections like ML does
* Fix include
* Added tutorial images
* Fixed images
* Add short title for FB tutorial
* Add missing files
* Incorporate review feedback
* review feedback
* Incorporated review feedback
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 16:01:08 -04:00
|
|
|
|
2020-02-04 19:45:18 -05:00
|
|
|
Index lifecycle policies can trigger actions such as:
|
|
|
|
|
2020-06-05 21:55:51 -04:00
|
|
|
* **Rollover**:
|
2020-02-04 19:45:18 -05:00
|
|
|
include::../glossary.asciidoc[tag=rollover-def-short]
|
2020-06-05 21:55:51 -04:00
|
|
|
* **Shrink**:
|
2020-02-04 19:45:18 -05:00
|
|
|
include::../glossary.asciidoc[tag=shrink-def-short]
|
2020-06-05 21:55:51 -04:00
|
|
|
* **Force merge**:
|
2020-02-04 19:45:18 -05:00
|
|
|
include::../glossary.asciidoc[tag=force-merge-def-short]
|
2020-06-05 21:55:51 -04:00
|
|
|
* **Freeze**:
|
2020-02-04 19:45:18 -05:00
|
|
|
include::../glossary.asciidoc[tag=freeze-def-short]
|
2020-06-05 21:55:51 -04:00
|
|
|
* **Delete**: Permanently remove an index, including all of its data and metadata.
|
2020-02-04 19:45:18 -05:00
|
|
|
|
2020-04-28 19:38:01 -04:00
|
|
|
{ilm-init} makes it easier to manage indices in hot-warm-cold architectures,
|
2020-08-24 11:18:07 -04:00
|
|
|
which are common when you're working with time series data such as logs and metrics.
|
2020-02-04 19:45:18 -05:00
|
|
|
|
2020-04-28 19:38:01 -04:00
|
|
|
You can specify:
|
2020-02-04 19:45:18 -05:00
|
|
|
|
2020-06-05 21:55:51 -04:00
|
|
|
* The maximum shard size, number of documents, or age at which you want to roll over to a new index.
|
2020-02-04 19:45:18 -05:00
|
|
|
* The point at which the index is no longer being updated and the number of
|
|
|
|
primary shards can be reduced.
|
2020-04-28 19:38:01 -04:00
|
|
|
* When to force a merge to permanently remove documents marked for deletion.
|
2020-02-04 19:45:18 -05:00
|
|
|
* The point at which the index can be moved to less performant hardware.
|
|
|
|
* The point at which the availability is not as critical and the number of
|
|
|
|
replicas can be reduced.
|
|
|
|
* When the index can be safely deleted.
|
|
|
|
|
|
|
|
For example, if you are indexing metrics data from a fleet of ATMs into
|
|
|
|
Elasticsearch, you might define a policy that says:
|
|
|
|
|
2020-11-03 13:24:13 -05:00
|
|
|
. When the total size of the index's primary shards reaches 50GB, roll over to a new
|
|
|
|
index.
|
[DOCS] Combo version of ILM docs. (#57909) (#59029)
* [DOCS] Combo version of ILM docs.
* [DOCS] Moved tutorial from Kibana.
* Adds documentation for index lifecycle policies (#28705)
* [DOCS] Adds documentation for index lifecycle policies
* [DOCS] Updated image for policy options to show all menu items
* Update create-policy.asciidoc
* [DOCS] Incorporated review comments on hot and warm phase
* [DOCS] Additional changes to warm phase
* [DOCS] Removed the word open in the warm phase
* Adds X-Pack icon for ILM (#34178)
* Add ILM tutorial (#59502)
* Add tutorial for ILM with filebeat
* Change screenshots and add additional steps
* Update screenshots, add numbered steps, and other minor edits
* Incorporate feedback: update links, formatting, and minor edits
* Move tip inline with list
* Apply suggestions from code review
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
* Move TIP inline . . . again
* Put TIP inline
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
* Updates for navigation redesign (#68709)
* [DOCS] Updates for navigation redesign
* Getting started
* Set up text
* Discover
* Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools
* Dev Tools, Stack Monitoring, Management
* Management
* Final changes
* [DOCS] Updates for navigation redesign
* [DOCS] Updates CCR monitoring screenshots
* updates SIEM screenshot and Cases overview text
* Added Brandon's APM image
* [DOCS] Refines CCR shard screenshot
* Removed merge conflict image file
Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
* [DOCS] Put API examples in collapsible sections like ML does
* Fix include
* Added tutorial images
* Fixed images
* Add short title for FB tutorial
* Add missing files
* Incorporate review feedback
* review feedback
* Incorporated review feedback
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 16:01:08 -04:00
|
|
|
. Move the old index into the warm phase, mark it read only, and shrink it down
|
2020-02-04 19:45:18 -05:00
|
|
|
to a single shard.
|
[DOCS] Combo version of ILM docs. (#57909) (#59029)
* [DOCS] Combo version of ILM docs.
* [DOCS] Moved tutorial from Kibana.
* Adds documentation for index lifecycle policies (#28705)
* [DOCS] Adds documentation for index lifecycle policies
* [DOCS] Updated image for policy options to show all menu items
* Update create-policy.asciidoc
* [DOCS] Incorporated review comments on hot and warm phase
* [DOCS] Additional changes to warm phase
* [DOCS] Removed the word open in the warm phase
* Adds X-Pack icon for ILM (#34178)
* Add ILM tutorial (#59502)
* Add tutorial for ILM with filebeat
* Change screenshots and add additional steps
* Update screenshots, add numbered steps, and other minor edits
* Incorporate feedback: update links, formatting, and minor edits
* Move tip inline with list
* Apply suggestions from code review
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
* Move TIP inline . . . again
* Put TIP inline
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
* Updates for navigation redesign (#68709)
* [DOCS] Updates for navigation redesign
* Getting started
* Set up text
* Discover
* Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools
* Dev Tools, Stack Monitoring, Management
* Management
* Final changes
* [DOCS] Updates for navigation redesign
* [DOCS] Updates CCR monitoring screenshots
* updates SIEM screenshot and Cases overview text
* Added Brandon's APM image
* [DOCS] Refines CCR shard screenshot
* Removed merge conflict image file
Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
* [DOCS] Put API examples in collapsible sections like ML does
* Fix include
* Added tutorial images
* Fixed images
* Add short title for FB tutorial
* Add missing files
* Incorporate review feedback
* review feedback
* Incorporated review feedback
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 16:01:08 -04:00
|
|
|
. After 7 days, move the index into the cold phase and move it to less expensive
|
2020-02-04 19:45:18 -05:00
|
|
|
hardware.
|
|
|
|
. Delete the index once the required 30 day retention period is reached.
|
|
|
|
|
|
|
|
[IMPORTANT]
|
|
|
|
===========================
|
|
|
|
To use {ilm-init}, all nodes in a cluster must run the same version.
|
|
|
|
Although it might be possible to create and apply policies in a mixed-version cluster,
|
|
|
|
there is no guarantee they will work as intended.
|
|
|
|
Attempting to use a policy that contains actions that aren't
|
|
|
|
supported on all nodes in a cluster will cause errors.
|
2020-06-05 21:55:51 -04:00
|
|
|
===========================
|