OpenSearch/docs/reference/ilm/ilm-overview.asciidoc

57 lines
2.2 KiB
Plaintext
Raw Normal View History

[role="xpack"]
[testenv="basic"]
[[overview-index-lifecycle-management]]
== {ilm-init} overview
++++
<titleabbrev>Overview</titleabbrev>
++++
You can create and apply {ilm-cap} ({ilm-init}) policies to automatically manage your indices
according to your performance, resiliency, and retention requirements.
[DOCS] Combo version of ILM docs. (#57909) (#59029) * [DOCS] Combo version of ILM docs. * [DOCS] Moved tutorial from Kibana. * Adds documentation for index lifecycle policies (#28705) * [DOCS] Adds documentation for index lifecycle policies * [DOCS] Updated image for policy options to show all menu items * Update create-policy.asciidoc * [DOCS] Incorporated review comments on hot and warm phase * [DOCS] Additional changes to warm phase * [DOCS] Removed the word open in the warm phase * Adds X-Pack icon for ILM (#34178) * Add ILM tutorial (#59502) * Add tutorial for ILM with filebeat * Change screenshots and add additional steps * Update screenshots, add numbered steps, and other minor edits * Incorporate feedback: update links, formatting, and minor edits * Move tip inline with list * Apply suggestions from code review Co-Authored-By: James Rodewig <james.rodewig@elastic.co> * Move TIP inline . . . again * Put TIP inline Co-authored-by: James Rodewig <james.rodewig@elastic.co> * Updates for navigation redesign (#68709) * [DOCS] Updates for navigation redesign * Getting started * Set up text * Discover * Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools * Dev Tools, Stack Monitoring, Management * Management * Final changes * [DOCS] Updates for navigation redesign * [DOCS] Updates CCR monitoring screenshots * updates SIEM screenshot and Cases overview text * Added Brandon's APM image * [DOCS] Refines CCR shard screenshot * Removed merge conflict image file Co-authored-by: lcawl <lcawley@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co> * [DOCS] Put API examples in collapsible sections like ML does * Fix include * Added tutorial images * Fixed images * Add short title for FB tutorial * Add missing files * Incorporate review feedback * review feedback * Incorporated review feedback Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co> Co-authored-by: Melori Arellano <melori@elastic.co> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co> Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co> Co-authored-by: Melori Arellano <melori@elastic.co> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 16:01:08 -04:00
Index lifecycle policies can trigger actions such as:
* **Rollover**:
include::../glossary.asciidoc[tag=rollover-def-short]
* **Shrink**:
include::../glossary.asciidoc[tag=shrink-def-short]
* **Force merge**:
include::../glossary.asciidoc[tag=force-merge-def-short]
* **Freeze**:
include::../glossary.asciidoc[tag=freeze-def-short]
* **Delete**: Permanently remove an index, including all of its data and metadata.
{ilm-init} makes it easier to manage indices in hot-warm-cold architectures,
which are common when you're working with time series data such as logs and metrics.
You can specify:
* The maximum shard size, number of documents, or age at which you want to roll over to a new index.
* The point at which the index is no longer being updated and the number of
primary shards can be reduced.
* When to force a merge to permanently remove documents marked for deletion.
* The point at which the index can be moved to less performant hardware.
* The point at which the availability is not as critical and the number of
replicas can be reduced.
* When the index can be safely deleted.
For example, if you are indexing metrics data from a fleet of ATMs into
Elasticsearch, you might define a policy that says:
. When the index reaches 50GB, roll over to a new index.
[DOCS] Combo version of ILM docs. (#57909) (#59029) * [DOCS] Combo version of ILM docs. * [DOCS] Moved tutorial from Kibana. * Adds documentation for index lifecycle policies (#28705) * [DOCS] Adds documentation for index lifecycle policies * [DOCS] Updated image for policy options to show all menu items * Update create-policy.asciidoc * [DOCS] Incorporated review comments on hot and warm phase * [DOCS] Additional changes to warm phase * [DOCS] Removed the word open in the warm phase * Adds X-Pack icon for ILM (#34178) * Add ILM tutorial (#59502) * Add tutorial for ILM with filebeat * Change screenshots and add additional steps * Update screenshots, add numbered steps, and other minor edits * Incorporate feedback: update links, formatting, and minor edits * Move tip inline with list * Apply suggestions from code review Co-Authored-By: James Rodewig <james.rodewig@elastic.co> * Move TIP inline . . . again * Put TIP inline Co-authored-by: James Rodewig <james.rodewig@elastic.co> * Updates for navigation redesign (#68709) * [DOCS] Updates for navigation redesign * Getting started * Set up text * Discover * Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools * Dev Tools, Stack Monitoring, Management * Management * Final changes * [DOCS] Updates for navigation redesign * [DOCS] Updates CCR monitoring screenshots * updates SIEM screenshot and Cases overview text * Added Brandon's APM image * [DOCS] Refines CCR shard screenshot * Removed merge conflict image file Co-authored-by: lcawl <lcawley@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co> * [DOCS] Put API examples in collapsible sections like ML does * Fix include * Added tutorial images * Fixed images * Add short title for FB tutorial * Add missing files * Incorporate review feedback * review feedback * Incorporated review feedback Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co> Co-authored-by: Melori Arellano <melori@elastic.co> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co> Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co> Co-authored-by: Melori Arellano <melori@elastic.co> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 16:01:08 -04:00
. Move the old index into the warm phase, mark it read only, and shrink it down
to a single shard.
[DOCS] Combo version of ILM docs. (#57909) (#59029) * [DOCS] Combo version of ILM docs. * [DOCS] Moved tutorial from Kibana. * Adds documentation for index lifecycle policies (#28705) * [DOCS] Adds documentation for index lifecycle policies * [DOCS] Updated image for policy options to show all menu items * Update create-policy.asciidoc * [DOCS] Incorporated review comments on hot and warm phase * [DOCS] Additional changes to warm phase * [DOCS] Removed the word open in the warm phase * Adds X-Pack icon for ILM (#34178) * Add ILM tutorial (#59502) * Add tutorial for ILM with filebeat * Change screenshots and add additional steps * Update screenshots, add numbered steps, and other minor edits * Incorporate feedback: update links, formatting, and minor edits * Move tip inline with list * Apply suggestions from code review Co-Authored-By: James Rodewig <james.rodewig@elastic.co> * Move TIP inline . . . again * Put TIP inline Co-authored-by: James Rodewig <james.rodewig@elastic.co> * Updates for navigation redesign (#68709) * [DOCS] Updates for navigation redesign * Getting started * Set up text * Discover * Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools * Dev Tools, Stack Monitoring, Management * Management * Final changes * [DOCS] Updates for navigation redesign * [DOCS] Updates CCR monitoring screenshots * updates SIEM screenshot and Cases overview text * Added Brandon's APM image * [DOCS] Refines CCR shard screenshot * Removed merge conflict image file Co-authored-by: lcawl <lcawley@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co> * [DOCS] Put API examples in collapsible sections like ML does * Fix include * Added tutorial images * Fixed images * Add short title for FB tutorial * Add missing files * Incorporate review feedback * review feedback * Incorporated review feedback Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co> Co-authored-by: Melori Arellano <melori@elastic.co> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co> Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co> Co-authored-by: Melori Arellano <melori@elastic.co> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co> Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 16:01:08 -04:00
. After 7 days, move the index into the cold phase and move it to less expensive
hardware.
. Delete the index once the required 30 day retention period is reached.
[IMPORTANT]
===========================
To use {ilm-init}, all nodes in a cluster must run the same version.
Although it might be possible to create and apply policies in a mixed-version cluster,
there is no guarantee they will work as intended.
Attempting to use a policy that contains actions that aren't
supported on all nodes in a cluster will cause errors.
===========================