OpenSearch/docs/reference/ilm/policy-definitions.asciidoc

beta[]
[role="xpack"]
[testenv="basic"]
[[ilm-policy-definition]]
== Policy phases and actions

beta[]

There are four stages in the index lifecycle, in the order
they are executed.

[options="header"]
|======
| Name     | Description
| `hot`    | The index is actively being written to
| `warm`   | The index is generally not being written to, but is still queried
| `cold`   | The index is no longer being updated and is seldom queried. The
information still needs to be searchable, but it's okay if those queries are
slower.
| `delete` | The index is no longer needed and can safely be deleted
|======

Each of these stages is called a "phase". A policy does not need to configure
each phase for an index. For example, one policy may define only the hot
phase and the delete phase, while another may define all four phases.

=== Timing

beta[]

Indices enter phases based on a phase's `min_age` parameter.
The index will not enter the phase until the index's age is older than that
of the `min_age`. The parameter is configured using a time
duration format (see <<time-units, Time Units>>).

`min_age` defaults to zero seconds `0s` for each phase if not specified.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "min_age": "1d",
        "actions": {
          "allocate": {
            "number_of_replicas": 1
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

The Above example configures a policy that moves the index into the warm
phase after one day. Until then, the index is in a waiting state. After
moving into the warm phase, it will wait until 30 days have elapsed before
moving to the delete phase and deleting the index.

`min_age` is usually the time elapsed from the time the index is created. If the
index is rolled over, then `min_age` is the time elapsed from the time the index
is rolled over. The intention here is to execute following phases and actions
relative to when data was written last to a rolled over index.

The previous phase's actions must complete before {ilm} will check `min_age`
and transition into the next phase.

=== Phase Execution

beta[]

The current phase definition, of an index's policy being executed, is stored
in the index's metadata. The phase and its actions are compiled into a series
of discrete steps that are executed sequentially. Since some {ilm-init} actions
are more complex and involve multiple operations against an index, each of these
operations are done in isolation in a unit called a "step". The
<<ilm-explain-lifecycle,Explain Lifecycle API>> exposes this information to us
to see which step our index is either to execute next, or is currently
executing.

=== Actions

beta[]

The below list shows the actions which are available in each phase.

* Hot
  - <<ilm-rollover-action,Rollover>>
* Warm
  - <<ilm-allocate-action,Allocate>>
  - <<ilm-readonly-action,Read-Only>>
  - <<ilm-forcemerge-action,Force Merge>>
  - <<ilm-shrink-action,Shrink>>
* Cold
  - <<ilm-allocate-action,Allocate>>
* Delete
  - <<ilm-delete-action,Delete>>

[[ilm-allocate-action]]
==== Allocate

Phases allowed: warm, cold.

The Allocate action allows you to specify which nodes are allowed to host the
shards of the index and set the number of replicas.
Behind the scenes, it is modifying the index settings
for shard filtering and/or replica counts. When updating the number of replicas,
configuring allocation rules is optional. When configuring allocation rules,
setting number of replicas is optional. Although this action can be treated as
two separate index settings updates, both can be configured at once.

Read more about index replicas <<getting-started-shards-and-replicas,here>>.
Read more about shard allocation filtering in
the <<shard-allocation-filtering,Shard allocation filtering documentation>>.

[[ilm-allocate-options]]
.Allocate Options
[options="header"]
|======
| Name                 | Required  | Default     | Description
| `number_of_replicas` | no        | -           | The number of replicas to
                                                   assign to the index
| `include`            | no        | -           | assigns an index to nodes
                                                   having at least _one_ of the attributes
| `exclude`            | no        | -           | assigns an index to nodes having
                                                   _none_ of the attributes
| `require`            | no        | -           | assigns an index to nodes having
                                                   _all_ of the attributes
|======

If `number_of_replicas` is not configured, then at least one of `include`,
`exclude`, and `require` is required. An empty Allocate Action with no configuration
is invalid.

===== Example: Change number of replicas

In this example, the index's number of replicas is changed to `2`, while allocation
rules are unchanged.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "allocate" : {
            "number_of_replicas" : 2
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

===== Example: Assign index to node with specific "box_type" attribute

This example assigns the index to nodes with `box_type` attribute of "hot" or "warm".

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "allocate" : {
            "include" : {
              "box_type": "hot,warm"
            }
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

===== Example: Assign index to a specific node and update replica settings

This example updates the index to have one replica per shard and be allocated
to nodes with a `box_type` attribute of "cold".

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "allocate" : {
            "number_of_replicas": 1,
            "require" : {
              "box_type": "cold"
            }
        }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

[[ilm-delete-action]]
==== Delete

Phases allowed: delete.

The Delete Action does just that, it deletes the index.

This action does not have any options associated with it.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "delete": {
        "actions": {
          "delete" : { }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

[[ilm-forcemerge-action]]
==== Force Merge

Phases allowed: warm.

NOTE: Index will be be made read-only when this action is run
(see: <<dynamic-index-settings,index.blocks.write>>)

The Force Merge Action <<indices-forcemerge,force merges>> the index into at
most a specific number of <<indices-segments,segments>>.

[[ilm-forcemerge-options]]
.Force Merge Options
[options="header"]
|======
| Name                 | Required  | Default             | Description
| `max_num_segments`   | yes       | -                   | The number of
                                                           segments to merge to.
                                                           To fully merge the
                                                           index, set it to `1`
|======

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "forcemerge" : {
            "max_num_segments": 1
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

[[ilm-readonly-action]]
==== Read-Only

Phases allowed: warm.

This action will set the index to be read-only
(see: <<dynamic-index-settings,index.blocks.write>>)

This action does not have any options associated with it.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "readonly" : { }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

[[ilm-rollover-action]]
==== Rollover

Phases allowed: hot.

[WARNING]
index format must match pattern '^.*-\\d+$', for example (`logs-000001`).
[WARNING]
The managed index must set `index.lifecycle.rollover_alias` as the
alias to rollover. The index must also be the write index for the alias.

For example, if an index to be managed has an alias `my_data`. The managed
index "my_index" must be the write index for the alias. For more information, read
<<indices-rollover-is-write-index,Write Index Alias Behavior>>.

[source,js]
--------------------------------------------------
PUT my_index
{
  "settings": {
    "index.lifecycle.name": "my_policy",
    "index.lifecycle.rollover_alias": "my_data"
  },
  "aliases": {
    "my_data": {
      "is_write_index": true
    }
  }
}
--------------------------------------------------
// CONSOLE

The Rollover Action rolls an alias over to a new index when the
existing index meets one of the rollover conditions.


[[ilm-rollover-options]]
.Rollover Options
[options="header"]
|======
| Name       | Required  | Default             | Description
| `max_size` | no        | -                   | max index storage size.
                                                 See <<byte-units, Byte Units>>
                                                 for formatting
| `max_docs` | no        | -                   | max number of documents an
                                                 index is to contain before
                                                 rolling over.
| `max_age`  | no        | -                   | max time elapsed from index
                                                 creation. See
                                                 <<time-units, Time Units>>
                                                 for formatting
|======

At least one of `max_size`, `max_docs`, `max_age` or any combinations of the
three are required to be specified.

===== Example: Rollover when index is too large

This example rolls the index over when it is at least 100 gigabytes.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover" : {
            "max_size": "100GB"
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

===== Example: Rollover when index has too many documents

This example rolls the index over when it contains at least
100000000 documents.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover" : {
            "max_docs": 100000000
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

===== Example: Rollover when index is too old

This example rolls the index over when it has been created at least
7 days ago.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover" : {
            "max_age": "7d"
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

===== Example: Rollover when index is too old or too large

This example rolls the index over when it has been created at least
7 days ago or it is at least 100 gigabytes. In this case, the index will be
rolled over when any of the conditions is met.

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover" : {
            "max_age": "7d",
            "max_size": "100GB"
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE


===== Example: Rollover condition stalls phase transition

The Rollover action will only complete once one of its conditions is
met. This means that any proceeding phases will be blocked until Rollover
succeeds.

[source,js]
--------------------------------------------------
PUT /_ilm/policy/rollover_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "50G"
          }
        }
      },
      "delete": {
        "min_age": "1d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE

The above example illustrates a policy which attempts to delete an
index one day after the index has been rolled over. It does not
delete the index one day after it has been created.

[[ilm-shrink-action]]
==== Shrink

NOTE: Index will be be made read-only when this action is run
(see: <<dynamic-index-settings,index.blocks.write>>)

This action shrinks an existing index into a new index with fewer primary
shards. It calls the <<indices-shrink-index,Shrink API>> to shrink the index.
Since allocating all the primary shards of the index to one node is a
prerequisite, this action will first allocate the primary shards to a valid
node. After shrinking, it will swap aliases pointing to the original index
into the new shrunken index. The new index will also have a new name:
"shrink-<origin-index-name>". So if the original index was called "logs",
then the new index will be named "shrink-logs".

[[ilm-shrink-options]]
.Shrink Options
[options="header"]
|======
| Name               | Required  | Default             | Description
| `number_of_shards` | yes       | -                   | The number of shards
                                                         to shrink to. must be
                                                         a factor of the number
                                                         of shards in the
                                                         source index.
|======

[source,js]
--------------------------------------------------
PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "shrink" : {
            "number_of_shards": 1
          }
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE


=== Full Policy

beta[]

With all of these actions, we can support complex management strategies for our
indices. This policy will define an index that will start in the hot phase,
rolling over every 50 GB or 7 days. After 30 days it enters the warm phase
and increases the replicas to 2, force merges and shrinks. After 60 days
it enters the cold phase and allocates to "cold" nodes, and after 90 days the
index is deleted.

[source,js]
--------------------------------------------------
PUT _ilm/policy/full_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "7d",
            "max_size": "50G"
          }
        }
      },
      "warm": {
        "min_age": "30d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          },
          "shrink": {
            "number_of_shards": 1
          },
          "allocate": {
            "number_of_replicas": 2
          }
        }
      },
      "cold": {
        "min_age": "60d",
        "actions": {
          "allocate": {
            "require": {
              "type": "cold"
            }
          }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}
--------------------------------------------------
// CONSOLE