OpenSearch/docs/reference/indices/rollover-index.asciidoc

[[indices-rollover-index]]
=== Rollover index API
++++
<titleabbrev>Rollover index</titleabbrev>
++++

Creates a new index for a rollover target when the target's existing index meets
a condition you provide. A rollover target can be either an
<<indices-aliases, index alias>> or a
<<indices-create-data-stream, data stream>>. When targeting an alias, the alias
is updated to point to the new index. When targeting a data stream, the new
index becomes the data stream's write index and its generation is incremented.

[source,console]
----
POST /alias1/_rollover/twitter
{
  "conditions": {
    "max_age":   "7d",
    "max_docs":  1000,
    "max_size": "5gb"
  }
}
----
// TEST[s/^/PUT my_old_index_name\nPUT my_old_index_name\/_alias\/alias1\n/]


[[rollover-index-api-request]]
==== {api-request-title}


`POST /<rollover-target>/_rollover/<target-index>`

`POST /<rollover-target>/_rollover/`


[[rollover-index-api-desc]]
==== {api-description-title}

The rollover index API rolls a rollover target to a new index when
the existing index meets a condition you provide. You can use this API to retire
an index that becomes too large or too old.

NOTE: To roll over an index, a condition must be met *when you call the API*.
{es} does not monitor the index after you receive an API response. To
automatically roll over indices when a condition is met, you can use {es}'s
<<index-lifecycle-management, index lifecycle management (ILM) policies>>.

The rollover index API accepts a rollover target name
and a list of `conditions`.

If the specified rollover target is an alias pointing to a single index,
the rollover request:

. Creates a new index
. Adds the alias to the new index
. Removes the alias from the original index

If the specified rollover target is an alias pointing to multiple indices,
one of these indices must have `is_write_index` set to `true`.
In this case, the rollover request:

. Creates a new index
. Sets `is_write_index` to `true` for the new index
. Sets `is_write_index` to `false` for the original index

If the specified rollover target is a data stream, the rollover request:

. Creates a new index
. Adds the new index as a backing index and the write index on the data stream
. Increments the `generation` attribute of the data stream

[[rollover-wait-active-shards]]
===== Wait for active shards

Because the rollover operation creates a new index to rollover to, the
<<create-index-wait-for-active-shards,`wait_for_active_shards`>> setting on
index creation applies to the rollover action.


[[rollover-index-api-path-params]]
==== {api-path-parms-title}

`<rollover-target>`::
(Required*, string)
Name of the existing index alias or data stream on which to
to assign to the target index.	perform the rollover.


`<target-index>`::
+
--
(Optional*, string)
Name of the target index to create and assign the index alias.

include::{es-repo-dir}/indices/create-index.asciidoc[tag=index-name-reqs]

*This parameter is not permitted if `rollover-target` is a data stream. In
that case, the new index name will be in the form `.ds-<rollover-target>-000001`
where the zero-padded number of length 6 is the generation of the data stream.

If `rollover-target` is an alias that is assigned to an index name that ends
with `-` and a number such as `logs-000001`. In this case, the name of the new
index follows the same pattern and increments the number. For example,
`logs-000001` increments to `logs-000002`. This number is zero-padded with a
length of 6, regardless of the prior index name.

If the existing index for the alias does not match this pattern, this parameter
is required.

--


[[rollover-index-api-query-params]]
==== {api-query-parms-title}

`dry_run`::
(Optional, boolean)
If `true`,
the request checks whether the index matches provided conditions
but does not perform a rollover.
Defaults to `false`.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=include-type-name]

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=timeoutparms]


[[rollover-index-api-request-body]]
==== {api-request-body-title}

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=aliases]

`conditions`::
+
--
(Optional, object)
If supplied, the set of conditions the rollover target's existing index must
meet to roll over. If omitted, the rollover will be performed unconditionally.

Parameters include:

`max_age`::
(Optional, <<time-units, time units>>)
Maximum age of the index.

`max_docs`::
(Optional, integer)
Maximum number of documents in the index.
Documents added since the last refresh are not included in the document count.
The document count does *not* include documents in replica shards.

`max_size`::
(Optional, <<byte-units, byte units>>)
Maximum index size.
This is the total size of all primary shards in the index.
Replicas are not counted toward the maximum index size.

TIP: To see the current index size, use the <<cat-indices, _cat indices>> API.
The `pri.store.size` value shows the combined size of all primary shards.
--

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=mappings]

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=settings]


[[rollover-index-api-example]]
==== {api-examples-title}

[[rollover-index-basic-ex]]
===== Basic example

[source,console]
--------------------------------------------------
PUT /logs-000001 <1>
{
  "aliases": {
    "logs_write": {}
  }
}

# Add > 1000 documents to logs-000001

POST /logs_write/_rollover <2>
{
  "conditions": {
    "max_age":   "7d",
    "max_docs":  1000,
    "max_size":  "5gb"
  }
}
--------------------------------------------------
// TEST[setup:huge_twitter]
// TEST[s/# Add > 1000 documents to logs-000001/POST _reindex?refresh\n{"source":{"index":"twitter"},"dest":{"index":"logs-000001"}}/]
<1> Creates an index called `logs-0000001` with the alias `logs_write`.
<2> If the index pointed to by `logs_write` was created 7 or more days ago, or
    contains 1,000 or more documents, or has an index size at least around 5GB, then the `logs-000002` index is created
    and the `logs_write` alias is updated to point to `logs-000002`.

The API returns the following response:

[source,console-result]
--------------------------------------------------
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "old_index": "logs-000001",
  "new_index": "logs-000002",
  "rolled_over": true, <1>
  "dry_run": false, <2>
  "conditions": { <3>
    "[max_age: 7d]": false,
    "[max_docs: 1000]": true,
    "[max_size: 5gb]": false,
  }
}
--------------------------------------------------

<1> Whether the index was rolled over.
<2> Whether the rollover was dry run.
<3> The result of each condition.


[[rollover-data-stream-ex]]
===== Roll over a data stream

[source,console]
-----------------------------------
PUT _index_template/template
{
  "index_patterns": ["my-data-stream*"],
  "template": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        }
      }
    }
  },
  "data_stream": {
    "timestamp_field": "@timestamp"
  }
}
-----------------------------------
// TEST

[source,console]
--------------------------------------------------
PUT /_data_stream/my-data-stream <1>

# Add > 1000 documents to my-data-stream

POST /my-data-stream/_rollover <2>
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  }
}
--------------------------------------------------
// TEST[continued]
// TEST[setup:huge_twitter]
// TEST[s/# Add > 1000 documents to my-data-stream/POST _reindex?refresh\n{"source":{"index":"twitter"},"dest":{"index":"my-data-stream","op_type":"create"},"script":{"source":"ctx._source.put('@timestamp',ctx._source.remove('date'))"}}/]
<1> Creates a data stream called `my-data-stream` with one initial backing index
named `my-data-stream-000001`.
<2> This request creates a new backing index, `my-data-stream-000002`, and adds
it as the write index for the `my-data-stream` data stream if the current
write index meets at least one of the following conditions:
+
--
* The index was created 7 or more days ago.
* The index has an index size of 5GB or greater.
* The index contains 1,000 or more documents.
--

The API returns the following response:

[source,console-result]
--------------------------------------------------
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "old_index": ".ds-my-data-stream-000001", <1>
  "new_index": ".ds-my-data-stream-000002", <2>
  "rolled_over": true, <3>
  "dry_run": false, <4>
  "conditions": { <5>
    "[max_age: 7d]": false,
    "[max_docs: 1000]": true,
    "[max_size: 5gb]": false,
  }
}
--------------------------------------------------

<1> The previous write index for the data stream.
<2> The new write index for the data stream.
<3> Whether the index was rolled over.
<4> Whether the rollover was dry run.
<5> The result of each condition.

////
[source,console]
-----------------------------------
DELETE /_data_stream/my-data-stream
DELETE /_index_template/template
-----------------------------------
// TEST[continued]
////

[[rollover-index-settings-ex]]
===== Specify settings for the target index

The settings, mappings, and aliases for the new index are taken from any
matching <<indices-templates,index templates>>. If the rollover target is an
index alias, you can specify `settings`, `mappings`, and `aliases` in the body
of the request just like the <<indices-create-index,create index>> API. Values
specified in the request override any values set in matching index templates.
For example, the following `rollover` request overrides the
`index.number_of_shards` setting:

[source,console]
--------------------------------------------------
PUT /logs-000001
{
  "aliases": {
    "logs_write": {}
  }
}

POST /logs_write/_rollover
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  },
  "settings": {
    "index.number_of_shards": 2
  }
}
--------------------------------------------------


[[rollover-index-specify-index-ex]]
===== Specify a target index name

If the rollover target is an index alias and the name of the existing index ends
with `-` and a number -- e.g. `logs-000001` -- then the name of the new index
will follow the same pattern, incrementing the number (`logs-000002`). The
number is zero-padded with a length of 6, regardless of the old index name.

If the old name doesn't match this pattern then you must specify the name for
the new index as follows:

[source,console]
--------------------------------------------------
POST /my_alias/_rollover/my_new_index_name
{
  "conditions": {
    "max_age":   "7d",
    "max_docs":  1000,
    "max_size": "5gb"
  }
}
--------------------------------------------------
// TEST[s/^/PUT my_old_index_name\nPUT my_old_index_name\/_alias\/my_alias\n/]


[[_using_date_math_with_the_rollover_api]]
===== Use date math with a rollover

If the rollover target is an index alias, it can be useful to use
<<date-math-index-names,date math>> to name the rollover index according to the
date that the index rolled over, e.g. `logstash-2016.02.03`.  The rollover API
supports date math, but requires the index name to end with a dash followed by
a number, e.g. `logstash-2016.02.03-1` which is incremented every time the
index is rolled over. For instance:

[source,console]
--------------------------------------------------
# PUT /<logs-{now/d}-1> with URI encoding:
PUT /%3Clogs-%7Bnow%2Fd%7D-1%3E <1>
{
  "aliases": {
    "logs_write": {}
  }
}

PUT logs_write/_doc/1
{
  "message": "a dummy log"
}

POST logs_write/_refresh

# Wait for a day to pass

POST /logs_write/_rollover <2>
{
  "conditions": {
    "max_docs":   "1"
  }
}
--------------------------------------------------
// TEST[s/now/2016.10.31%7C%7C/]

<1> Creates an index named with today's date (e.g.) `logs-2016.10.31-1`
<2> Rolls over to a new index with today's date, e.g. `logs-2016.10.31-000002` if run immediately, or `logs-2016.11.01-000002` if run after 24 hours

//////////////////////////

[source,console]
--------------------------------------------------
GET _alias
--------------------------------------------------
// TEST[continued]

[source,console-result]
--------------------------------------------------
{
  "logs-2016.10.31-000002": {
    "aliases": {
      "logs_write": {}
    }
  },
  "logs-2016.10.31-1": {
    "aliases": {}
  }
}
--------------------------------------------------

//////////////////////////

These indices can then be referenced as described in the
<<date-math-index-names,date math documentation>>.  For example, to search
over indices created in the last three days, you could do the following:

[source,console]
--------------------------------------------------
# GET /<logs-{now/d}-*>,<logs-{now/d-1d}-*>,<logs-{now/d-2d}-*>/_search
GET /%3Clogs-%7Bnow%2Fd%7D-*%3E%2C%3Clogs-%7Bnow%2Fd-1d%7D-*%3E%2C%3Clogs-%7Bnow%2Fd-2d%7D-*%3E/_search
--------------------------------------------------
// TEST[continued]
// TEST[s/now/2016.10.31%7C%7C/]


[[rollover-index-api-dry-run-ex]]
===== Dry run

The rollover API supports `dry_run` mode, where request conditions can be
checked without performing the actual rollover.

[source,console]
--------------------------------------------------
POST /logs_write/_rollover?dry_run
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  }
}
--------------------------------------------------
// TEST[s/^/PUT logs-000001\nPUT logs-000001\/_alias\/logs_write\n/]


[[indices-rollover-is-write-index]]
===== Roll over a write index

If the rollover target is an index alias for  a write index that has `is_write_index` explicitly set to `true`, it is not
swapped during rollover actions. Since having an alias point to multiple indices is ambiguous in distinguishing
which is the correct write index to roll over, it is not valid to rollover an alias that points to multiple indices.
For this reason, the default behavior is to swap which index is being pointed to by the write-oriented alias. This
was `logs_write` in some of the above examples. Since setting `is_write_index` enables an alias to point to multiple indices
while also being explicit as to which is the write index that rollover should target, removing the alias from the rolled over
index is not necessary. This simplifies things by allowing for one alias to behave both as the write and read aliases for
indices that are being managed with Rollover.

Look at the behavior of the aliases in the following example where `is_write_index` is set on the rolled over index.

[source,console]
--------------------------------------------------
PUT my_logs_index-000001
{
  "aliases": {
    "logs": { "is_write_index": true } <1>
  }
}

PUT logs/_doc/1
{
  "message": "a dummy log"
}

POST logs/_refresh

POST /logs/_rollover
{
  "conditions": {
    "max_docs":   "1"
  }
}

PUT logs/_doc/2 <2>
{
  "message": "a newer log"
}
--------------------------------------------------

<1> configures `my_logs_index` as the write index for the `logs` alias
<2> newly indexed documents against the `logs` alias will write to the new index

[source,console-result]
--------------------------------------------------
{
  "_index" : "my_logs_index-000002",
  "_type" : "_doc",
  "_id" : "2",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}
--------------------------------------------------

//////////////////////////
[source,console]
--------------------------------------------------
GET my_logs_index-000001,my_logs_index-000002/_alias
--------------------------------------------------
// TEST[continued]
//////////////////////////

After the rollover, the alias metadata for the two indices will have the `is_write_index` setting
reflect each index's role, with the newly created index as the write index.

[source,console-result]
--------------------------------------------------
{
  "my_logs_index-000002": {
    "aliases": {
      "logs": { "is_write_index": true }
    }
  },
  "my_logs_index-000001": {
    "aliases": {
      "logs": { "is_write_index" : false }
    }
  }
}
--------------------------------------------------