550 lines
16 KiB
Plaintext
550 lines
16 KiB
Plaintext
[[indices-rollover-index]]
|
|
=== Rollover index API
|
|
++++
|
|
<titleabbrev>Rollover index</titleabbrev>
|
|
++++
|
|
|
|
Creates a new index for a rollover target when the target's existing index meets
|
|
a condition you provide. A rollover target can be either an
|
|
<<indices-aliases, index alias>> or a
|
|
<<indices-create-data-stream, data stream>>. When targeting an alias, the alias
|
|
is updated to point to the new index. When targeting a data stream, the new
|
|
index becomes the data stream's write index and its generation is incremented.
|
|
|
|
[source,console]
|
|
----
|
|
POST /alias1/_rollover/my-index-000002
|
|
{
|
|
"conditions": {
|
|
"max_age": "7d",
|
|
"max_docs": 1000,
|
|
"max_size": "5gb"
|
|
}
|
|
}
|
|
----
|
|
// TEST[s/^/PUT my_old_index_name\nPUT my_old_index_name\/_alias\/alias1\n/]
|
|
|
|
|
|
[[rollover-index-api-request]]
|
|
==== {api-request-title}
|
|
|
|
|
|
`POST /<rollover-target>/_rollover/<target-index>`
|
|
|
|
`POST /<rollover-target>/_rollover/`
|
|
|
|
|
|
[[rollover-index-api-desc]]
|
|
==== {api-description-title}
|
|
|
|
The rollover index API rolls a rollover target to a new index when
|
|
the existing index meets a condition you provide. You can use this API to retire
|
|
an index that becomes too large or too old.
|
|
|
|
NOTE: To roll over an index, a condition must be met *when you call the API*.
|
|
{es} does not monitor the index after you receive an API response. To
|
|
automatically roll over indices when a condition is met, you can use {es}'s
|
|
<<index-lifecycle-management, index lifecycle management (ILM) policies>>.
|
|
|
|
The rollover index API accepts a rollover target name
|
|
and a list of `conditions`.
|
|
|
|
If the specified rollover target is an alias pointing to a single index,
|
|
the rollover request:
|
|
|
|
. Creates a new index
|
|
. Adds the alias to the new index
|
|
. Removes the alias from the original index
|
|
|
|
If the specified rollover target is an alias pointing to multiple indices,
|
|
one of these indices must have `is_write_index` set to `true`.
|
|
In this case, the rollover request:
|
|
|
|
. Creates a new index
|
|
. Sets `is_write_index` to `true` for the new index
|
|
. Sets `is_write_index` to `false` for the original index
|
|
|
|
If the specified rollover target is a data stream, the rollover request:
|
|
|
|
. Creates a new index
|
|
. Adds the new index as a backing index and the write index on the data stream
|
|
. Increments the `generation` attribute of the data stream
|
|
|
|
[[rollover-wait-active-shards]]
|
|
===== Wait for active shards
|
|
|
|
Because the rollover operation creates a new index to rollover to, the
|
|
<<create-index-wait-for-active-shards,`wait_for_active_shards`>> setting on
|
|
index creation applies to the rollover action.
|
|
|
|
|
|
[[rollover-index-api-path-params]]
|
|
==== {api-path-parms-title}
|
|
|
|
`<rollover-target>`::
|
|
(Required*, string)
|
|
Name of the existing index alias or data stream on which to
|
|
to assign to the target index. perform the rollover.
|
|
|
|
|
|
`<target-index>`::
|
|
+
|
|
--
|
|
(Optional*, string)
|
|
Name of the target index to create and assign the index alias.
|
|
|
|
include::{es-repo-dir}/indices/create-index.asciidoc[tag=index-name-reqs]
|
|
|
|
*This parameter is not permitted if `rollover-target` is a data stream. In
|
|
that case, the new index name will be in the form `.ds-<rollover-target>-000001`
|
|
where the zero-padded number of length 6 is the generation of the data stream.
|
|
|
|
If `rollover-target` is an alias that is assigned to an index name that ends
|
|
with `-` and a number such as `logs-000001`. In this case, the name of the new
|
|
index follows the same pattern and increments the number. For example,
|
|
`logs-000001` increments to `logs-000002`. This number is zero-padded with a
|
|
length of 6, regardless of the prior index name.
|
|
|
|
If the existing index for the alias does not match this pattern, this parameter
|
|
is required.
|
|
|
|
--
|
|
|
|
|
|
[[rollover-index-api-query-params]]
|
|
==== {api-query-parms-title}
|
|
|
|
`dry_run`::
|
|
(Optional, boolean)
|
|
If `true`,
|
|
the request checks whether the index matches provided conditions
|
|
but does not perform a rollover.
|
|
Defaults to `false`.
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=include-type-name]
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=timeoutparms]
|
|
|
|
|
|
[[rollover-index-api-request-body]]
|
|
==== {api-request-body-title}
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=aliases]
|
|
|
|
`conditions`::
|
|
+
|
|
--
|
|
(Optional, object)
|
|
If supplied, the set of conditions the rollover target's existing index must
|
|
meet to roll over. If omitted, the rollover will be performed unconditionally.
|
|
|
|
Parameters include:
|
|
|
|
`max_age`::
|
|
(Optional, <<time-units, time units>>)
|
|
Maximum age of the index.
|
|
|
|
`max_docs`::
|
|
(Optional, integer)
|
|
Maximum number of documents in the index.
|
|
Documents added since the last refresh are not included in the document count.
|
|
The document count does *not* include documents in replica shards.
|
|
|
|
`max_size`::
|
|
(Optional, <<byte-units, byte units>>)
|
|
Maximum index size.
|
|
This is the total size of all primary shards in the index.
|
|
Replicas are not counted toward the maximum index size.
|
|
|
|
TIP: To see the current index size, use the <<cat-indices, _cat indices>> API.
|
|
The `pri.store.size` value shows the combined size of all primary shards.
|
|
--
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=mappings]
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=settings]
|
|
|
|
|
|
[[rollover-index-api-example]]
|
|
==== {api-examples-title}
|
|
|
|
[[rollover-index-basic-ex]]
|
|
===== Basic example
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /logs-000001 <1>
|
|
{
|
|
"aliases": {
|
|
"logs_write": {}
|
|
}
|
|
}
|
|
|
|
# Add > 1000 documents to logs-000001
|
|
|
|
POST /logs_write/_rollover <2>
|
|
{
|
|
"conditions": {
|
|
"max_age": "7d",
|
|
"max_docs": 1000,
|
|
"max_size": "5gb"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[setup:my_index_huge]
|
|
// TEST[s/# Add > 1000 documents to logs-000001/POST _reindex?refresh\n{"source":{"index":"my-index-000001"},"dest":{"index":"logs-000001"}}/]
|
|
<1> Creates an index called `logs-0000001` with the alias `logs_write`.
|
|
<2> If the index pointed to by `logs_write` was created 7 or more days ago, or
|
|
contains 1,000 or more documents, or has an index size at least around 5GB, then the `logs-000002` index is created
|
|
and the `logs_write` alias is updated to point to `logs-000002`.
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"acknowledged": true,
|
|
"shards_acknowledged": true,
|
|
"old_index": "logs-000001",
|
|
"new_index": "logs-000002",
|
|
"rolled_over": true, <1>
|
|
"dry_run": false, <2>
|
|
"conditions": { <3>
|
|
"[max_age: 7d]": false,
|
|
"[max_docs: 1000]": true,
|
|
"[max_size: 5gb]": false,
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> Whether the index was rolled over.
|
|
<2> Whether the rollover was dry run.
|
|
<3> The result of each condition.
|
|
|
|
|
|
[[rollover-data-stream-ex]]
|
|
===== Roll over a data stream
|
|
|
|
[source,console]
|
|
-----------------------------------
|
|
PUT _index_template/template
|
|
{
|
|
"index_patterns": ["my-data-stream*"],
|
|
"data_stream": { }
|
|
}
|
|
-----------------------------------
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /_data_stream/my-data-stream <1>
|
|
|
|
# Add > 1000 documents to my-data-stream
|
|
|
|
POST /my-data-stream/_rollover <2>
|
|
{
|
|
"conditions" : {
|
|
"max_age": "7d",
|
|
"max_docs": 1000,
|
|
"max_size": "5gb"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
// TEST[setup:my_index_huge]
|
|
// TEST[s/# Add > 1000 documents to my-data-stream/POST _reindex?refresh\n{ "source": { "index": "my-index-000001" }, "dest": { "index": "my-data-stream", "op_type": "create" } }/]
|
|
<1> Creates a data stream called `my-data-stream` with one initial backing index
|
|
named `my-data-stream-000001`.
|
|
<2> This request creates a new backing index, `my-data-stream-000002`, and adds
|
|
it as the write index for the `my-data-stream` data stream if the current
|
|
write index meets at least one of the following conditions:
|
|
+
|
|
--
|
|
* The index was created 7 or more days ago.
|
|
* The index has an index size of 5GB or greater.
|
|
* The index contains 1,000 or more documents.
|
|
--
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"acknowledged": true,
|
|
"shards_acknowledged": true,
|
|
"old_index": ".ds-my-data-stream-000001", <1>
|
|
"new_index": ".ds-my-data-stream-000002", <2>
|
|
"rolled_over": true, <3>
|
|
"dry_run": false, <4>
|
|
"conditions": { <5>
|
|
"[max_age: 7d]": false,
|
|
"[max_docs: 1000]": true,
|
|
"[max_size: 5gb]": false,
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> The previous write index for the data stream.
|
|
<2> The new write index for the data stream.
|
|
<3> Whether the index was rolled over.
|
|
<4> Whether the rollover was dry run.
|
|
<5> The result of each condition.
|
|
|
|
////
|
|
[source,console]
|
|
-----------------------------------
|
|
DELETE /_data_stream/my-data-stream
|
|
DELETE /_index_template/template
|
|
-----------------------------------
|
|
// TEST[continued]
|
|
////
|
|
|
|
[[rollover-index-settings-ex]]
|
|
===== Specify settings for the target index
|
|
|
|
The settings, mappings, and aliases for the new index are taken from any
|
|
matching <<indices-templates,index templates>>. If the rollover target is an
|
|
index alias, you can specify `settings`, `mappings`, and `aliases` in the body
|
|
of the request just like the <<indices-create-index,create index>> API. Values
|
|
specified in the request override any values set in matching index templates.
|
|
For example, the following `rollover` request overrides the
|
|
`index.number_of_shards` setting:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT /logs-000001
|
|
{
|
|
"aliases": {
|
|
"logs_write": {}
|
|
}
|
|
}
|
|
|
|
POST /logs_write/_rollover
|
|
{
|
|
"conditions" : {
|
|
"max_age": "7d",
|
|
"max_docs": 1000,
|
|
"max_size": "5gb"
|
|
},
|
|
"settings": {
|
|
"index.number_of_shards": 2
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
|
|
[[rollover-index-specify-index-ex]]
|
|
===== Specify a target index name
|
|
|
|
If the rollover target is an index alias and the name of the existing index ends
|
|
with `-` and a number -- e.g. `logs-000001` -- then the name of the new index
|
|
will follow the same pattern, incrementing the number (`logs-000002`). The
|
|
number is zero-padded with a length of 6, regardless of the old index name.
|
|
|
|
If the old name doesn't match this pattern then you must specify the name for
|
|
the new index as follows:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST /my_alias/_rollover/my_new_index_name
|
|
{
|
|
"conditions": {
|
|
"max_age": "7d",
|
|
"max_docs": 1000,
|
|
"max_size": "5gb"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[s/^/PUT my_old_index_name\nPUT my_old_index_name\/_alias\/my_alias\n/]
|
|
|
|
|
|
[[_using_date_math_with_the_rollover_api]]
|
|
===== Use date math with a rollover
|
|
|
|
If the rollover target is an index alias, it can be useful to use
|
|
<<date-math-index-names,date math>> to name the rollover index according to the
|
|
date that the index rolled over, e.g. `logstash-2016.02.03`. The rollover API
|
|
supports date math, but requires the index name to end with a dash followed by
|
|
a number, e.g. `logstash-2016.02.03-1` which is incremented every time the
|
|
index is rolled over. For instance:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
# PUT /<logs-{now/d}-1> with URI encoding:
|
|
PUT /%3Clogs-%7Bnow%2Fd%7D-1%3E <1>
|
|
{
|
|
"aliases": {
|
|
"logs_write": {}
|
|
}
|
|
}
|
|
|
|
PUT logs_write/_doc/1
|
|
{
|
|
"message": "a dummy log"
|
|
}
|
|
|
|
POST logs_write/_refresh
|
|
|
|
# Wait for a day to pass
|
|
|
|
POST /logs_write/_rollover <2>
|
|
{
|
|
"conditions": {
|
|
"max_docs": "1"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[s/now/2016.10.31%7C%7C/]
|
|
|
|
<1> Creates an index named with today's date (e.g.) `logs-2016.10.31-1`
|
|
<2> Rolls over to a new index with today's date, e.g. `logs-2016.10.31-000002` if run immediately, or `logs-2016.11.01-000002` if run after 24 hours
|
|
|
|
//////////////////////////
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET _alias
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"logs-2016.10.31-000002": {
|
|
"aliases": {
|
|
"logs_write": {}
|
|
}
|
|
},
|
|
"logs-2016.10.31-1": {
|
|
"aliases": {}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
//////////////////////////
|
|
|
|
These indices can then be referenced as described in the
|
|
<<date-math-index-names,date math documentation>>. For example, to search
|
|
over indices created in the last three days, you could do the following:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
# GET /<logs-{now/d}-*>,<logs-{now/d-1d}-*>,<logs-{now/d-2d}-*>/_search
|
|
GET /%3Clogs-%7Bnow%2Fd%7D-*%3E%2C%3Clogs-%7Bnow%2Fd-1d%7D-*%3E%2C%3Clogs-%7Bnow%2Fd-2d%7D-*%3E/_search
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
// TEST[s/now/2016.10.31%7C%7C/]
|
|
|
|
|
|
[[rollover-index-api-dry-run-ex]]
|
|
===== Dry run
|
|
|
|
The rollover API supports `dry_run` mode, where request conditions can be
|
|
checked without performing the actual rollover.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
POST /logs_write/_rollover?dry_run
|
|
{
|
|
"conditions" : {
|
|
"max_age": "7d",
|
|
"max_docs": 1000,
|
|
"max_size": "5gb"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[s/^/PUT logs-000001\nPUT logs-000001\/_alias\/logs_write\n/]
|
|
|
|
|
|
[[indices-rollover-is-write-index]]
|
|
===== Roll over a write index
|
|
|
|
If the rollover target is an index alias for a write index that has `is_write_index` explicitly set to `true`, it is not
|
|
swapped during rollover actions. Since having an alias point to multiple indices is ambiguous in distinguishing
|
|
which is the correct write index to roll over, it is not valid to rollover an alias that points to multiple indices.
|
|
For this reason, the default behavior is to swap which index is being pointed to by the write-oriented alias. This
|
|
was `logs_write` in some of the above examples. Since setting `is_write_index` enables an alias to point to multiple indices
|
|
while also being explicit as to which is the write index that rollover should target, removing the alias from the rolled over
|
|
index is not necessary. This simplifies things by allowing for one alias to behave both as the write and read aliases for
|
|
indices that are being managed with Rollover.
|
|
|
|
Look at the behavior of the aliases in the following example where `is_write_index` is set on the rolled over index.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT my_logs_index-000001
|
|
{
|
|
"aliases": {
|
|
"logs": { "is_write_index": true } <1>
|
|
}
|
|
}
|
|
|
|
PUT logs/_doc/1
|
|
{
|
|
"message": "a dummy log"
|
|
}
|
|
|
|
POST logs/_refresh
|
|
|
|
POST /logs/_rollover
|
|
{
|
|
"conditions": {
|
|
"max_docs": "1"
|
|
}
|
|
}
|
|
|
|
PUT logs/_doc/2 <2>
|
|
{
|
|
"message": "a newer log"
|
|
}
|
|
--------------------------------------------------
|
|
|
|
<1> configures `my_logs_index` as the write index for the `logs` alias
|
|
<2> newly indexed documents against the `logs` alias will write to the new index
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_index" : "my_logs_index-000002",
|
|
"_type" : "_doc",
|
|
"_id" : "2",
|
|
"_version" : 1,
|
|
"result" : "created",
|
|
"_shards" : {
|
|
"total" : 2,
|
|
"successful" : 1,
|
|
"failed" : 0
|
|
},
|
|
"_seq_no" : 0,
|
|
"_primary_term" : 1
|
|
}
|
|
--------------------------------------------------
|
|
|
|
//////////////////////////
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET my_logs_index-000001,my_logs_index-000002/_alias
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
//////////////////////////
|
|
|
|
After the rollover, the alias metadata for the two indices will have the `is_write_index` setting
|
|
reflect each index's role, with the newly created index as the write index.
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"my_logs_index-000002": {
|
|
"aliases": {
|
|
"logs": { "is_write_index": true }
|
|
}
|
|
},
|
|
"my_logs_index-000001": {
|
|
"aliases": {
|
|
"logs": { "is_write_index" : false }
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|