2016-05-31 04:41:44 -04:00
|
|
|
|
[[indices-shrink-index]]
|
2019-10-01 17:07:28 -04:00
|
|
|
|
=== Shrink index API
|
|
|
|
|
++++
|
|
|
|
|
<titleabbrev>Shrink index</titleabbrev>
|
|
|
|
|
++++
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
Shrinks an existing index into a new index with fewer primary shards.
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
[source,console]
|
|
|
|
|
----
|
|
|
|
|
POST /twitter/_shrink/shrunk-twitter-index
|
|
|
|
|
----
|
|
|
|
|
// TEST[s/^/PUT twitter\n{"settings":{"index.number_of_shards":2,"blocks.write":true}}\n/]
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
[[shrink-index-api-request]]
|
|
|
|
|
==== {api-request-title}
|
|
|
|
|
|
|
|
|
|
`POST /<index>/_shrink/<target-index>`
|
|
|
|
|
|
|
|
|
|
`PUT /<index>/_shrink/<target-index>`
|
|
|
|
|
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
[[shrink-index-api-prereqs]]
|
|
|
|
|
==== {api-prereq-title}
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
Before you can shrink an index:
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
* The index must be read-only.
|
2020-07-22 14:06:40 -04:00
|
|
|
|
* All primary shards for the index must reside on the same node.
|
|
|
|
|
* The index must have a `green` <<cluster-health,health status>>.
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
2020-07-22 14:06:40 -04:00
|
|
|
|
To make shard allocation easier, we recommend you also remove the index's
|
|
|
|
|
replica shards. You can later re-add replica shards as part of the shrink
|
|
|
|
|
operation.
|
|
|
|
|
|
|
|
|
|
You can use the following <<indices-update-settings,update index settings API>>
|
|
|
|
|
request to remove an index's replica shards, relocates the index's remaining
|
|
|
|
|
shards to the same node, and make the index read-only.
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
|
[source,console]
|
2016-05-31 04:41:44 -04:00
|
|
|
|
--------------------------------------------------
|
2016-06-03 03:50:51 -04:00
|
|
|
|
PUT /my_source_index/_settings
|
|
|
|
|
{
|
|
|
|
|
"settings": {
|
2020-07-22 14:06:40 -04:00
|
|
|
|
"index.number_of_replicas": 0, <1>
|
|
|
|
|
"index.routing.allocation.require._name": "shrink_node_name", <2>
|
|
|
|
|
"index.blocks.write": true <3>
|
2016-06-03 03:50:51 -04:00
|
|
|
|
}
|
|
|
|
|
}
|
2016-05-31 04:41:44 -04:00
|
|
|
|
--------------------------------------------------
|
2018-05-14 12:22:35 -04:00
|
|
|
|
// TEST[s/^/PUT my_source_index\n{"settings":{"index.number_of_shards":2}}\n/]
|
2019-09-06 11:31:13 -04:00
|
|
|
|
|
2020-07-22 14:06:40 -04:00
|
|
|
|
<1> Removes replica shards for the index.
|
|
|
|
|
<2> Relocates the index's shards to the `shrink_node_name` node.
|
|
|
|
|
See <<shard-allocation-filtering>>.
|
|
|
|
|
<3> Prevents write operations to this index. Metadata changes, such as deleting
|
|
|
|
|
the index, are still allowed.
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2016-06-03 03:50:51 -04:00
|
|
|
|
It can take a while to relocate the source index. Progress can be tracked
|
|
|
|
|
with the <<cat-recovery,`_cat recovery` API>>, or the <<cluster-health,
|
|
|
|
|
`cluster health` API>> can be used to wait until all shards have relocated
|
2016-08-31 11:58:19 -04:00
|
|
|
|
with the `wait_for_no_relocating_shards` parameter.
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
|
|
|
|
[[shrink-index-api-desc]]
|
|
|
|
|
==== {api-description-title}
|
|
|
|
|
|
|
|
|
|
The shrink index API allows you to shrink an existing index into a new index
|
|
|
|
|
with fewer primary shards. The requested number of primary shards in the target index
|
|
|
|
|
must be a factor of the number of shards in the source index. For example an index with
|
|
|
|
|
`8` primary shards can be shrunk into `4`, `2` or `1` primary shards or an index
|
|
|
|
|
with `15` primary shards can be shrunk into `5`, `3` or `1`. If the number
|
|
|
|
|
of shards in the index is a prime number it can only be shrunk into a single
|
|
|
|
|
primary shard. Before shrinking, a (primary or replica) copy of every shard
|
|
|
|
|
in the index must be present on the same node.
|
|
|
|
|
|
2020-06-22 08:29:26 -04:00
|
|
|
|
The current write index on a data stream cannot be shrunk. In order to shrink
|
|
|
|
|
the current write index, the data stream must first be
|
|
|
|
|
<<rollover-data-stream-ex,rolled over>> so that a new write index is created
|
|
|
|
|
and then the previous write index can be shrunk.
|
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
[[how-shrink-works]]
|
|
|
|
|
===== How shrinking works
|
|
|
|
|
|
|
|
|
|
A shrink operation:
|
|
|
|
|
|
|
|
|
|
. Creates a new target index with the same definition as the source
|
|
|
|
|
index, but with a smaller number of primary shards.
|
|
|
|
|
|
|
|
|
|
. Hard-links segments from the source index into the target index. (If
|
|
|
|
|
the file system doesn't support hard-linking, then all segments are copied
|
2020-06-22 08:29:26 -04:00
|
|
|
|
into the new index, which is a much more time consuming process. Also if using
|
|
|
|
|
multiple data paths, shards on different data paths require a full copy of
|
2019-10-04 14:00:18 -04:00
|
|
|
|
segment files if they are not on the same disk since hardlinks don’t work across
|
|
|
|
|
disks)
|
|
|
|
|
|
|
|
|
|
. Recovers the target index as though it were a closed index which
|
|
|
|
|
had just been re-opened.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[[_shrinking_an_index]]
|
|
|
|
|
===== Shrink an index
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
|
|
|
|
To shrink `my_source_index` into a new index called `my_target_index`, issue
|
|
|
|
|
the following request:
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
|
[source,console]
|
2016-05-31 04:41:44 -04:00
|
|
|
|
--------------------------------------------------
|
2019-10-04 14:00:18 -04:00
|
|
|
|
POST /my_source_index/_shrink/my_target_index
|
2018-05-13 10:30:05 -04:00
|
|
|
|
{
|
|
|
|
|
"settings": {
|
|
|
|
|
"index.routing.allocation.require._name": null, <1>
|
|
|
|
|
"index.blocks.write": null <2>
|
|
|
|
|
}
|
|
|
|
|
}
|
2016-05-31 04:41:44 -04:00
|
|
|
|
--------------------------------------------------
|
2016-09-02 18:22:30 -04:00
|
|
|
|
// TEST[continued]
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2018-05-13 10:30:05 -04:00
|
|
|
|
<1> Clear the allocation requirement copied from the source index.
|
|
|
|
|
<2> Clear the index write block copied from the source index.
|
|
|
|
|
|
2016-06-03 03:50:51 -04:00
|
|
|
|
The above request returns immediately once the target index has been added to
|
|
|
|
|
the cluster state -- it doesn't wait for the shrink operation to start.
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2016-06-03 03:50:51 -04:00
|
|
|
|
[IMPORTANT]
|
|
|
|
|
=====================================
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2016-06-07 04:06:41 -04:00
|
|
|
|
Indices can only be shrunk if they satisfy the following requirements:
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2016-06-07 04:06:41 -04:00
|
|
|
|
* the target index must not exist
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2016-06-07 04:06:41 -04:00
|
|
|
|
* The index must have more primary shards than the target index.
|
|
|
|
|
|
|
|
|
|
* The number of primary shards in the target index must be a factor of the
|
2016-07-05 07:34:58 -04:00
|
|
|
|
number of primary shards in the source index. The source index must have
|
|
|
|
|
more primary shards than the target index.
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
|
|
|
|
* The index must not contain more than `2,147,483,519` documents in total
|
2016-06-07 04:06:41 -04:00
|
|
|
|
across all shards that will be shrunk into a single shard on the target index
|
|
|
|
|
as this is the maximum number of docs that can fit into a single shard.
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
|
|
|
|
* The node handling the shrink process must have sufficient free disk space to
|
|
|
|
|
accommodate a second copy of the existing index.
|
|
|
|
|
|
|
|
|
|
=====================================
|
|
|
|
|
|
|
|
|
|
The `_shrink` API is similar to the <<indices-create-index, `create index` API>>
|
|
|
|
|
and accepts `settings` and `aliases` parameters for the target index:
|
|
|
|
|
|
2019-09-06 11:31:13 -04:00
|
|
|
|
[source,console]
|
2016-06-03 03:50:51 -04:00
|
|
|
|
--------------------------------------------------
|
2019-10-04 14:00:18 -04:00
|
|
|
|
POST /my_source_index/_shrink/my_target_index
|
2016-06-03 03:50:51 -04:00
|
|
|
|
{
|
|
|
|
|
"settings": {
|
|
|
|
|
"index.number_of_replicas": 1,
|
2016-06-07 04:06:41 -04:00
|
|
|
|
"index.number_of_shards": 1, <1>
|
|
|
|
|
"index.codec": "best_compression" <2>
|
2016-06-03 03:50:51 -04:00
|
|
|
|
},
|
|
|
|
|
"aliases": {
|
|
|
|
|
"my_search_indices": {}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
--------------------------------------------------
|
2018-05-14 12:22:35 -04:00
|
|
|
|
// TEST[s/^/PUT my_source_index\n{"settings": {"index.number_of_shards":5,"index.blocks.write": true}}\n/]
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
2016-06-07 04:06:41 -04:00
|
|
|
|
<1> The number of shards in the target index. This must be a factor of the
|
|
|
|
|
number of shards in the source index.
|
|
|
|
|
<2> Best compression will only take affect when new writes are made to the
|
2016-06-03 03:50:51 -04:00
|
|
|
|
index, such as when <<indices-forcemerge,force-merging>> the shard to a single
|
|
|
|
|
segment.
|
|
|
|
|
|
2016-06-07 04:06:41 -04:00
|
|
|
|
|
2018-04-30 07:31:36 -04:00
|
|
|
|
NOTE: Mappings may not be specified in the `_shrink` request.
|
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
|
|
|
|
[[monitor-shrink]]
|
|
|
|
|
===== Monitor the shrink process
|
2016-06-03 03:50:51 -04:00
|
|
|
|
|
|
|
|
|
The shrink process can be monitored with the <<cat-recovery,`_cat recovery`
|
|
|
|
|
API>>, or the <<cluster-health, `cluster health` API>> can be used to wait
|
|
|
|
|
until all primary shards have been allocated by setting the `wait_for_status`
|
|
|
|
|
parameter to `yellow`.
|
|
|
|
|
|
|
|
|
|
The `_shrink` API returns as soon as the target index has been added to the
|
|
|
|
|
cluster state, before any shards have been allocated. At this point, all
|
|
|
|
|
shards are in the state `unassigned`. If, for any reason, the target index
|
|
|
|
|
can't be allocated on the shrink node, its primary shard will remain
|
|
|
|
|
`unassigned` until it can be allocated on that node.
|
|
|
|
|
|
|
|
|
|
Once the primary shard is allocated, it moves to state `initializing`, and the
|
|
|
|
|
shrink process begins. When the shrink operation completes, the shard will
|
|
|
|
|
become `active`. At that point, Elasticsearch will try to allocate any
|
|
|
|
|
replicas and may decide to relocate the primary shard to another node.
|
2016-05-31 04:41:44 -04:00
|
|
|
|
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
|
|
|
|
[[shrink-wait-active-shards]]
|
|
|
|
|
===== Wait for active shards
|
2016-08-02 09:15:01 -04:00
|
|
|
|
|
2016-09-02 18:22:30 -04:00
|
|
|
|
Because the shrink operation creates a new index to shrink the shards to,
|
|
|
|
|
the <<create-index-wait-for-active-shards,wait for active shards>> setting
|
2016-08-02 09:15:01 -04:00
|
|
|
|
on index creation applies to the shrink index action as well.
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[[shrink-index-api-path-params]]
|
|
|
|
|
==== {api-path-parms-title}
|
|
|
|
|
|
|
|
|
|
`<index>`::
|
|
|
|
|
(Required, string)
|
|
|
|
|
Name of the source index to shrink.
|
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=target-index]
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
|
|
|
|
[[shrink-index-api-query-params]]
|
|
|
|
|
==== {api-query-parms-title}
|
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=timeoutparms]
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
[[shrink-index-api-request-body]]
|
|
|
|
|
==== {api-request-body-title}
|
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=target-index-aliases]
|
2019-10-04 14:00:18 -04:00
|
|
|
|
|
2020-06-01 19:42:53 -04:00
|
|
|
|
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=target-index-settings]
|