parent
a46f312ded
commit
bb82addf35
|
@ -4,6 +4,58 @@
|
||||||
<titleabbrev>Split index</titleabbrev>
|
<titleabbrev>Split index</titleabbrev>
|
||||||
++++
|
++++
|
||||||
|
|
||||||
|
Splits an existing index into a new index with more primary shards.
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
----
|
||||||
|
POST /twitter/_split/split-twitter-index
|
||||||
|
{
|
||||||
|
"settings": {
|
||||||
|
"index.number_of_shards": 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
----
|
||||||
|
// TEST[s/^/PUT twitter\n{"settings":{"blocks.write":true}}\n/]
|
||||||
|
|
||||||
|
|
||||||
|
[[split-index-api-request]]
|
||||||
|
==== {api-request-title}
|
||||||
|
|
||||||
|
`POST /<index>/_shrink/<target-index>`
|
||||||
|
|
||||||
|
`PUT /<index>/_shrink/<target-index>`
|
||||||
|
|
||||||
|
|
||||||
|
[[split-index-api-prereqs]]
|
||||||
|
==== {api-prereq-title}
|
||||||
|
|
||||||
|
|
||||||
|
Before you can split an index:
|
||||||
|
|
||||||
|
* The index must be read-only.
|
||||||
|
* The <<cluster-health, cluster health>> status must be green.
|
||||||
|
|
||||||
|
You can do make an index read-only
|
||||||
|
with the following request:
|
||||||
|
|
||||||
|
[source,console]
|
||||||
|
--------------------------------------------------
|
||||||
|
PUT /my_source_index/_settings
|
||||||
|
{
|
||||||
|
"settings": {
|
||||||
|
"index.blocks.write": true <1>
|
||||||
|
}
|
||||||
|
}
|
||||||
|
--------------------------------------------------
|
||||||
|
// TEST[s/^/PUT my_source_index\n/]
|
||||||
|
|
||||||
|
<1> Prevents write operations to this index while still allowing metadata
|
||||||
|
changes like deleting the index.
|
||||||
|
|
||||||
|
|
||||||
|
[[split-index-api-desc]]
|
||||||
|
==== {api-description-title}
|
||||||
|
|
||||||
The split index API allows you to split an existing index into a new index,
|
The split index API allows you to split an existing index into a new index,
|
||||||
where each original primary shard is split into two or more primary shards in
|
where each original primary shard is split into two or more primary shards in
|
||||||
the new index.
|
the new index.
|
||||||
|
@ -34,27 +86,28 @@ index may by split into an arbitrary number of shards greater than 1. The
|
||||||
properties of the default number of routing shards will then apply to the
|
properties of the default number of routing shards will then apply to the
|
||||||
newly split index.
|
newly split index.
|
||||||
|
|
||||||
[float]
|
|
||||||
==== How does splitting work?
|
|
||||||
|
|
||||||
Splitting works as follows:
|
[[how-split-works]]
|
||||||
|
===== How splitting works
|
||||||
|
|
||||||
* First, it creates a new target index with the same definition as the source
|
A split operation:
|
||||||
|
|
||||||
|
. Creates a new target index with the same definition as the source
|
||||||
index, but with a larger number of primary shards.
|
index, but with a larger number of primary shards.
|
||||||
|
|
||||||
* Then it hard-links segments from the source index into the target index. (If
|
. Hard-links segments from the source index into the target index. (If
|
||||||
the file system doesn't support hard-linking, then all segments are copied
|
the file system doesn't support hard-linking, then all segments are copied
|
||||||
into the new index, which is a much more time consuming process.)
|
into the new index, which is a much more time consuming process.)
|
||||||
|
|
||||||
* Once the low level files are created all documents will be `hashed` again to delete
|
. Hashes all documents again, after low level files are created, to delete
|
||||||
documents that belong to a different shard.
|
documents that belong to a different shard.
|
||||||
|
|
||||||
* Finally, it recovers the target index as though it were a closed index which
|
. Recovers the target index as though it were a closed index which
|
||||||
had just been re-opened.
|
had just been re-opened.
|
||||||
|
|
||||||
[float]
|
|
||||||
[[incremental-resharding]]
|
[[incremental-resharding]]
|
||||||
==== Why doesn't Elasticsearch support incremental resharding?
|
===== Why doesn't Elasticsearch support incremental resharding?
|
||||||
|
|
||||||
Going from `N` shards to `N+1` shards, aka. incremental resharding, is indeed a
|
Going from `N` shards to `N+1` shards, aka. incremental resharding, is indeed a
|
||||||
feature that is supported by many key-value stores. Adding a new shard and
|
feature that is supported by many key-value stores. Adding a new shard and
|
||||||
|
@ -83,49 +136,16 @@ covers both the old and the new index for read operations. Assuming that the
|
||||||
old and new indices have respectively +M+ and +N+ shards, this has no overhead
|
old and new indices have respectively +M+ and +N+ shards, this has no overhead
|
||||||
compared to searching an index that would have +M+N+ shards.
|
compared to searching an index that would have +M+N+ shards.
|
||||||
|
|
||||||
[float]
|
|
||||||
==== Preparing an index for splitting
|
|
||||||
|
|
||||||
Create a new index:
|
[[split-index]]
|
||||||
|
===== Split an index
|
||||||
[source,console]
|
|
||||||
--------------------------------------------------
|
|
||||||
PUT my_source_index
|
|
||||||
{
|
|
||||||
"settings": {
|
|
||||||
"index.number_of_shards" : 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
--------------------------------------------------
|
|
||||||
|
|
||||||
In order to split an index, the index must be marked as read-only,
|
|
||||||
and have <<cluster-health,health>> `green`.
|
|
||||||
|
|
||||||
This can be achieved with the following request:
|
|
||||||
|
|
||||||
[source,console]
|
|
||||||
--------------------------------------------------
|
|
||||||
PUT /my_source_index/_settings
|
|
||||||
{
|
|
||||||
"settings": {
|
|
||||||
"index.blocks.write": true <1>
|
|
||||||
}
|
|
||||||
}
|
|
||||||
--------------------------------------------------
|
|
||||||
// TEST[continued]
|
|
||||||
|
|
||||||
<1> Prevents write operations to this index while still allowing metadata
|
|
||||||
changes like deleting the index.
|
|
||||||
|
|
||||||
[float]
|
|
||||||
==== Splitting an index
|
|
||||||
|
|
||||||
To split `my_source_index` into a new index called `my_target_index`, issue
|
To split `my_source_index` into a new index called `my_target_index`, issue
|
||||||
the following request:
|
the following request:
|
||||||
|
|
||||||
[source,console]
|
[source,console]
|
||||||
--------------------------------------------------
|
--------------------------------------------------
|
||||||
POST my_source_index/_split/my_target_index
|
POST /my_source_index/_split/my_target_index
|
||||||
{
|
{
|
||||||
"settings": {
|
"settings": {
|
||||||
"index.number_of_shards": 2
|
"index.number_of_shards": 2
|
||||||
|
@ -159,7 +179,7 @@ and accepts `settings` and `aliases` parameters for the target index:
|
||||||
|
|
||||||
[source,console]
|
[source,console]
|
||||||
--------------------------------------------------
|
--------------------------------------------------
|
||||||
POST my_source_index/_split/my_target_index
|
POST /my_source_index/_split/my_target_index
|
||||||
{
|
{
|
||||||
"settings": {
|
"settings": {
|
||||||
"index.number_of_shards": 5 <1>
|
"index.number_of_shards": 5 <1>
|
||||||
|
@ -177,8 +197,9 @@ POST my_source_index/_split/my_target_index
|
||||||
|
|
||||||
NOTE: Mappings may not be specified in the `_split` request.
|
NOTE: Mappings may not be specified in the `_split` request.
|
||||||
|
|
||||||
[float]
|
|
||||||
==== Monitoring the split process
|
[[monitor-split]]
|
||||||
|
==== Monitor the split process
|
||||||
|
|
||||||
The split process can be monitored with the <<cat-recovery,`_cat recovery`
|
The split process can be monitored with the <<cat-recovery,`_cat recovery`
|
||||||
API>>, or the <<cluster-health, `cluster health` API>> can be used to wait
|
API>>, or the <<cluster-health, `cluster health` API>> can be used to wait
|
||||||
|
@ -196,9 +217,36 @@ split process begins. When the split operation completes, the shard will
|
||||||
become `active`. At that point, Elasticsearch will try to allocate any
|
become `active`. At that point, Elasticsearch will try to allocate any
|
||||||
replicas and may decide to relocate the primary shard to another node.
|
replicas and may decide to relocate the primary shard to another node.
|
||||||
|
|
||||||
[float]
|
|
||||||
==== Wait For Active Shards
|
[[split-wait-active-shards]]
|
||||||
|
==== Wait for active shards
|
||||||
|
|
||||||
Because the split operation creates a new index to split the shards to,
|
Because the split operation creates a new index to split the shards to,
|
||||||
the <<create-index-wait-for-active-shards,wait for active shards>> setting
|
the <<create-index-wait-for-active-shards,wait for active shards>> setting
|
||||||
on index creation applies to the split index action as well.
|
on index creation applies to the split index action as well.
|
||||||
|
|
||||||
|
|
||||||
|
[[split-index-api-path-params]]
|
||||||
|
==== {api-path-parms-title}
|
||||||
|
|
||||||
|
`<index>`::
|
||||||
|
(Required, string)
|
||||||
|
Name of the source index to split.
|
||||||
|
|
||||||
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=target-index]
|
||||||
|
|
||||||
|
|
||||||
|
[[split-index-api-query-params]]
|
||||||
|
==== {api-query-parms-title}
|
||||||
|
|
||||||
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=wait_for_active_shards]
|
||||||
|
|
||||||
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=timeoutparms]
|
||||||
|
|
||||||
|
|
||||||
|
[[split-index-api-request-body]]
|
||||||
|
==== {api-request-body-title}
|
||||||
|
|
||||||
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=target-index-aliases]
|
||||||
|
|
||||||
|
include::{docdir}/rest-api/common-parms.asciidoc[tag=target-index-settings]
|
||||||
|
|
Loading…
Reference in New Issue