OpenSearch/plugins
Simon Willnauer 502a775a7c Add primitive to shrink an index into a single shard (#18270)
This adds a low level primitive operations to shrink an existing
index into a new index with a single shard. This primitive expects
all shards of the source index to allocated on a single node. Once the target index is initializing on the shrink node it takes a snapshot of the source index shards and copies all files into the target indices data folder. An [optimization](https://issues.apache.org/jira/browse/LUCENE-7300) coming in Lucene 6.1 will also allow for optional constant time copy if hard-links are supported by the filesystem. All mappings are merged into the new indexes metadata once the snapshots have been taken on the merge node.

To shrink an existing index all shards must be moved to a single node (one instance of each shard) and the index must be read-only:

```BASH
$ curl -XPUT 'http://localhost:9200/logs/_settings' -d '{
    "settings" : {
        "index.routing.allocation.require._name" : "shrink_node_name",
        "index.blocks.write" : true 
    }
}
```
once all shards are started on the shrink node. the new index can be created via:

```BASH
$ curl -XPUT 'http://localhost:9200/logs/_shrink/logs_single_shard' -d '{
    "settings" : {
        "index.codec" : "best_compression",
        "index.number_of_replicas" : 1
    }
}'
```

This API will perform all needed check before the new index is created and selects the shrink node based on the allocation of the source index. This call returns immediately, to monitor shrink progress the recovery API should be used since all copy operations are reflected in the recovery API with byte copy progress etc.

The shrink operation does not modify the source index, if a shrink operation should
be canceled or if the shrink failed, the target index can simply be deleted and
all resources are released.
2016-05-31 10:41:44 +02:00
..
analysis-icu Remove deprecation suppression 2016-05-25 17:15:36 -04:00
analysis-kuromoji Analyze API : Rename filters/token_filters/char_filter to filter/token_filter/char_filter 2016-04-21 18:05:11 +09:00
analysis-phonetic Upgrade Lucene 6 Release 2016-04-11 16:50:04 -05:00
analysis-smartcn Upgrade Lucene 6 Release 2016-04-11 16:50:04 -05:00
analysis-stempel Analyze API : Rename filters/token_filters/char_filter to filter/token_filter/char_filter 2016-04-21 18:05:11 +09:00
discovery-azure Introduce dedicated master nodes in testing infrastructure (#18514) 2016-05-27 08:44:20 +02:00
discovery-ec2 Introduce dedicated master nodes in testing infrastructure (#18514) 2016-05-27 08:44:20 +02:00
discovery-gce Introduce dedicated master nodes in testing infrastructure (#18514) 2016-05-27 08:44:20 +02:00
ingest-attachment Remove custom Base64 implementation. #18413 2016-05-23 11:32:42 +02:00
ingest-geoip [TEST] Add missing sort processor to tests, continued 2016-05-17 16:39:53 -04:00
jvm-example Remove last vestigates of /bin/sh shebangs 2016-05-18 11:03:00 -04:00
lang-javascript Remove LeafSearchScript.runAsFloat(): Nothing calls it. 2016-05-15 22:59:28 -04:00
lang-python Remove LeafSearchScript.runAsFloat(): Nothing calls it. 2016-05-15 22:59:28 -04:00
mapper-attachments Remove deprecation suppression 2016-05-25 17:15:36 -04:00
mapper-murmur3 Lessen leniency of the query dsl. #18276 2016-05-16 17:37:00 +02:00
mapper-size Disable XContent auto closing of object and arrays 2016-05-25 16:46:09 +02:00
repository-azure Introduce dedicated master nodes in testing infrastructure (#18514) 2016-05-27 08:44:20 +02:00
repository-gcs Change BlobPath.buildAsString() method 2016-05-23 10:50:40 +02:00
repository-hdfs Remove Settings.settingsBuilder. 2016-04-08 18:10:02 +02:00
repository-s3 Change BlobPath.buildAsString() method 2016-05-23 10:50:40 +02:00
store-smb Add primitive to shrink an index into a single shard (#18270) 2016-05-31 10:41:44 +02:00
build.gradle Revert "Build: Switch to maven-publish plugin" 2016-03-18 17:22:25 -07:00