Commit Graph

5061 Commits

Author SHA1 Message Date
Tal Levy 3455445d2c add integration test to indexlifecyclemanagement
This test verifies that we have sufficient failover code so that
a newly elected master re-registers old schedules and fires them off.
All times are relative to the index creation date.
2017-10-30 17:51:01 -07:00
Jason Tedor 769349a9ab Add following engine implementation
This commit is a first step towards a following engine
implementation. Future work will build on this by using this engine to
execute operations on a following engine from another engine (typically
a remote leader engine) that has already assigned sequence numbers to
such operations.

Relates #2776
2017-10-30 13:38:02 -04:00
Tal Levy de4d87c6e6 test 2017-10-26 15:32:08 -07:00
Martijn van Groningen 41c3dc91c1 Changed the shard changes api to use get threadpool instead of management threadpool. 2017-10-17 10:02:28 +02:00
Martijn van Groningen 2663b5ef39 remove unused import 2017-10-16 10:36:22 +02:00
Jason Tedor 85c1935d9d Increase generations in operations between test
This commit sets an index setting for the size of a translog generation
and increases the number of documents indexed to increase the chance of
multiple generations being present when testing getting operations
between two sequence numbers.
2017-10-16 08:59:18 +02:00
Jason Tedor bcd61bfca5 Fix off-by-one error in shard changes action test
This commit fixes an off-by-one error in the shard changes action test
for getting operations between two sequence numbers. The off-by-one
error arises because sequence numbers are indexed from zero, so if N
documents are indexed then the maximum sequence number starting from
zero would be N - 1.
2017-10-16 08:54:04 +02:00
Martijn van Groningen 1f495f59a1 [CCR] Add an internal api to read translog operations between a sequence number range (#2563)
* xdcr: Add an internal api to read translog operations between a sequence number range.

This api will be used later by the persistent task for the following index to pull data from the leader index.

The persistent task can fetch the global checkpoint from the shard stats for each primary shard of the leader index.
Based on the global checkpoint of the primary shards of the following index, the persistent task can send several
calls to the internal api added in this commit to replicate changes from follow index to leader index in a batched manner.
2017-10-16 07:38:03 +02:00
Colin Goodheart-Smithe 4aaec2ef81 Creates a new X-Pack feature for index lifecycle
Feature consists of a shell of a persistant task which will later be used to inspect the index settings and apply curator like changes to the index (move from hot to warm, rollover, shrink etc.)
2017-10-12 16:22:42 +01:00
Jason Tedor 3033aba67b Introduce pluggable engine factory for CCR
This commit utilizes the pluggable engine factory feature in core to
introduce a pluggable engine factory for XDCR. For now this is only a
skeleton implementation to proof out the pluggable engine factory
concept. Future work will implement a genuine following engine for XDCR.

Relates #2655
2017-10-12 10:52:33 -04:00
Jason Tedor 063a76c3db Introduce CCR container class
This commit introduces the container class for CCR functionality. Future
work will expose more specific CCR functionality to the X-Pack plugin
through this class.

Relates #2704
2017-10-06 14:46:39 -04:00