6575 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe
d509834c00 Fixes checkstyle error 2018-04-13 14:39:37 +01:00
Colin Goodheart-Smithe
af6456983f Moves registing of NamedWriteables to plugin:core
This is needed so that clients have the ability to serialise and deserialise the objects it creates and gets back.
2018-04-13 14:23:54 +01:00
Colin Goodheart-Smithe
bf3db3f1f7 Adds phase and action change times on state transition 2018-04-13 13:38:44 +01:00
Colin Goodheart-Smithe
d17886f1b0 Fixes checkstyle error 2018-04-13 10:44:45 +01:00
Tal Levy
f04c95f4c2 change policy logs to be DEBUG 2018-04-12 15:40:20 -07:00
Colin Goodheart-Smithe
644d82e165 Renames AliasStep to ShrinkSetAliasStep 2018-04-12 18:29:12 +01:00
Colin Goodheart-Smithe
77be66b8b9 Adds shrink check that original index is deleted 2018-04-12 18:13:57 +01:00
Colin Goodheart-Smithe
59e0979b92 Fixes shrunk shards allocation check step
- Renames EnoughShardsWaitStep to ShrunkShardsAllocatedStep
- Changes ShrunkShardsAllocatedStep to check the shards of the shrunken index rather than the current one
- shrink index prefix is now passed into the steps of the shrink aciton
- Related Test Changes
2018-04-12 18:06:04 +01:00
Tal Levy
34e15011a1 fix ilm rest-tests to reflect latest get_policy changes 2018-04-12 07:47:57 -07:00
Colin Goodheart-Smithe
088894019e Tidies up unit tests
Also fixes failing EnoughShardsWaitStep
2018-04-12 12:37:36 +01:00
Colin Goodheart-Smithe
1c2e7bc431 Increases iterations on step equality tests
To make it in line with the fact that other equality tests use multiple iterations
2018-04-12 12:23:35 +01:00
Colin Goodheart-Smithe
e8cb6419f8 Removes unused imports 2018-04-12 12:16:43 +01:00
Colin Goodheart-Smithe
7b247fe41e Fixes compile errors from TimeValue serialisation 2018-04-12 12:15:36 +01:00
Tal Levy
20b6aaf6b6 migrate step tests to extend AbstractStepTestCase 2018-04-11 18:56:53 -07:00
Tal Levy
c1b00d0154 cleanup tests
- make DeleteActionTests pass
- remove getType from LifecycleType
2018-04-11 18:28:46 -07:00
Tal Levy
7f4801f03c clean up PhaseTests 2018-04-11 18:16:28 -07:00
Tal Levy
73ee5a4aa0 introduce refactored ShrinkAction
this resulted in a few new steps being introduced

- AliasStep
- EnoughShardsWaitStep
- ShrinkStep
- ShrunkenIndexCheckStep
2018-04-11 18:10:13 -07:00
Colin Goodheart-Smithe
20485cf7fb Adds more missing equality tests
Specifically for the initialPolicyContextStep and the PhaseAfterStep
2018-04-11 13:24:31 +01:00
Colin Goodheart-Smithe
482de191f2 Tidies up some of the unit tests
Adds some missing tests including checking the hashcode and equals methods of `DeleteStep`, `StepKey`, and `TerminalPolicyStep` as well as adding a test for `DeleteAction.toSteps()`
2018-04-11 12:38:55 +01:00
Colin Goodheart-Smithe
66b4e8c4aa Removes unused imports 2018-04-11 10:47:07 +01:00
Tal Levy
ef34f982f1 refactor a collection of step-related things + shrink_step
- added ShrinkStep/Tests
- AsyncActionStep now passes in IndexMetaData instead of Index
- Delete usage of ClusterStateActionStep
- with ClusterStateActionStep gone, InitializePolicyContextStep
  is the only other ClusterState-nonWait step
- Migrate setting-updates to UpdateSettingsStep
2018-04-10 17:13:10 -07:00
Colin Goodheart-Smithe
8d91f197d4 Changes update replica and alllocation to async steps
Also renames EnoughShardsWaitStep to ReplicasAllocatedStep, removes it from the allocate action and adds a check that th number of replicas in the cluster state is correct to it.
2018-04-10 16:20:29 +01:00
Colin Goodheart-Smithe
aa198b637e Cleans up warnings 2018-04-10 09:53:57 +01:00
Colin Goodheart-Smithe
c24dc07346 Removes unused and unneeded code
Various classes had some code that was not used and is not going to be needed so this change cleans up those classes so we don’t have dead code hanging around
2018-04-10 09:28:03 +01:00
Colin Goodheart-Smithe
b8d573772f Fixes compilation issue in test 2018-04-10 09:10:52 +01:00
Tal Levy
f5b23df349 add more force-merge related steps and tests
The force-merge is an a TODO state due to the
unresolved issue around best_compression.

- updated ReadOnlyStep with tests
- implemented an update to the ForceMergeAction
- added UpdateBestCompressionSettingsStep
- added tests for SegmentCountStep
2018-04-09 16:28:25 -07:00
Colin Goodheart-Smithe
06e969b430 Adds rollover step and implements RolloverAction.toSteps 2018-04-09 15:55:06 +01:00
Colin Goodheart-Smithe
54c6d280a5 Adds Steps for ReplicasAction
Also adds tests for those steps and ReplicasAction.toSteps
2018-04-09 14:43:57 +01:00
Colin Goodheart-Smithe
2eb8fd9336 Adds tests for AllocateAction and its steps
Also tweaks some of the code in the steps of the allocate action
2018-04-09 13:59:25 +01:00
Tal Levy
47ed2e5b23 add more tests 2018-04-05 18:00:41 -07:00
Tal Levy
b58159ddad add more tests 2018-04-05 17:43:32 -07:00
Tal Levy
228fc8c842 add some stuff 2018-04-05 17:21:44 -07:00
Tal Levy
c0acea3721 :check yourself greeny 2018-04-05 13:16:19 -07:00
Tal Levy
1a2afac98b fix tests 2018-04-05 12:43:10 -07:00
Colin Goodheart-Smithe
e3e669067c Adds loggic for TerminalPolicyStep 2018-04-05 17:58:22 +01:00
Colin Goodheart-Smithe
05c2c5655b replaces cause with boolean 2018-04-05 17:53:15 +01:00
Colin Goodheart-Smithe
ed260a0430 Removes Cause stuff 2018-04-05 17:10:56 +01:00
Martijn van Groningen
d77f756f5c ccr: use indices stats api to fetch global checkpoint of the follower shards and
keep track of shard follow stats inside shard follow stats' node task instead of persistent task status.

By maintaining the shard follow stats inside its node task the stats update is quicker as
no cluster state update is required. The stats are now transient; meaning if the task
is going to run a different node then the stats are gone too. Currently only the processed
global checkpoint is being tracked and this is being restored when a shard follow node task
starts via the indices stats api (the reason of the first change of this change). Other stats
that we may add in the future (like fetch_time, see: https://gist.github.com/s1monw/dba13daf8493bf48431b72365e110717)
it is ok if we start from zero in case a shard follow task moves to another node.
2018-04-05 14:52:20 +02:00
Tal Levy
907586c9e4 add MoveToNextStepUpdateTaskTests 2018-04-04 20:39:00 -07:00
Tal Levy
55d2a62fec add tests for ExecuteStepsUpdateTask 2018-04-04 15:32:36 -07:00
Colin Goodheart-Smithe
a8dc8a7c98 Adds more unit tests for IndexLifecycleRunner 2018-04-04 15:59:10 +01:00
Colin Goodheart-Smithe
9ef26dbe51 Unit tests static methods in IndeexLifecycleRunner 2018-04-04 13:29:30 +01:00
Tal Levy
47dcc8fe9c more changes + updated api 2018-04-03 17:42:25 -07:00
Tal Levy
1ac1ee413f moar changes 2018-04-02 16:21:48 -07:00
Tal Levy
d2e87a66e5 moar changes
- set `indexSurvives` for Step
- extract the two ClusterStateUpdateTasks to separate classes

- simple Delete Policy works!
2018-03-29 16:23:57 -07:00
Martijn van Groningen
d976fa44e7 Removed LocalCheckpointTracker usage. 2018-03-29 07:41:23 +02:00
Tal Levy
74eaba2078 moar changes from Tal 2018-03-28 17:10:50 -07:00
Colin Goodheart-Smithe
179074d3b0 Refactors steps and adds some execution logic
This is far from complete but its necessary to push so the new step classes can be shared and used.
2018-03-28 16:34:34 +01:00
Martijn van Groningen
a22a7d079d ccr: Added maximum translog limit that a single shard changes response can return.
This limit is based on the number of estimate bytes in each translog
operation that fall between the minimum and maximum request sequence number.

If this limit is met then the shard follow task executor will make sure
that a subsequent shard changes request will be performed to fetch the
remaining translog operations.

This limit is needed in order to protect against returning too many
translog operations in a single shard changes response.

Relates to #2436
2018-03-28 15:49:57 +02:00
Martijn van Groningen
282740610b Fixed test after merging in master branch. 2018-03-28 09:54:41 +02:00