3087 Commits

Author SHA1 Message Date
Ryan Ernst
9a45662309 Convert license header check to use the new Elastic License
This commit adapts the license headers check to no longer look for the
ealsticsearch confidential license, but instead to look for the new
Elastic License header.
2018-04-20 15:33:59 -07:00
Jason Tedor
e64e6d8996 Add distribution flavor to startup scripts
This commit adds the distribution flavor (default versus oss) to the
build process which is passed through the startup scripts to
Elasticsearch. This change will be used to customize the message on
attempting to install/remove x-pack based on the distribution flavor.
2018-04-20 15:33:58 -07:00
Ryan Ernst
fab5e21e7d Build: Split distributions into oss and default
This commit makes x-pack a module and adds it to the default
distrubtion. It also creates distributions for zip, tar, deb and rpm
which contain only oss code.
2018-04-20 15:33:57 -07:00
Ryan Ernst
cb56bf49d5 Fixup build after opening x-pack 2018-04-20 15:33:46 -07:00
Ryan Ernst
2efd22454a Migrate x-pack-elasticsearch source to elasticsearch 2018-04-20 15:29:54 -07:00
Martijn van Groningen
9b9d0f9057 Enabled licence header check and fixed unchecked casts. (#4408) 2018-04-20 11:15:52 +02:00
Martijn van Groningen
cfd7847628 fixed issues after merging in master 2018-04-20 07:59:13 +02:00
Nhat Nguyen
f97aec7b8b Sibling of enforce access to translog via engine
Since elastic/elasticsearch#29542, we no longer expose translog instance
but only provide creating translog snapshot method. This commit adapts
that change in CCR branch.

Relates elastic/elasticsearch#29542
2018-04-18 11:54:00 -04:00
Martijn van Groningen
56ca59a513 Add the ability to the follow index to follow an index in a remote cluster.
The follow index api completely reuses CCS infrastructure that was exposed via:
https://github.com/elastic/elasticsearch/pull/29495

This means that the leader index parameter support the same ccs index
to indicate that an index resides in a different cluster.

I also added a qa module that smoke tests the cross cluster nature of ccr.
The idea is that this test just verifies that ccr can read data from a
remote leader index and that is it, no crazy randomization or indirectly
testing other features.
2018-04-17 07:36:40 +02:00
Tal Levy
689d628d67 move phase-after steps to have the previous phase's phase in its key (#4387)
Previously, phase X's `after` step had `X` as its associated phase. This causes confusion because
we have only entered phase `X` once the `after` step is complete. Therefore, this refactor
pushes the after's phase to be associated with the previous phase. This first phase is an exception. 
The first phase's `after` step is associated with the first phase (not some non-existent prior phase).
2018-04-16 21:19:30 -07:00
Martijn van Groningen
c0d42e9cd1 Fixed test 2018-04-16 10:48:46 +02:00
Martijn van Groningen
a94b38b88e Fixed compile errors and test failures after merging master into ccr. 2018-04-13 16:35:09 +02:00
Colin Goodheart-Smithe
d509834c00 Fixes checkstyle error 2018-04-13 14:39:37 +01:00
Colin Goodheart-Smithe
af6456983f Moves registing of NamedWriteables to plugin:core
This is needed so that clients have the ability to serialise and deserialise the objects it creates and gets back.
2018-04-13 14:23:54 +01:00
Colin Goodheart-Smithe
bf3db3f1f7 Adds phase and action change times on state transition 2018-04-13 13:38:44 +01:00
Colin Goodheart-Smithe
d17886f1b0 Fixes checkstyle error 2018-04-13 10:44:45 +01:00
Tal Levy
f04c95f4c2 change policy logs to be DEBUG 2018-04-12 15:40:20 -07:00
Colin Goodheart-Smithe
644d82e165 Renames AliasStep to ShrinkSetAliasStep 2018-04-12 18:29:12 +01:00
Colin Goodheart-Smithe
77be66b8b9 Adds shrink check that original index is deleted 2018-04-12 18:13:57 +01:00
Colin Goodheart-Smithe
59e0979b92 Fixes shrunk shards allocation check step
- Renames EnoughShardsWaitStep to ShrunkShardsAllocatedStep
- Changes ShrunkShardsAllocatedStep to check the shards of the shrunken index rather than the current one
- shrink index prefix is now passed into the steps of the shrink aciton
- Related Test Changes
2018-04-12 18:06:04 +01:00
Tal Levy
34e15011a1 fix ilm rest-tests to reflect latest get_policy changes 2018-04-12 07:47:57 -07:00
Colin Goodheart-Smithe
088894019e Tidies up unit tests
Also fixes failing EnoughShardsWaitStep
2018-04-12 12:37:36 +01:00
Colin Goodheart-Smithe
1c2e7bc431 Increases iterations on step equality tests
To make it in line with the fact that other equality tests use multiple iterations
2018-04-12 12:23:35 +01:00
Colin Goodheart-Smithe
e8cb6419f8 Removes unused imports 2018-04-12 12:16:43 +01:00
Colin Goodheart-Smithe
7b247fe41e Fixes compile errors from TimeValue serialisation 2018-04-12 12:15:36 +01:00
Tal Levy
20b6aaf6b6 migrate step tests to extend AbstractStepTestCase 2018-04-11 18:56:53 -07:00
Tal Levy
c1b00d0154 cleanup tests
- make DeleteActionTests pass
- remove getType from LifecycleType
2018-04-11 18:28:46 -07:00
Tal Levy
7f4801f03c clean up PhaseTests 2018-04-11 18:16:28 -07:00
Tal Levy
73ee5a4aa0 introduce refactored ShrinkAction
this resulted in a few new steps being introduced

- AliasStep
- EnoughShardsWaitStep
- ShrinkStep
- ShrunkenIndexCheckStep
2018-04-11 18:10:13 -07:00
Colin Goodheart-Smithe
20485cf7fb Adds more missing equality tests
Specifically for the initialPolicyContextStep and the PhaseAfterStep
2018-04-11 13:24:31 +01:00
Colin Goodheart-Smithe
482de191f2 Tidies up some of the unit tests
Adds some missing tests including checking the hashcode and equals methods of `DeleteStep`, `StepKey`, and `TerminalPolicyStep` as well as adding a test for `DeleteAction.toSteps()`
2018-04-11 12:38:55 +01:00
Colin Goodheart-Smithe
66b4e8c4aa Removes unused imports 2018-04-11 10:47:07 +01:00
Tal Levy
ef34f982f1 refactor a collection of step-related things + shrink_step
- added ShrinkStep/Tests
- AsyncActionStep now passes in IndexMetaData instead of Index
- Delete usage of ClusterStateActionStep
- with ClusterStateActionStep gone, InitializePolicyContextStep
  is the only other ClusterState-nonWait step
- Migrate setting-updates to UpdateSettingsStep
2018-04-10 17:13:10 -07:00
Colin Goodheart-Smithe
8d91f197d4 Changes update replica and alllocation to async steps
Also renames EnoughShardsWaitStep to ReplicasAllocatedStep, removes it from the allocate action and adds a check that th number of replicas in the cluster state is correct to it.
2018-04-10 16:20:29 +01:00
Colin Goodheart-Smithe
aa198b637e Cleans up warnings 2018-04-10 09:53:57 +01:00
Colin Goodheart-Smithe
c24dc07346 Removes unused and unneeded code
Various classes had some code that was not used and is not going to be needed so this change cleans up those classes so we don’t have dead code hanging around
2018-04-10 09:28:03 +01:00
Colin Goodheart-Smithe
b8d573772f Fixes compilation issue in test 2018-04-10 09:10:52 +01:00
Tal Levy
f5b23df349 add more force-merge related steps and tests
The force-merge is an a TODO state due to the
unresolved issue around best_compression.

- updated ReadOnlyStep with tests
- implemented an update to the ForceMergeAction
- added UpdateBestCompressionSettingsStep
- added tests for SegmentCountStep
2018-04-09 16:28:25 -07:00
Colin Goodheart-Smithe
06e969b430 Adds rollover step and implements RolloverAction.toSteps 2018-04-09 15:55:06 +01:00
Colin Goodheart-Smithe
54c6d280a5 Adds Steps for ReplicasAction
Also adds tests for those steps and ReplicasAction.toSteps
2018-04-09 14:43:57 +01:00
Colin Goodheart-Smithe
2eb8fd9336 Adds tests for AllocateAction and its steps
Also tweaks some of the code in the steps of the allocate action
2018-04-09 13:59:25 +01:00
Tal Levy
47ed2e5b23 add more tests 2018-04-05 18:00:41 -07:00
Tal Levy
b58159ddad add more tests 2018-04-05 17:43:32 -07:00
Tal Levy
228fc8c842 add some stuff 2018-04-05 17:21:44 -07:00
Tal Levy
c0acea3721 :check yourself greeny 2018-04-05 13:16:19 -07:00
Tal Levy
1a2afac98b fix tests 2018-04-05 12:43:10 -07:00
Colin Goodheart-Smithe
e3e669067c Adds loggic for TerminalPolicyStep 2018-04-05 17:58:22 +01:00
Colin Goodheart-Smithe
05c2c5655b replaces cause with boolean 2018-04-05 17:53:15 +01:00
Colin Goodheart-Smithe
ed260a0430 Removes Cause stuff 2018-04-05 17:10:56 +01:00
Martijn van Groningen
d77f756f5c ccr: use indices stats api to fetch global checkpoint of the follower shards and
keep track of shard follow stats inside shard follow stats' node task instead of persistent task status.

By maintaining the shard follow stats inside its node task the stats update is quicker as
no cluster state update is required. The stats are now transient; meaning if the task
is going to run a different node then the stats are gone too. Currently only the processed
global checkpoint is being tracked and this is being restored when a shard follow node task
starts via the indices stats api (the reason of the first change of this change). Other stats
that we may add in the future (like fetch_time, see: https://gist.github.com/s1monw/dba13daf8493bf48431b72365e110717)
it is ok if we start from zero in case a shard follow task moves to another node.
2018-04-05 14:52:20 +02:00