38895 Commits

Author SHA1 Message Date
Tal Levy
73ee5a4aa0 introduce refactored ShrinkAction
this resulted in a few new steps being introduced

- AliasStep
- EnoughShardsWaitStep
- ShrinkStep
- ShrunkenIndexCheckStep
2018-04-11 18:10:13 -07:00
Nhat Nguyen
067fbb8ecd Backport periodic flush count to v6.3.0
Relates #29360
2018-04-11 17:14:28 -04:00
Lee Hinman
9eaec0c808 Adjust to decoupling TimeValue from Writeable (elastic/x-pack-elasticsearch#4338)
This is the x-pack side of https://github.com/elastic/elasticsearch/pull/29454

Original commit: elastic/x-pack-elasticsearch@fab4d511ca
2018-04-11 14:58:19 -06:00
Lee Hinman
263349f628
Decouple TimeValue from Elasticsearch server classes (#29454)
* Decouple TimeValue from Elasticsearch server classes

This commit decouples the `TimeValue` class from the other server classes. This
is in preperation to move `TimeValue` into the `elasticsearch-core` jar,
allowing us to use it from projects that cannot depend on the elasticsearch-core
library.

Relates to #28504
2018-04-11 14:58:15 -06:00
Nik Everett
0a21533097 Docs: Fix callouts after _parent removed
Fix the callouts in the docs for the high level rest client after
_parent support was removed.
2018-04-11 12:56:13 -04:00
Jim Ferenczi
fb81e2cacf Fix template _msearch with extra tokens
This change removes the check for extra tokens when parsing a source generated by a templated
_msearch request. This was added unintentionally in #29428 but the intent of this modification was to validate
simple _search request only.
2018-04-11 18:04:10 +02:00
Nhat Nguyen
0ae627fc79
ElasticsearchMergePolicy extend from MergePolicyWrapper (#29476)
The skeleton of ElasticsearchMergePolicy is quite similar to
MergePolicyWrapper. This commit therefore makes ElasticsearchMergePolicy
inherited from MergePolicyWrapper instead of MergePolicy.
2018-04-11 11:32:19 -04:00
Nhat Nguyen
4e6a8900a3
Add periodic flush count to flush stats (#29360)
Currently, a flush stats contains only the total flush which is the sum
of manual flush (via API) and periodic flush (async triggered when the
uncommitted translog size is exceeded the flush threshold). Sometimes,
it's useful to know these two numbers independently. This commit tracks
and returns a periodic flush count in a flush stats.
2018-04-11 11:15:33 -04:00
Adrien Grand
6a6c0ea5e6
Add an include_type_name option. (#29453)
This adds an `include_type_name` option to the `indices.create`,
`indices.get_mapping` and `indices.put_mapping` APIs, which defaults to `true`.
When set to `false`, then mappings will be returned directly in the body of
the `indices.get_mapping` API, without keying them by the type name, the
`indices.create` will expect mappings directly under the `mappings` key, and
the `indices.put_mapping` will use `_doc` as a type name and fail if a `type`
is provided explicitly.

Relates #15613
2018-04-11 15:54:16 +02:00
Simon Willnauer
45e7e24736
Restrict Document list access in ParseContext (#29463)
Today we expose a mutable list of documents in ParseContext via
ParseContext#docs(). This, on the one hand places knowledge how
to access nested documnts in multiple places and on the other
allows for potential illegal access to nested only docs after
the docs are reversed. This change restricts the access and
streamlines nested / non-root doc access.
2018-04-11 15:09:44 +02:00
Colin Goodheart-Smithe
20485cf7fb Adds more missing equality tests
Specifically for the initialPolicyContextStep and the PhaseAfterStep
2018-04-11 13:24:31 +01:00
Adrien Grand
3a147b442a Fix docs build. 2018-04-11 13:48:53 +02:00
Colin Goodheart-Smithe
482de191f2 Tidies up some of the unit tests
Adds some missing tests including checking the hashcode and equals methods of `DeleteStep`, `StepKey`, and `TerminalPolicyStep` as well as adding a test for `DeleteAction.toSteps()`
2018-04-11 12:38:55 +01:00
Jim Ferenczi
1b6d5e531b
Fail _search request with trailing tokens (#29428)
This change validates that the `_search` request does not have trailing
tokens after the main object and fails the request with a parsing exception otherwise.

Closes #28995
2018-04-11 13:10:22 +02:00
Colin Goodheart-Smithe
66b4e8c4aa Removes unused imports 2018-04-11 10:47:07 +01:00
Adrien Grand
4918924fae
Remove legacy mapping code. (#29224)
Some features have been deprecated since `6.0` like the `_parent` field or the
ability to have multiple types per index. This allows to remove quite some
code, which in-turn will hopefully make it easier to proceed with the removal
of types.
2018-04-11 09:41:37 +02:00
Adrien Grand
6949c888bf Remove useless skip.
Relates #29353
2018-04-11 09:40:28 +02:00
Andrew Odendaal
d15cad4afb Grammar matters.. (#29462)
Update `all indices on this node will marked read-only` to `all indices on this node will be marked read-only`
2018-04-11 09:30:33 +02:00
Adrien Grand
bd834b7357 Remove legacy mapping code. (elastic/x-pack-elasticsearch#4258)
This is a sibling of elastic/elasticsearch#29224.

Original commit: elastic/x-pack-elasticsearch@1c16d86f78
2018-04-11 08:43:52 +02:00
Jason Tedor
663a52ad55
Add useful message when no input from terminal (#29369)
Today when a user runs a CLI tool with standard input closed and no tty
attached, the result from reading is null and this usually leads to a
null pointer exception when we try to parse this input. This arises for
example when the user runs the plugin installer through a Docker
container without leaving standard input open and attaching a tty
(docker exec <container ID> bin/elasticsearch-plugin install). When we
try to read that the user accepts the plugin requiring additional
security permissions we will get back null. This commit addresses this
for all cases by throwing an illegal state exception. The solution for
the user is leave standard input open and attach a tty (or, for some
tools, use batch mode).
2018-04-10 21:50:39 -04:00
Tal Levy
ef34f982f1 refactor a collection of step-related things + shrink_step
- added ShrinkStep/Tests
- AsyncActionStep now passes in IndexMetaData instead of Index
- Delete usage of ClusterStateActionStep
- with ClusterStateActionStep gone, InitializePolicyContextStep
  is the only other ClusterState-nonWait step
- Migrate setting-updates to UpdateSettingsStep
2018-04-10 17:13:10 -07:00
Zachary Tong
c341b41c54 [TEST] Temporarily silence MovAvgIT tests due to change in double comparisons
#29409 removed the nearlyEquals() double comparison snippet, which
makes these tests very flaky because they can generate very large or
very small doubles which don't work well with absolute error comparison.

We need to either refactor these tests to guarantee they stay in a small
range (which could be difficult due to holt/holt-winters) or re-implement
the more robust double comparison.

Tracking issue: #29456
2018-04-10 20:45:33 +00:00
Zachary Tong
20dbd75623 [Rollup] Rename job config size to page_size (elastic/x-pack-elasticsearch#4309)
Renaming should hopefully make it more clear that this is the size
of pages to process during rolling up, nothing to do with the size
of the various groups, metrics, etc.

Original commit: elastic/x-pack-elasticsearch@8a0a44f04b
2018-04-10 13:34:40 -07:00
Zachary Tong
cf965dca27 [Docs] Mark Rollup as Experimental instead of Beta
Original commit: elastic/x-pack-elasticsearch@bce120daae
2018-04-10 15:51:09 +00:00
Colin Goodheart-Smithe
8d91f197d4 Changes update replica and alllocation to async steps
Also renames EnoughShardsWaitStep to ReplicasAllocatedStep, removes it from the allocate action and adds a check that th number of replicas in the cluster state is correct to it.
2018-04-10 16:20:29 +01:00
Jason Tedor
bca192a327
Simplify TranslogWriter#closeWithTragicEvent (#29412)
This commit simplifies the exception handling in
TranslogWriter#closeWithTragicEvent. When invoking this method, the
inner close method could throw an exception which we always catch and
suppress into the exception that led us to tragically close. This commit
moves that repeated logic into closeWithTragicException and now callers
simply need to catch, invoke closeWithTragicException, and rethrow.
2018-04-10 10:15:54 -04:00
Lee Hinman
0f40199d10
Remove custom PeriodType formatting from TimeValue (#29433)
In order to decouple TimeValue from Joda, this removes the unused `format`
methods.

Relates to #28504
2018-04-10 08:02:56 -06:00
Lee Hinman
9e3b03531c Remove custom PeriodType for watcher PeriodThrottler (elastic/x-pack-elasticsearch#4327)
This constructor was actually never used, other than in tests, and even then,
there is no need for a custom period type as the human-readable toString value
will suffice.

Original commit: elastic/x-pack-elasticsearch@fc666a04b9
2018-04-10 08:01:32 -06:00
tomcallahan
2574064e66
Enable rest tests via IDEs (#29439)
Currently rest-based tests do not work from the IDE, as the security
manager is configured to permit certain network operations when
using the snapshot jars compiled by gradle.  We have an existing
workaround that explicitly associates a codebase with the path
from which the classes are loaded (in this case, the IDE build
directory).  This PR adds the rest client to this workaround list.
2018-04-10 09:08:58 -04:00
Dimitris Athanasiou
4c7bd71bdf [ML] Improve comment on why task is marked completed on kill process
Original commit: elastic/x-pack-elasticsearch@5c16cf8e26
2018-04-10 13:36:13 +01:00
Adrien Grand
aeac682869
Make purely negative queries return scores of 0. (#26015)
It would make them consistent with queries that are only made of filters.

Closes #23449
2018-04-10 14:31:06 +02:00
Adrien Grand
a091d950a7
Deprecate slicing on _uid. (#29353)
Deprecate slicing on `_uid`.

`_id` should be used instead on 6.x.
2018-04-10 14:28:30 +02:00
David Kyle
7e4e1dabcf [ML] Add categorical exclude condition (elastic/x-pack-elasticsearch#4326)
Original commit: elastic/x-pack-elasticsearch@6c80988e08
2018-04-10 13:19:00 +01:00
Vladimir Dolzhenko
03d1a7e132
Version conflict exception message enhancement (#29432)
Report doc is not found rather on PUT ?version=X rather current version [-1] is different than the one provided

Closes #21278
2018-04-10 13:42:59 +02:00
Martijn van Groningen
182cf11f37
Fixed bug when non percolator docs end up in the search hits.
In the case that a document with a percolator field is matched when using the `percolate` query then
the fetch phase can fail due to the fact that the percolator can't resolve any query from that document.

Closes #29429
2018-04-10 13:33:31 +02:00
Christoph Büscher
9f0c5ccf34 [Docs] Correct typos in rank-eval and multi-search 2018-04-10 12:48:16 +02:00
Christoph Büscher
13da9dd7c0
Remove 5x bwc in LocaleUtils#parse (#29417)
Remove the special treatment of parsing the locale property for old 5.x
indices since in 7.0 we only need to support reading from 6.x indices.
2018-04-10 12:40:36 +02:00
Tim Vernum
411f683521 Document ssl.verification_mode (elastic/x-pack-elasticsearch#4253)
It is common for users to wish to adjust the verification_mode in SSL
settings, usually with the intention of skipping hostname
verification. This has been supported for a long time, but the
relevant configuration setting was not clearly documented, which would
sometimes lead users to set `verification_mode` to `none`, and disable
more checks than they intended.

This commit adds clearer documentation regarding the options available
for `verification_mode` and actively discourages the use of `none`.

Original commit: elastic/x-pack-elasticsearch@2fdf53b42f
2018-04-10 20:27:23 +10:00
Colin Goodheart-Smithe
aa198b637e Cleans up warnings 2018-04-10 09:53:57 +01:00
D Pinto
8d6a368402 [Docs] Correct typo in pipeline.asciidoc (#29431) 2018-04-10 10:42:07 +02:00
Colin Goodheart-Smithe
c24dc07346 Removes unused and unneeded code
Various classes had some code that was not used and is not going to be needed so this change cleans up those classes so we don’t have dead code hanging around
2018-04-10 09:28:03 +01:00
Colin Goodheart-Smithe
b8d573772f Fixes compilation issue in test 2018-04-10 09:10:52 +01:00
Martijn van Groningen
2346f7fa89
removed unused import 2018-04-10 07:44:51 +02:00
Martijn van Groningen
f4395c0c94
Fixed a msm accounting error that can occur during analyzing a percolator query.
In case of a disjunction query with both range and term based clauses and
msm specified, the query analyzer needs to also reduce the msn if a range
based clause for the same field is encountered. This did not happen.

Instead of fixing this bug the logic has been simplified to just set a
percolator query's msm to 1 if a disjunction contains range clauses and
msm on disjunction has been specified. The logic would otherwise just get
to complex and the performance gain isn't that much for this kind of
percolator queries.

In case a percolator query has clauses that have duplicate terms or ranges then
for disjunction clauses with a minimum should match the query extraction of the
clause with the lowest msm should be used and for conjunction queries query
extractions wiht duplicate terms/ranges the msn should be ignored. If this
is not done then percolator queries that should match never match.

Example percolator query: value1 OR value2 OR value2 OR value3 OR value3 OR value3 OR value4 OR value5 (msm set to 3)
In the above example query the extracted msm would be 3
Example document1: value1 value2 value3
With the msm and extracted terms this would match and is expected behaviour
Example document2: value3
This document should match too (value3 appears in 3 clauses), but with msm set to 3 and the fact
that fact that only distinct values are indexed in extracted terms field this document would

Also added another random duel test.

Closes #29393
2018-04-10 07:25:12 +02:00
Tal Levy
f5b23df349 add more force-merge related steps and tests
The force-merge is an a TODO state due to the
unresolved issue around best_compression.

- updated ReadOnlyStep with tests
- implemented an update to the ForceMergeAction
- added UpdateBestCompressionSettingsStep
- added tests for SegmentCountStep
2018-04-09 16:28:25 -07:00
tomcallahan
ec65710926
Remove copy-pasted code (#29409)
* Remove copy-pasted code

We had two instances of copy-pasted code with a bad license from
another website.  The code was doing something rather simple, and
that functionality already exists within junit.  This PR simply leverages
the junit functionality.
2018-04-09 18:32:32 -04:00
Igor Motov
2ba7258603 SQL: Extract CSV spec parser into Utils class (elastic/x-pack-elasticsearch#4317)
For Geo SQL we need to have a separate set of CSV tests. This commit
extracts CSV spec parsing logic into a separate file for a more
straight forward reuse.

Relates elastic/x-pack-elasticsearch#4080

Original commit: elastic/x-pack-elasticsearch@29034ef051
2018-04-09 17:01:24 -04:00
Lisa Cawley
17543dbc00 [DOCS] Augmented token API description (elastic/x-pack-elasticsearch#4321)
Original commit: elastic/x-pack-elasticsearch@c24713ee43
2018-04-09 11:39:21 -07:00
Lee Hinman
218e9a57bb Correct documentation for number of salt generation rounds (elastic/x-pack-elasticsearch#4322)
For the user cache, the crypt option rounds are actually the log2 of the number
of rounds. This commits updates the documentation to reflect this.

Original commit: elastic/x-pack-elasticsearch@d3cc2b7f29
2018-04-09 10:59:06 -06:00
Lisa Cawley
28e9ef3c83
[DOCS] Updated installation pages with X-Pack indices (#29077) 2018-04-09 08:19:38 -07:00