Commit Graph

40750 Commits

Author SHA1 Message Date
Dimitris Athanasiou 46c35db1df [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature
As the multibucket feature was merged in, this test hit a side effect
which means buckets trailing an anomaly could become anomalous.
This commit fixes the problem by filtering low score records when
we request them.
2018-08-17 17:52:29 +01:00
Lisa Cawley c5de9ec79d
[DOCS] Splits the roles API documentation into multiple pages (#32794) 2018-08-17 09:18:08 -07:00
Ioannis Kakavas e3aa68b0a9
[TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901)
Elasticsearch versions earlier than 6.4.0 cannot properly run in a
FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for
nodes that we spin up in BWC tests even when we're testing FIPS.
2018-08-17 18:23:13 +03:00
Igor Motov da6b61e8ef
Make Geo Context Mapping Parsing More Strict (#32821)
Currently, if geo context is represented by something other than
geo_point or an object with lat and lon fields, the parsing of it
as a geo context can result in ignoring the context altogether,
returning confusing errors such as number_format_exception or trying
to parse the number specifying as long-encoded hash code. It would also
fail if the geo_point was stored.

This commit makes the mapping parsing more strict and will fail during
mapping update or index creation if the geo context doesn't point to
a geo_point field.

Supersedes #32412

Closes #32202
2018-08-17 08:13:16 -07:00
Benjamin Trent 9cec4aa14b
[ML] fix updating opened jobs scheduled events (#31651) (#32881)
* ML: fix updating opened jobs scheduled events (#31651)

* Adding UpdateParamsTests license header

* Adding integration test and addressing PR comments

* addressing test and job names
2018-08-17 07:21:17 -05:00
Jonathan Little a08127c072 Scripted metric aggregations: add deprecation warning and system property to control legacy params (#31597)
* Scripted metric aggregations: add deprecation warning and system property to control legacy params

Scripted metric aggregation params._agg/_aggs are replaced by state/states context variables. By default the old params are still present, and a deprecation warning is emitted when Scripted Metric Aggregations are used. A new system property can be used to disable the legacy params. This functionality will be removed in a future revision.

* Fix minor style issue and docs test failure

* Disable deprecated params._agg/_aggs in tests and revise tests to use state/states instead

* Add integration test covering deprecated scripted metrics aggs params._agg/_aggs access

* Disable deprecated params._agg/_aggs in docs integration tests and revise stored scripts to use state/states instead

* Revert unnecessary migrations doc change

A relevant note should be added in the changes destined for 7.0; this PR is going to be backported to 6.x.

* Replace deprecated _agg param bwc integration test with a couple of unit tests

* Fix compatibility test after merge

* Rename backwards compatibility system property per code review feedback

* Tweak deprecation warning text per review feedback
2018-08-17 13:11:18 +01:00
Alexander Reelsen 0d92f377fd Tests: Fix timezone conversion in DateTimeUnitTests
This fix prevernts trying to parse unknown timezone ids by converting
the joda time zone via java.util.TimeZone to a java time based ZoneId.

Closes #32927
2018-08-17 14:09:01 +02:00
Ioannis Kakavas 75014a22d7
Enable FIPS140LicenseBootstrapCheck (#32903)
This commit ensures that xpack.security.fips_mode.enabled: true 
cannot be set in a node that doesn't have the appropriate license.
2018-08-17 14:06:24 +03:00
Paul Sanwald ca54aacbb5
Fix InternalAutoDateHistogram reproducible failure (#32723)
Update test logic to correctly bucket intervals.
2018-08-17 07:03:25 -04:00
Andrey Ershov 2fa028cfa1
Remove assertion in testDocStats on deletedDocs counter (#32914)
testDocStats test is flaky and sometimes it's failing on jenkins and
failure is not reproducible locally. The reason for this failure is in
timing. If the number of deleted documents is greater than 33% of inserted
documents, Lucene will schedule segments to merge if TieredMergePolicy is
used (it's not the case for LogMergePolicy, but ES is only using
TieredMergePolicy). If this merge is performed before stats are
retrieved - we will get 0 for "deleted" counter.
So basically this counter could be either 0 or numOfDeletedDocs at this point,
but this is the too loose assertion and we decided to remove it at all.
Closes #32766
2018-08-17 12:36:45 +02:00
Dimitris Athanasiou 76aba8ad7b
HLRC: Move ML request converters into their own class (#32906) 2018-08-17 10:57:00 +01:00
markwalkom ae38cfbaec [DOCS] Update getting-started.asciidoc (#29518)
Highlighted that you can change shard counts using `_shrink` and `_split`.
2018-08-17 11:09:22 +02:00
JB Nizet dd5a5aab88 Fix allowed value for HighlighterBuilder encoder in javadocs (#32780)
Relates to #32745
2018-08-17 10:59:26 +02:00
JeffSaxeVA efdad7d5fc [DOCS] Add "remove a tag" script logic as an example (#32556)
It took me quite a while of online searching and experimenting to realize the function-call asymmetry in the Add versus Remove from a list, like the "tags" list! I realize we cannot give examples for every single thing the user wants to do in Painless, but this is such a common use case (removing a tag from a single doc, or from a set of docs with Update-By-Query) that I believe it ought to be demonstrated immediately after the "add a tag" example. We have an example of removing an entire document field, but not removing one element of a list (a multi-valued field).

Also, a minor grammar fix: I have added an apostrophe to the word "its" in the accompanying text of the example just above.
2018-08-17 10:56:06 +02:00
Alpar Torok 148a76f0c7 Fix failing BuildExamplePluginsIT test 2018-08-17 11:14:58 +03:00
Alpar Torok d16562eab5
RFC: Test that example plugins build stand-alone (#32235)
Add tests for build-tools to make sure example plugins build stand-alone using it.

This will catch issues such as referencing files from the buildSrc directly, breaking external uses of build-tools.
2018-08-17 09:41:39 +03:00
Jay Modi 1136a95837
Security: remove put privilege API (#32879)
This commit removes the put privilege API in favor of having a single API to
create and update privileges. If we see the need to have an API like this in
the future we can always add it back.
2018-08-16 21:16:06 -06:00
Julie Tibshirani cbf160a4e6
For filters aggs, make sure that rewrites preserve other_bucket. (#32921) 2018-08-16 17:36:58 -07:00
Tal Levy 711ac402b6 Merge branch 'master' into index-lifecycle 2018-08-16 15:08:22 -07:00
Jason Tedor 0876630b30
Guard against null in email admin watches (#32923)
The Kibana settings docs that these watches rely on can sometimes
contain no xpack settings. When this is the case, we will end up with a
null pointer exception in the script. We need to guard against in these
scripts so this commit does that.
2018-08-16 16:25:34 -04:00
Ed Savage 62559d2b3c
Re enable ml bwc tests (#32916)
[ML] Re-enabling BWC tests

Re-enable BWC tests for ML now that #32816 has been backported to 6.x
2018-08-16 18:23:26 +01:00
Ed Savage d604b3e3a1 Temporarily disabled ML BWC tests for backporting
https://github.com/elastic/elasticsearch/pull/32816
2018-08-16 17:18:51 +01:00
Tal Levy 62819826c7
copy more actions to protocol.xpack (#32892)
this PR creates client-side instances of the following actions:

- DeleteAction
- ForceMergeAction
- ReadOnlyAction
- RolloverAction
- ForceMergeAction

AllocateAction was done separately
2018-08-16 08:48:26 -07:00
Tal Levy c9de707f58 Merge branch 'master' into index-lifecycle 2018-08-16 08:41:57 -07:00
Tal Levy 238eb93893
migrate allocate action pojo/xcontent to xpack.protocol (#32853) 2018-08-16 08:41:07 -07:00
Jason Tedor b5a85362e3
Fix docs for fixed filename for heap dump path (#32882)
The docs here incorrectly state that it is okay for a heap dump file to
exist when heap dump path is configured to a fixed filename. This is
incorrect, the JVM will fail to write the heap dump if a heap dump file
already exists at the specified location (see the DumpWriter constructor
DumpWriter::DumpWriter(const char* path) in the JVM source).
2018-08-16 11:32:35 -04:00
Jack Conradson d9fd74bcdc
Painless: Special Case def (#32871)
This removes def from the classes map in PainlessLookup and instead always special 
cases it. This prevents potential calls against the def type that shouldn't be made and 
forces all cases of def throughout Painless code to be special cased.
2018-08-16 08:03:21 -07:00
Adrien Grand eaaf37a1f9 AwaitFix FullClusterRestartIT#testRollupIDSchemeAfterRestart. 2018-08-16 16:35:10 +02:00
Jim Ferenczi 3dd1677cdc [Test] Fix DuelScrollIT#testDuelIndexOrderQueryThenFetch
This commit disables the automatic `refresh_interval` in order to ensure
that index readers cannot differ between the normal and scroll search.
This issue is related to the 7.5 Lucene upgrade which contains a change that
makes single segment merge more likely to occur (max deletes percentage).

Closes #32682
2018-08-16 15:33:17 +02:00
Benjamin Trent aedc2c1c49
HLRC: adding machine learning delete job (#32820)
* HLRC: adding machine learning delete job

* Fixing whitespace

* Moving docs and tests around

* Unifying ml asciidoc file naming convention
2018-08-16 07:18:43 -05:00
Hazem Khaled b87f3062b7 [DOCS] Update WordPress plugins links (#32194) 2018-08-16 13:55:40 +02:00
Jason Tedor f8c7414ee8
Remove passphrase support from reload settings API (#32889)
We do not support passphrases on the secure settings storage (the
keystore). Yet, we added support for this in the API layer. This commit
removes this support so that we are not limited in our future options,
or have to make a breaking change.
2018-08-16 07:24:05 -04:00
Adrien Grand e35be01901 AwaitFix AckIT.
Relates #32767
2018-08-16 12:31:58 +02:00
Colin Goodheart-Smithe d80457ee2a
Mutes test in DuelScrollIT
Due to https://github.com/elastic/elasticsearch/issues/32682
2018-08-16 11:08:00 +01:00
Adrien Grand 039babddf5 CharArraysTests: Fix test bug. 2018-08-16 11:54:39 +02:00
David Kyle 7f6802cb51 [ML] Choose seconds to fix intermittent DatafeeedConfigTest failure 2018-08-16 10:48:56 +01:00
Jim Ferenczi 996ed73d73 Test: Fix unpredictive merges in DocumentSubsetReaderTests
The merge policy that was used could lead to unpredictive merges due to the
randomization of `setDeletesPctAllowed`.

Closes #32457
2018-08-16 11:44:57 +02:00
datosh e6bfba1d79 [DOCS] Clarify sentence in network-host.asciidoc (#32429) 2018-08-16 11:38:19 +02:00
markharwood 70d80a3d09
Docs enhancement: added reference to cluster-level setting `search.default_allow_partial_results` (#32810)
Closes #32809
2018-08-16 10:21:37 +01:00
Tal Levy 9406aa5af9
remove extraneous CreateIndexRequest context in ShrinkAction (#32893) 2018-08-15 21:55:18 -07:00
Tal Levy 33522d4fb4
update cluster-state task execution to halt on new phase (#32886)
As we migrate to a per-phase execution model, we need to prepare our cluster-state-step execution model to be aligned. It is the case that the final iteration into the next "currentStep" from the next phase would not be available in the registry yet. This change exits the execution loop early as to not jump into executing the next phase's steps before the registry is properly updated
2018-08-15 19:30:21 -07:00
debadair 93ada98eb5
[DOCS] Fixing cross doc link to Stack Overview security topic. 2018-08-15 15:09:41 -07:00
Tal Levy 4baa721459
remove `type` config from LifecyclePolicy JSON (#32660)
Since there is only one production policy, Timeseries, there
is no reason to expose the `type` argument to the user.
2018-08-15 14:47:22 -07:00
Tal Levy d1ab5dd650 fix merge with master 2018-08-15 14:46:31 -07:00
Jay Modi 1a45b27d8b
Move CharArrays to core lib (#32851)
This change cleans up some methods in the CharArrays class from x-pack, which
includes the unification of char[] to utf8 and utf8 to char[] conversions that
intentionally do not use strings. There was previously an implementation in
x-pack and in the reloading of secure settings. The method from the reloading
of secure settings was adopted as it handled more scenarios related to the
backing byte and char buffers that were used to perform the conversions. The
cleaned up class is moved into libs/core to allow it to be used by requests
that will be migrated to the high level rest client.

Relates #32332
2018-08-15 15:26:00 -06:00
Tal Levy ec93756600 Merge branch 'master' into index-lifecycle 2018-08-15 12:56:01 -07:00
Jason Tedor 364ccc36d6
Fix global checkpoint listeners test
This commit fixes a global checkpoint listeners test wherein we were
expecting an executor to have been used even if there were no
listeners. This is silliness, so this commit adjusts the assertion to
verify that the executor never fires if there are no listeners, and
fires exactly once if there is one or more listeners.
2018-08-15 15:53:15 -04:00
Benjamin Trent f90f06ede1
HLRC: adding machine learning open job (#32860)
* HLRC: adding machine learning open job

HLRC: adding ML open job API call

* Addressing @droberts195 comments and fixing minor style issue
2018-08-15 13:19:41 -05:00
David Roberts 5ba04e23fc
[ML] Add log structure finder functionality (#32788)
This change adds a library to ML that can be used to deduce a log
file's structure given only a sample of the log file.

Eventually this will be used to add an endpoint to ML to make the
functionality available to end users, but this will follow in a
separate change.

The functionality is split into a library so that it can also be
used by a command line tool without requiring the command line
tool to include all server code.
2018-08-15 18:04:21 +01:00
Armin Braun 986c55b830
INGEST: Add Configuration Except. Data to Metdata (#32322)
* closes #27728
2018-08-15 19:02:19 +02:00