Commit Graph

40669 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe 3736097e19
Merge branch 'master' into index-lifecycle 2018-08-20 10:06:02 +01:00
Colin Goodheart-Smithe b6e95cde3a
Fixes libs:dissect when in eclipse 2018-08-20 10:05:25 +01:00
Jonathan Little 676091aafb Protect ScriptedMetricIT test cases against failures on 0-doc shards (#32959) (#32968)
Randomized test conditions that cause some shards to have no docs on them
failed due to test asserts that relied on a lazy initialization side effect
from the map script. After this fix:

- Test cases with the relevant init script are protected
- Test cases with the relevant combine or reduce scripts were already
  protected, because the combine and reduce scripts safely handle this case.
2018-08-20 08:55:43 +01:00
Yogesh Gaikwad e143cce865
[Kerberos] Add documentation for Kerberos realm (#32662)
This commit adds documentation for configuring Kerberos realm.
Configuring Kerberos realm documentation highlights important
terminology and requirements before creating Kerberos realm.
Most of the documentation is centered around configuration from
Elasticsearch rather than go deep into Kerberos implementation.
Kerberos realm settings are mentioned in the security settings
for Kerberos realm.
2018-08-20 17:23:14 +10:00
Alexander Reelsen 3fa36807f8
Watcher: Properly find next valid date in cron expressions (#32734)
When a list/an array of cron expressions is provided, and one of those addresses
is already expired, the expired one will be considered as an option
instead of the valid next one.

This commit also reduces the visibility of the CronnableSchedule and
refactors a comparator to look like java 8.
2018-08-20 09:18:51 +02:00
Tim Ryan dce72c7985 Fix some small issues in the getting started docs (#30346)
* Modified a reference to real time to match the previous line reference of
  realtime.
* Modified eg to e.g. as it's an abbreviation for the latin exempli gratia
* Added missing pronoun to `_executing_filters` section.
2018-08-20 07:54:03 +01:00
Alpar Torok 4b34b3f4aa
Set forbidden APIs target compatibility to compiler java version (#32935)
Set forbidden apis target compatibility to compiler version

Fix outstanding deprecation
2018-08-20 09:27:02 +03:00
Tim Brooks de92d2ef1f
Move connection listener to ConnectionManager (#32956)
This is a followup to #31886. After that commit the
TransportConnectionListener had to be propogated to both the
Transport and the ConnectionManager. This commit moves that listener
to completely live in the ConnectionManager. The request and response
related methods are moved to a TransportMessageListener. That listener
continues to live in the Transport class.
2018-08-18 10:09:24 -06:00
Armin Braun f82bb64feb
NETWORKING: Make RemoteClusterConn. Lazy Resolve DNS (#32764)
* Lazy resolve DNS (i.e. `String` to `DiscoveryNode`) to not run into indefinitely caching lookup issues (provided the JVM dns cache is configured correctly as explained in https://www.elastic.co/guide/en/elasticsearch/reference/6.3/networkaddress-cache-ttl.html)
   * Changed `InetAddress` type to `String` for that higher up the stack
   * Passed down `Supplier<DiscoveryNode>` instead of outright `DiscoveryNode` from `RemoteClusterAware#buildRemoteClustersSeeds` on to lazy resolve DNS when the `DiscoveryNode` is actually used (could've also passed down the value of `clusterName = REMOTE_CLUSTERS_SEEDS.getNamespace(concreteSetting)` together with the `List<String>` of hosts, but this route seemed to introduce less duplication and resulted in a significantly smaller changeset).
* Closes #28858
2018-08-18 08:46:44 +02:00
Lisa Cawley 532d552ffd
[DOCS] Splits the users API documentation into multiple pages (#32825) 2018-08-17 23:17:33 -07:00
Lisa Cawley fb1c3990d7
[DOCS] Splits the token APIs into separate pages (#32865) 2018-08-17 22:22:09 -07:00
lcawl 1efee66d16 [DOCS] Creates redirects for role management APIs page 2018-08-17 21:55:18 -07:00
Lee Hinman 77016add19
Store phase steps for index in PolicyStepsRegistry (#32926)
* Store phase steps for index in PolicyStepsRegistry

This changes the way that steps are retrieved from `PolicyStepsRegistry` to
store the steps on a per-index basis (in memory for now, though that will change
in subsequent PRs). These steps are rebuilt as the index changes phases.

This also fixes a bug where an action with the same phase and name was not being
considered changed (and thus updated) in the compiled steps list. These are now
correctly considered as "upsert" diffs.

Relates to #29823
2018-08-17 22:45:15 -06:00
Tal Levy a26e108590 Merge branch 'master' into index-lifecycle 2018-08-17 13:57:28 -07:00
Benjamin Trent 647705e00a Bypassing failing test PainlessDomainSplitIT#testHRDSplit (#32966) 2018-08-17 15:30:31 -05:00
Nhat Nguyen 86ffce4bbc TEST: Mute testRetentionPolicyChangeDuringRecovery
Tracked at #32089
2018-08-17 14:12:45 -04:00
lcawl 967b1785fa [DOCS] Fixes more broken links to role management APIs 2018-08-17 10:41:06 -07:00
Zachary Tong 899e94a29b [Docs] Tweaks and fixes to rollup docs
- Missing links to new IndexCaps API
- Incorrect security permissions on IndexCaps API
- GetJobs API must supply a job (or `_all`), omitting throws error
- Link to search/agg limitations from RollupSearch API
- Tweak URLs in quick reference
- Formatting of overview page
2018-08-17 13:37:11 -04:00
lcawl a608205510 [DOCS] Fixes links to role management APIs 2018-08-17 10:22:06 -07:00
Dimitris Athanasiou 46c35db1df [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature
As the multibucket feature was merged in, this test hit a side effect
which means buckets trailing an anomaly could become anomalous.
This commit fixes the problem by filtering low score records when
we request them.
2018-08-17 17:52:29 +01:00
Lisa Cawley c5de9ec79d
[DOCS] Splits the roles API documentation into multiple pages (#32794) 2018-08-17 09:18:08 -07:00
Ioannis Kakavas e3aa68b0a9
[TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901)
Elasticsearch versions earlier than 6.4.0 cannot properly run in a
FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for
nodes that we spin up in BWC tests even when we're testing FIPS.
2018-08-17 18:23:13 +03:00
Igor Motov da6b61e8ef
Make Geo Context Mapping Parsing More Strict (#32821)
Currently, if geo context is represented by something other than
geo_point or an object with lat and lon fields, the parsing of it
as a geo context can result in ignoring the context altogether,
returning confusing errors such as number_format_exception or trying
to parse the number specifying as long-encoded hash code. It would also
fail if the geo_point was stored.

This commit makes the mapping parsing more strict and will fail during
mapping update or index creation if the geo context doesn't point to
a geo_point field.

Supersedes #32412

Closes #32202
2018-08-17 08:13:16 -07:00
Benjamin Trent 9cec4aa14b
[ML] fix updating opened jobs scheduled events (#31651) (#32881)
* ML: fix updating opened jobs scheduled events (#31651)

* Adding UpdateParamsTests license header

* Adding integration test and addressing PR comments

* addressing test and job names
2018-08-17 07:21:17 -05:00
Jonathan Little a08127c072 Scripted metric aggregations: add deprecation warning and system property to control legacy params (#31597)
* Scripted metric aggregations: add deprecation warning and system property to control legacy params

Scripted metric aggregation params._agg/_aggs are replaced by state/states context variables. By default the old params are still present, and a deprecation warning is emitted when Scripted Metric Aggregations are used. A new system property can be used to disable the legacy params. This functionality will be removed in a future revision.

* Fix minor style issue and docs test failure

* Disable deprecated params._agg/_aggs in tests and revise tests to use state/states instead

* Add integration test covering deprecated scripted metrics aggs params._agg/_aggs access

* Disable deprecated params._agg/_aggs in docs integration tests and revise stored scripts to use state/states instead

* Revert unnecessary migrations doc change

A relevant note should be added in the changes destined for 7.0; this PR is going to be backported to 6.x.

* Replace deprecated _agg param bwc integration test with a couple of unit tests

* Fix compatibility test after merge

* Rename backwards compatibility system property per code review feedback

* Tweak deprecation warning text per review feedback
2018-08-17 13:11:18 +01:00
Alexander Reelsen 0d92f377fd Tests: Fix timezone conversion in DateTimeUnitTests
This fix prevernts trying to parse unknown timezone ids by converting
the joda time zone via java.util.TimeZone to a java time based ZoneId.

Closes #32927
2018-08-17 14:09:01 +02:00
Ioannis Kakavas 75014a22d7
Enable FIPS140LicenseBootstrapCheck (#32903)
This commit ensures that xpack.security.fips_mode.enabled: true 
cannot be set in a node that doesn't have the appropriate license.
2018-08-17 14:06:24 +03:00
Paul Sanwald ca54aacbb5
Fix InternalAutoDateHistogram reproducible failure (#32723)
Update test logic to correctly bucket intervals.
2018-08-17 07:03:25 -04:00
Andrey Ershov 2fa028cfa1
Remove assertion in testDocStats on deletedDocs counter (#32914)
testDocStats test is flaky and sometimes it's failing on jenkins and
failure is not reproducible locally. The reason for this failure is in
timing. If the number of deleted documents is greater than 33% of inserted
documents, Lucene will schedule segments to merge if TieredMergePolicy is
used (it's not the case for LogMergePolicy, but ES is only using
TieredMergePolicy). If this merge is performed before stats are
retrieved - we will get 0 for "deleted" counter.
So basically this counter could be either 0 or numOfDeletedDocs at this point,
but this is the too loose assertion and we decided to remove it at all.
Closes #32766
2018-08-17 12:36:45 +02:00
Dimitris Athanasiou 76aba8ad7b
HLRC: Move ML request converters into their own class (#32906) 2018-08-17 10:57:00 +01:00
markwalkom ae38cfbaec [DOCS] Update getting-started.asciidoc (#29518)
Highlighted that you can change shard counts using `_shrink` and `_split`.
2018-08-17 11:09:22 +02:00
JB Nizet dd5a5aab88 Fix allowed value for HighlighterBuilder encoder in javadocs (#32780)
Relates to #32745
2018-08-17 10:59:26 +02:00
JeffSaxeVA efdad7d5fc [DOCS] Add "remove a tag" script logic as an example (#32556)
It took me quite a while of online searching and experimenting to realize the function-call asymmetry in the Add versus Remove from a list, like the "tags" list! I realize we cannot give examples for every single thing the user wants to do in Painless, but this is such a common use case (removing a tag from a single doc, or from a set of docs with Update-By-Query) that I believe it ought to be demonstrated immediately after the "add a tag" example. We have an example of removing an entire document field, but not removing one element of a list (a multi-valued field).

Also, a minor grammar fix: I have added an apostrophe to the word "its" in the accompanying text of the example just above.
2018-08-17 10:56:06 +02:00
Alpar Torok 148a76f0c7 Fix failing BuildExamplePluginsIT test 2018-08-17 11:14:58 +03:00
Alpar Torok d16562eab5
RFC: Test that example plugins build stand-alone (#32235)
Add tests for build-tools to make sure example plugins build stand-alone using it.

This will catch issues such as referencing files from the buildSrc directly, breaking external uses of build-tools.
2018-08-17 09:41:39 +03:00
Jay Modi 1136a95837
Security: remove put privilege API (#32879)
This commit removes the put privilege API in favor of having a single API to
create and update privileges. If we see the need to have an API like this in
the future we can always add it back.
2018-08-16 21:16:06 -06:00
Julie Tibshirani cbf160a4e6
For filters aggs, make sure that rewrites preserve other_bucket. (#32921) 2018-08-16 17:36:58 -07:00
Tal Levy 711ac402b6 Merge branch 'master' into index-lifecycle 2018-08-16 15:08:22 -07:00
Jason Tedor 0876630b30
Guard against null in email admin watches (#32923)
The Kibana settings docs that these watches rely on can sometimes
contain no xpack settings. When this is the case, we will end up with a
null pointer exception in the script. We need to guard against in these
scripts so this commit does that.
2018-08-16 16:25:34 -04:00
Ed Savage 62559d2b3c
Re enable ml bwc tests (#32916)
[ML] Re-enabling BWC tests

Re-enable BWC tests for ML now that #32816 has been backported to 6.x
2018-08-16 18:23:26 +01:00
Ed Savage d604b3e3a1 Temporarily disabled ML BWC tests for backporting
https://github.com/elastic/elasticsearch/pull/32816
2018-08-16 17:18:51 +01:00
Tal Levy 62819826c7
copy more actions to protocol.xpack (#32892)
this PR creates client-side instances of the following actions:

- DeleteAction
- ForceMergeAction
- ReadOnlyAction
- RolloverAction
- ForceMergeAction

AllocateAction was done separately
2018-08-16 08:48:26 -07:00
Tal Levy c9de707f58 Merge branch 'master' into index-lifecycle 2018-08-16 08:41:57 -07:00
Tal Levy 238eb93893
migrate allocate action pojo/xcontent to xpack.protocol (#32853) 2018-08-16 08:41:07 -07:00
Jason Tedor b5a85362e3
Fix docs for fixed filename for heap dump path (#32882)
The docs here incorrectly state that it is okay for a heap dump file to
exist when heap dump path is configured to a fixed filename. This is
incorrect, the JVM will fail to write the heap dump if a heap dump file
already exists at the specified location (see the DumpWriter constructor
DumpWriter::DumpWriter(const char* path) in the JVM source).
2018-08-16 11:32:35 -04:00
Jack Conradson d9fd74bcdc
Painless: Special Case def (#32871)
This removes def from the classes map in PainlessLookup and instead always special 
cases it. This prevents potential calls against the def type that shouldn't be made and 
forces all cases of def throughout Painless code to be special cased.
2018-08-16 08:03:21 -07:00
Adrien Grand eaaf37a1f9 AwaitFix FullClusterRestartIT#testRollupIDSchemeAfterRestart. 2018-08-16 16:35:10 +02:00
Jim Ferenczi 3dd1677cdc [Test] Fix DuelScrollIT#testDuelIndexOrderQueryThenFetch
This commit disables the automatic `refresh_interval` in order to ensure
that index readers cannot differ between the normal and scroll search.
This issue is related to the 7.5 Lucene upgrade which contains a change that
makes single segment merge more likely to occur (max deletes percentage).

Closes #32682
2018-08-16 15:33:17 +02:00
Benjamin Trent aedc2c1c49
HLRC: adding machine learning delete job (#32820)
* HLRC: adding machine learning delete job

* Fixing whitespace

* Moving docs and tests around

* Unifying ml asciidoc file naming convention
2018-08-16 07:18:43 -05:00
Hazem Khaled b87f3062b7 [DOCS] Update WordPress plugins links (#32194) 2018-08-16 13:55:40 +02:00