Commit Graph

41065 Commits

Author SHA1 Message Date
Armin Braun 917e5a8c94
TESTS: Fix Random Fail in MockTcpTransportTests (#33061)
* `foobar.txGet()` appears to return before `serviceB.stop()` returns, causing `ServiceB.close()` to run concurrently with the `stop` call and running into a race codition
* Closes #32863
2018-08-23 13:19:21 +02:00
Armin Braun 61f5c188e0 HLRC: Fix Compile Error From Missing Throws (#33083)
* 50441f97ae745814db96c262e99d0f465aca5b2c#diff-53a95fe7ded21313483f1b2f15977395L72 removed the throws breaking compilation here
2018-08-23 11:24:32 +01:00
Albert Zaharovits 07cce95305 [DOCS] Remove reload password from docs cf. #32889
Reload call `_nodes/reload_secure_settings` is not requiring
an empty password anymore (#32889). Reflect this in docs.
2018-08-23 12:11:39 +03:00
Dimitris Athanasiou 50441f97ae
HLRC: Add ML Get Buckets API (#33056)
Relates #29827
2018-08-23 09:35:06 +01:00
lipsill f84ed14294 Watcher: Improve error messages for CronEvalTool (#32800)
CronEvalTool prints an error only for cron expressions that result in
no upcoming time events.

If a cron expression results in less than the specified count
(default 10) time events, now all the coming times are printed
without displaying error message.

Closes #32735
2018-08-23 10:07:59 +02:00
Ignacio Vera d7219c05a2
Search: Support of wildcard on docvalue_fields (#32980)
* Search: Support of wildcard on docvalue_fields

For consistency with stored_fields, docvalue_fields should support the use of wildcards. 
Documentation of doc values fields is updated accordingly.

See also: #26390

Closes #26299
2018-08-23 10:04:00 +02:00
Jim Ferenczi ffe895e16e
Change query field expansion (#33020)
This commit changes the query field expansion for query parsers
to not rely on an hardcoded list of field types. Instead we rely on
the type of exception that is thrown by MappedFieldType#termQuery to
include/exclude an expanded field.

Supersedes #31655

Closes #31798
2018-08-23 09:52:48 +02:00
Armin Braun 46247ff1f9
INGEST: Cleanup Redundant Put Method (#33034) 2018-08-23 07:43:36 +02:00
Andrei Stefan de95dead2d
SQL: skip uppercasing/lowercasing function tests for AZ locales as well (#32910)
* Added the rest of the Locales that have different behavior for uppercasing/lowercasing scenarios to the skip list
2018-08-23 00:21:38 +03:00
Alpar Torok edd477a15e Fix the default pom file name (#33063)
Before this change the default was fixed at compile time and not picking
up changes in the build script.
2018-08-22 14:53:42 -04:00
Nik Everett 0cc99d270c
Switch ml basic tests to new style Requests (#32483)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/ml-basic-multi-node` project to use
the new versions.
2018-08-22 14:23:43 -04:00
Nik Everett c3438bc8d8
Switch some watcher tests to new style Requests (#33044)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/smoke-test-monitoring-with-watcher`,
`x-pack/qa/smoke-test-watcher`, and
`x-pack/qa/smoke-test-watcher-with-security` projects to use the new
versions.
2018-08-22 14:02:39 -04:00
Luca Cavanna 393eec1482
Set maxScore for empty TopDocs to Nan rather than 0 (#32938)
We used to set `maxScore` to `0` within `TopDocs` in situations where there is really no score as the size was set to `0` and scores were not even tracked. In such scenarios, `Float.Nan` is more appropriate, which gets converted to `max_score: null` on the REST layer. That's also more consistent with lucene which set `maxScore` to `Float.Nan` when merging empty `TopDocs` (see `TopDocs#merge`).
2018-08-22 17:23:54 +02:00
Dimitrios Liappis abb4c183f1
Clarify ignore_above behavior with arrays of strings
Currently docs don't explain how `ignore_above` behaves with arrays of
strings.

Clarify how `ignore_above` applies for arrays of strings and
also note that all string(s) will still be visible in the
`_source` field.

Relates #33057
2018-08-22 18:18:30 +03:00
Jason Tedor 528e727999
Fix method reference in comment in SchedulerEngine
This commit fixes the name of a method reference in a comment in
SchedulerEngine.
2018-08-22 10:20:20 -04:00
Michael Basnight e9912081dd
HLRC: Create server agnostic request and response (#32912)
The HLRC has historically reused the same Request and Response classes
that the server module uses. This commit deprecates the use of any
server module Request and Response classes, and adds a small bit of
validation logic that differs from server slightly, in that it does not
assume a check for a null ValidationException class is not enough to
determine if validation failed.
2018-08-22 09:19:58 -05:00
Jason Tedor 67bfb765ee
Refactor Netty4Utils#maybeDie (#33021)
In our Netty layer we have had to take extra precautions against Netty
catching throwables which prevents them from reaching the uncaught
exception handler. This code has taken on additional uses in NIO layer
and now in the scheduler engine because there are other components in
stack traces that could catch throwables and suppress them from reaching
the uncaught exception handler. This commit is a simple cleanup of the
iterative evolution of this code to refactor all uses into a single
method in ExceptionsHelper.
2018-08-22 10:18:07 -04:00
Simon Willnauer ead198bf2e
Add settings updater for 2 affix settings (#33050)
Today we can only have non-affix settings updated and consumed _together_.
Yet, there are use-cases where two affix settings depend on each other which
makes using the hard without consuming updates together. Unfortunately, there is
not straight forward way to have N settings updated together in a type-safe way
having 2 still serves a large portion of use-cases.
2018-08-22 14:13:27 +02:00
Nhat Nguyen 262d3c0783
Allow engine to recover from translog upto a seqno (#33032)
This change allows an engine to recover from its local translog up to
the given seqno. The extended API can be used in these use cases:

When a replica starts following a new primary, it resets its index to
the safe commit, then replays its local translog up to the current
global checkpoint (see #32867).

When a replica starts a peer-recovery, it can initialize the
start_sequence_number to the persisted global checkpoint instead of the
local checkpoint of the safe commit. A replica will then replay its
local translog up to that global checkpoint before accepting remote
translog from the primary. This change will increase the chance of
operation-based recovery. I will make this in a follow-up.

Relates #32867
2018-08-22 07:57:44 -04:00
Benjamin Trent b02150a5ed
HLRC: close job refactor (#33031)
* HLRC: close job refactor

* Changing refactor to make job_id a string

* Changing set entity methodology
2018-08-22 06:54:08 -05:00
Sergey 43f6f435f5 [DOCS] Update remote-info.asciidoc (#32978) 2018-08-22 11:29:10 +02:00
David Turner ab000323fa
Allow extension of CapturingTransport by subclasses (#33012)
Today, CapturingTransport#createCapturingTransportService creates a transport
service with a connection manager with reasonable default behaviours, but
overriding this behaviour in a consumer is a litle tricky. Additionally, the
default behaviour for opening a connection duplicates the content of the
CapturingTransport#openConnection() method.

This change removes this duplication by delegating to openConnection() and
introduces overridable nodeConnected() and onSendRequest() methods so that
consumers can alter this behaviour more easily.

Relates #32246 in which we test the mechanisms for opening connections to
unknown (and possibly unreachable) nodes.
2018-08-22 09:09:08 +01:00
Simon Willnauer ffb1a5d5b7
Expose `max_concurrent_shard_requests` in `_msearch` (#33016)
Today `_msearch` doesn't allow modifying the `max_concurrent_shard_requests`
per sub search request. This change adds support for setting this parameter on
all sub-search requests in an `_msearch`.

Relates to #31877
2018-08-22 08:45:08 +02:00
Alpar Torok 82d10b484a
Run forbidden api checks with runtimeJavaVersion (#32947)
Run forbidden APIs checks with runtime hava version
2018-08-22 09:05:22 +03:00
Ioannis Kakavas 0a4b55c9c0
[DOCS] Add RequestedAuthnContext Documentation (#32946)
Add documentation for #31238

- Add documentation for the req_authn_context_class_ref setting
- Add a section in SAML Guide regarding the use of SAML
  Authentication Context.
2018-08-22 08:37:50 +03:00
Yogesh Gaikwad 9f588c953f
[TEST] Split tests and skip file permission test on Windows (#32781)
Changes to split tests for keytab file test cases instead of
randomized testing for testing branches in the code in the
same test.
On windows platform, for keytab file permission test, we
required additional security permissions for the test
framework. As this was the only test that required those
permissions, skipping that test on windows platform.
The same scenario gets tested in *nix environments.

Closes#32768
2018-08-22 15:23:39 +10:00
Benjamin Trent e2ea83d217
HLRC: Add ML Get Job (#32960)
* HLRC: Adding GET ML Job info API

* HLRC: Adding GET Job ML API

* Fixing QueryPage license header

* Adding serialization tests, addressing minor issues

* Renaming querypage, changing the dependency on it

* Making things immutable

* Fixing build failure due to method rename
2018-08-21 21:02:28 -05:00
Russ Cam 07b3ff9fe7
Add beta label to MSI on install Elasticsearch page (#28126)
The main installation instructions page for the Windows MSI installer includes a header at the top to indicate that the installer is in beta, but the Installing Elasticsearch page does not. This commit adds the beta label to the MSI entry within the installation options.
2018-08-22 11:26:53 +10:00
Nik Everett 2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Julie Tibshirani 67b5a83a9a
Ensure that _exists queries on keyword fields use norms when they're available. (#33006) 2018-08-21 16:33:42 -07:00
Jim Ferenczi 767c69593c
Fix quoted _exists_ query (#33019)
This change in the `query_string` query fixes the detection of the special
`_exists_` field when it is used with a quoted term.

Closes #28922
2018-08-21 22:15:09 +02:00
debadair 15727ae8ed
[DOCS] Fixed formatting of Example headings. (#33038) 2018-08-21 13:13:07 -07:00
Jim Ferenczi 8b43e21521
Fix multi fields empty query (#33017)
This change fixes empty query removal when all fields remove the search term in
`simple_query_string`, `multi_match` and `query_string`.

Closes #33009
2018-08-21 22:12:53 +02:00
Nik Everett f311680176 Monitoring: Clean up MonitoringIT
We recently reenabled MonitoringIT to hunt down #29880 but some of its
assertions were out of date. This updates the assertions.
2018-08-21 16:04:16 -04:00
Igor Motov 3973bb4028
Fix north pole overflow error in GeoHashUtils.bbox() (#32891)
Fixes an overflow error in  GeoHashUtils.bbox() calculation of a
bounding box for geohashes with maximum precision located next to the
north pole.
2018-08-21 14:59:37 -04:00
Nik Everett fcf8cadd9a
Switch some x-pack tests to new style Requests (#32500)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/audit-tests`,
`x-pack/qa/ml-disabled`, and `x-pack/qa/multi-node` projects to use the
new versions.
2018-08-21 14:48:53 -04:00
Dimitris Athanasiou 28a0df2c7f
HLRC: Clear ML data after client tests (#33023)
This commit duplicates the `MlRestTestStateCleaner` to make sure
all ML data is removed after each test. After implementing
the job and datafeed APIs in the HLRC, we shall replace this
implementation with one using the HLRC itself.

Closes #32993
2018-08-21 18:12:28 +01:00
Jason Tedor 9623cf6cde
Find CCR QA sub-projects automatically (#33027)
Today we are by-hand maintaining a list of CCR QA sub-projects that the
check task depends on. This commit simplifies this by finding these
sub-projects automatically and adding their check task as dependencies
of the CCR check task.
2018-08-21 12:51:55 -04:00
Jason Tedor 28d12b05b7
Move ML tests to be sub-projects of ML (#33026)
This commit moves the ML QA tests to be a sub-project of ML. The purpose
of this refactoring is to enable ML developers to run
:x-pack:plugin:ml:check and run the vast majority of a ML tests with a
single command (this still does not contain the ML REST tests, nor the
upgrade tests). This simplifies local development for faster iteration.
2018-08-21 12:23:21 -04:00
Jason Tedor bdfcc326d7
Enable avoiding mmap bootstrap check (#32421)
The maximum map count boostrap check can be a hindrance to users that do
not own the underlying platform on which they are executing
Elasticsearch. This is because addressing it requires tuning the kernel
and a platform provider might now allow this, especially on shared
infrastructure. However, this bootstrap check is not needed if mmapfs is
not in use. Today we do not have a way for the user to communicate that
they are not going to use mmapfs. This commit therefore adds a setting
that enables the user to disallow mmapfs. When mmapfs is disallowed, the
maximum map count bootstrap check is not enforced. Additionally, we
fallback to a different default index store and prevent the explicit use
of mmapfs for an index.
2018-08-21 11:02:25 -04:00
Adrien Grand 5b446b81ef Reenable MonitoringIT#testMonitoringService.
When discussing this test, it made little sense that testMonitoringService
would fail but not testMonitoringBulk given their similarity. So we argeed to
enable it again.

Relates #29880
2018-08-21 16:53:59 +02:00
Ioannis Kakavas 1b583978e9
[DOCS] Add FIPS 140-2 documentation (#32928)
* Add relevant documentation for FIPS 140-2 compliance.
* Introduce `fips_mode` setting.
* Discuss necessary configuration for FIPS 140-2
* Discuss introduced limitations by FIPS 140-2
2018-08-21 16:20:00 +03:00
Benjamin Trent 3f91bbfa6b
[ML] Allowing _close to accept body payloads for options (#32989) (#33000) 2018-08-21 08:08:26 -05:00
markharwood 38bdf9ce32
HLRC GraphClient and associated tests (#32366)
GraphClient for the high level REST client and associated tests.
Part of #29827 work
2018-08-21 13:29:18 +01:00
Simon Willnauer 92076497e5
Use a dedicated ConnectionManger for RemoteClusterConnection (#32988)
This change introduces a dedicated ConnectionManager for every RemoteClusterConnection
such that there is not state shared with the TransportService internal ConnectionManager.
All connections to a remote cluster are isolated from the TransportService but still uses
the TransportService and it's internal properties like the Transport, tracing and internal
listener actions on disconnects etc.
This allows a remote cluster connection to have a different lifecycle than a local cluster connection,
also local discovery code doesn't get notified if there is a disconnect on from a remote cluster and
each connection can use it's own dedicated connection profile which allows to have a reduced set of
connections per cluster without conflicting with the local cluster.

Closes #31835
2018-08-21 12:43:25 +02:00
Ioannis Kakavas 65d4f27873
[DOCS] Add configurable password hashing docs (#32849)
* [DOCS] Add configurable password hashing docs

Adds documentation about the newly introduced configuration option
for setting the password hashing algorithm to be used for the users
cache and for storing credentials for the native and file realm.
2018-08-21 12:05:42 +03:00
Armin Braun 200078734c
INGEST: Simplify IngestService (#33008)
* INGEST: Simplify IngestService

* Follow up to #32617
* Flatten redundant inner classes of `IngestService`
2018-08-21 10:13:32 +02:00
Tim Vernum b595b1a20c
Handle 6.4.0+ BWC for Application Privileges (#32929)
When the application privileges feature was backported to 6.x/6.4 the
BWC version checks on the backport were updated to 6.4.0, but master
was not updated.

This commit updates all relevant version checks, and adds tests.
2018-08-21 17:58:37 +10:00
Yogesh Gaikwad 38886e8f23
[DOCS] Add Kerberos troubleshooting documentation (#32803)
This commit adds troubleshooting section for Kerberos.
Most of the times the problems seen are caused due to invalid
configurations like keytab missing principals or credentials
not up to date. Time synchronization is an important part for
Kerberos infrastructure and the time skew can cause problems.
To debug further documentation explains how to enable JAAS
Kerberos login module debugging and Kerberos/SPNEGO debugging
by setting JVM system properties.
2018-08-21 16:30:28 +10:00
Jason Tedor b08d02e3b7
Implement CCR licensing (#33002)
This commit implements licensing for CCR. CCR will require a platinum
license, and administrative endpoints will be disabled when a license is
non-compliant.
2018-08-20 23:33:18 -04:00