512 Commits

Author SHA1 Message Date
Tal Levy
13a0d822d0 Merge branch 'master' into index-lifecycle 2018-08-30 15:04:17 -07:00
Costin Leau
83c3d7a6cf
SQL: prevent duplicate generation for repeated aggs (#33252)
Prevent generation of duplicate aggs caused by repetitive functions,
leading to invalid query.

Fix #30287
2018-08-31 00:13:03 +03:00
Nhat Nguyen
07faa0b06a TEST: Mute testMonitorClusterHealth
Tracked at #32299
2018-08-30 14:26:32 -04:00
Zachary Tong
d93b2a2e9a
[Rollup] Only allow aggregating on multiples of configured interval (#32052)
We need to limit the search request aggregations to whole multiples
of the configured interval for both histogram and date_histogram.
Otherwise, agg buckets won't overlap with the rolled up buckets
and the results will be incorrect.

For histogram, the validation is very simple: request must be >= the config,
and modulo evenly.

Dates are more tricky.
- If both request and config are fixed dates, we can convert to millis
and treat them just like the histo
- If both are calendar, we make sure the request is >= the config with
a static lookup map that ranks the calendar values relatively.  All
calendar units are "singles", so they are evenly divisible already
- We disallow any other combination (one fixed, one calendar, etc)
2018-08-29 17:10:00 -04:00
Gordon Brown
454ce99b01 Merge branch 'master' into index-lifecycle 2018-08-29 08:28:23 -06:00
Jay Modi
1e11b05b58
Remove unused User class from protocol (#33137)
This commit removes the unused User class from the protocol project.
This class was originally moved into protocol in preparation for moving
more request and response classes, but given the change in direction
for the HLRC this is no longer needed. Additionally, this change also
changes the package name for the User object in x-pack/plugin/core to
its original name.
2018-08-28 08:55:29 -06:00
Gordon Brown
50368656ee Merge branch 'master' into index-lifecycle 2018-08-27 15:35:19 -06:00
Jay Modi
5d9c270608
Token API supports the client_credentials grant (#33106)
This change adds support for the client credentials grant type to the
token api. The client credentials grant allows for a client to
authenticate with the authorization server and obtain a token to access
as itself. Per RFC 6749, a refresh token should not be included with
the access token and as such a refresh token is not issued when the
client credentials grant is used.

The addition of the client credentials grant will allow users
authenticated with mechanisms such as kerberos or PKI to obtain a token
that can be used for subsequent access.
2018-08-27 10:56:21 -06:00
Tal Levy
5783545222 Merge branch 'master' into index-lifecycle 2018-08-27 08:19:05 -07:00
Andrei Stefan
3d9ca4baee
SQL: Enable aggregations to create a separate bucket for missing values (#32832)
Enable aggregations to create a separate bucket for missing values.
2018-08-27 17:56:28 +03:00
Shaunak Kashyap
1779d3376a
APM server monitoring (#32515)
* Adding new MonitoredSystem for APM server

* Teaching Monitoring template utils about APM server monitoring indices

* Documenting new monitoring index for APM server

* Adding monitoring index template for APM server

* Copy pasta typo

* Removing metrics.libbeat.config section from mapping

* Adding built-in user and role for APM server user

* Actually define the role :)

* Adding missing import

* Removing index template and system ID for apm server

* Shortening line lengths

* Updating expected number of built-in users in integration test

* Removing "system" from role and user names

* Rearranging users to make tests pass
2018-08-27 08:42:40 -04:00
Nik Everett
1e9144d8e6
Switch remaining x-pack tests to new style Requests (#33108)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/saml-idp-tests` and
`x-pack/qa/security-setup-password-tests` projects to use the new
versions.
2018-08-24 16:39:08 -04:00
Gordon Brown
1f13c77b49 Merge branch 'master' into index-lifecycle 2018-08-23 11:52:59 -06:00
Andrei Stefan
de95dead2d
SQL: skip uppercasing/lowercasing function tests for AZ locales as well (#32910)
* Added the rest of the Locales that have different behavior for uppercasing/lowercasing scenarios to the skip list
2018-08-23 00:21:38 +03:00
Nik Everett
c3438bc8d8
Switch some watcher tests to new style Requests (#33044)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/smoke-test-monitoring-with-watcher`,
`x-pack/qa/smoke-test-watcher`, and
`x-pack/qa/smoke-test-watcher-with-security` projects to use the new
versions.
2018-08-22 14:02:39 -04:00
Alpar Torok
82d10b484a
Run forbidden api checks with runtimeJavaVersion (#32947)
Run forbidden APIs checks with runtime hava version
2018-08-22 09:05:22 +03:00
Nik Everett
2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Nik Everett
fcf8cadd9a
Switch some x-pack tests to new style Requests (#32500)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. This
changes all calls in the `x-pack/qa/audit-tests`,
`x-pack/qa/ml-disabled`, and `x-pack/qa/multi-node` projects to use the
new versions.
2018-08-21 14:48:53 -04:00
Jason Tedor
28d12b05b7
Move ML tests to be sub-projects of ML (#33026)
This commit moves the ML QA tests to be a sub-project of ML. The purpose
of this refactoring is to enable ML developers to run
:x-pack:plugin:ml:check and run the vast majority of a ML tests with a
single command (this still does not contain the ML REST tests, nor the
upgrade tests). This simplifies local development for faster iteration.
2018-08-21 12:23:21 -04:00
Benjamin Trent
3f91bbfa6b
[ML] Allowing _close to accept body payloads for options (#32989) (#33000) 2018-08-21 08:08:26 -05:00
Colin Goodheart-Smithe
10c60fae93
Merge branch 'master' into index-lifecycle 2018-08-21 11:54:06 +01:00
Jason Tedor
ad0a965db9
Protect scheduler engine against throwing listeners (#32998)
There are two problems with the scheduler engine today. Both relate to
listeners that throw.

The first problem is that any triggered listener that throws a plain old
exception will cause no additional listeners to be triggered for the
event, and will also cause the scheduler to never be invoked again. This
leads to lost events and is bad.

The second problem is that any triggered listener that throws an error
of the fatal kind will not lead to that error because caught by the
uncaught exception handler. This is because the triggered listener is
executed as a future task under a scheduled thread pool executor. A
throwable there goes caught by the JDK framework and set as the outcome
on the future task. Since we never inspect these tasks for their
outcomes, nor is there a good place to do this, we have to handle these
errors ourselves. To do this, we catch them and dispatch them to the
uncaught exception handler via a forked thread. This is similar to our
handling in Netty.
2018-08-20 22:07:16 -04:00
Tal Levy
a26e108590 Merge branch 'master' into index-lifecycle 2018-08-17 13:57:28 -07:00
Benjamin Trent
647705e00a Bypassing failing test PainlessDomainSplitIT#testHRDSplit (#32966) 2018-08-17 15:30:31 -05:00
Dimitris Athanasiou
46c35db1df [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature
As the multibucket feature was merged in, this test hit a side effect
which means buckets trailing an anomaly could become anomalous.
This commit fixes the problem by filtering low score records when
we request them.
2018-08-17 17:52:29 +01:00
Ioannis Kakavas
e3aa68b0a9
[TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901)
Elasticsearch versions earlier than 6.4.0 cannot properly run in a
FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for
nodes that we spin up in BWC tests even when we're testing FIPS.
2018-08-17 18:23:13 +03:00
Benjamin Trent
9cec4aa14b
[ML] fix updating opened jobs scheduled events (#31651) (#32881)
* ML: fix updating opened jobs scheduled events (#31651)

* Adding UpdateParamsTests license header

* Adding integration test and addressing PR comments

* addressing test and job names
2018-08-17 07:21:17 -05:00
Tal Levy
711ac402b6 Merge branch 'master' into index-lifecycle 2018-08-16 15:08:22 -07:00
Ed Savage
62559d2b3c
Re enable ml bwc tests (#32916)
[ML] Re-enabling BWC tests

Re-enable BWC tests for ML now that #32816 has been backported to 6.x
2018-08-16 18:23:26 +01:00
Ed Savage
d604b3e3a1 Temporarily disabled ML BWC tests for backporting
https://github.com/elastic/elasticsearch/pull/32816
2018-08-16 17:18:51 +01:00
Tal Levy
c9de707f58 Merge branch 'master' into index-lifecycle 2018-08-16 08:41:57 -07:00
Adrien Grand
eaaf37a1f9 AwaitFix FullClusterRestartIT#testRollupIDSchemeAfterRestart. 2018-08-16 16:35:10 +02:00
Jay Modi
1a45b27d8b
Move CharArrays to core lib (#32851)
This change cleans up some methods in the CharArrays class from x-pack, which
includes the unification of char[] to utf8 and utf8 to char[] conversions that
intentionally do not use strings. There was previously an implementation in
x-pack and in the reloading of secure settings. The method from the reloading
of secure settings was adopted as it handled more scenarios related to the
backing byte and char buffers that were used to perform the conversions. The
cleaned up class is moved into libs/core to allow it to be used by requests
that will be migrated to the high level rest client.

Relates #32332
2018-08-15 15:26:00 -06:00
Tal Levy
ec93756600 Merge branch 'master' into index-lifecycle 2018-08-15 12:56:01 -07:00
Ed Savage
51cece1900 [ML] Temporarily disabling rolling-upgrade tests
BWC tests disabled while backporting https://github.com/elastic/elasticsearch/pull/32816
2018-08-15 15:48:50 +01:00
Lee Hinman
48281ac5bc
Use generic AcknowledgedResponse instead of extended classes (#32859)
This removes custom Response classes that extend `AcknowledgedResponse` and do nothing, these classes are not needed and we can directly use the non-abstract super-class instead.

While this appears to be a large PR, no code has actually changed, only class names have been changed and entire classes removed.
2018-08-15 08:06:14 -06:00
Tal Levy
e78f537e58 Merge branch 'master' into index-lifecycle 2018-08-13 16:31:40 -07:00
Nhat Nguyen
cba7fceb53 AwaitsFix case-functions.sql-spec
Relates #32589
2018-08-13 15:06:33 -04:00
Zachary Tong
4da0b52a19
[TEST] Force a stop to save rollup state before continuing (#32787)
We only upgrade the ID when the state is saved in one of four scenarios:

- when we reach a checkpoint (every 50 pages)
- when we run out of data
- when explicitly stopped
- on failure

The test was relying on the pre-upgrade to finish, save state and then
the post-upgrade to start, hit the end of data and upgrade ID.  THEN
get the new doc and apply the new ID.

But I think this is vulnerable to timing issues. If the pre-upgrade
portion shutdown before it saved the state, when restarting we would run
through all the data from the beginning with the old ID, meaning both
docs would still have the old scheme.

This change makes the pre-upgrade wait for the job to go back to STARTED
so that we know it persisted the end point.  Post-upgrade, it stops and
restarts the job to ensure the state was persisted and the ID upgraded.

That _should_ rule out the above timing issue.

Closes #32773
2018-08-13 12:53:22 -04:00
Tal Levy
a771478940 Merge branch 'master' into index-lifecycle 2018-08-13 09:14:00 -07:00
Benjamin Trent
b08416b899
Clear Job#finished_time when it is opened (#32605) (#32755)
* Clear Job#finished_time when it is opened (#32605)

* not returning failure when Job#finished_time is not reset

* Changing error log string and source string
2018-08-10 13:52:00 -05:00
Tal Levy
93637e2135 Merge branch 'master' into index-lifecycle 2018-08-10 10:23:14 -07:00
Alexander Reelsen
e49717393a
Tests: Disable rolling upgrade tests with system key on fips JVM (#32775)
This disables the x-pack rolling upgrade tests using a fips JVM, as
there are problems creating the keystore.

Relates #32737
2018-08-10 16:50:11 +02:00
Tal Levy
c7a2c357a3 Merge branch 'master' into index-lifecycle 2018-08-09 18:00:40 -07:00
Tal Levy
41e6d98af8
move qa yaml tests to inside the ILM plugin (#32693)
The qa tests with security haven't actually gone as far as testing security roles yet, so this is a start in the hopes of both bringing the tests into the ilm plugin
2018-08-09 16:09:20 -07:00
Alpar Torok
5bbed5ed9a Merge branch 'master' of github.com:elastic/elasticsearch 2018-08-09 21:58:03 +03:00
Alpar Torok
c5e66cb345 mute test #32737 2018-08-09 21:54:58 +03:00
Andrei Stefan
6750e15773
SQL: Ignore H2 comparative tests for uppercasing/lowercasing string functions (#32604)
Skip the comparative tests using lowercasing/uppercasing against H2 (which considers the Locale).
ES-SQL is, so far, ignoring the Locale.
Still, the same queries are executed against ES-SQL alone and results asserted to be correct.
2018-08-09 19:28:48 +03:00
Colin Goodheart-Smithe
b9c04adb29
Merge branch 'master' into index-lifecycle 2018-08-07 12:35:32 +01:00
Yogesh Gaikwad
615aa85f4e
[Kerberos] Use canonical host name (#32588)
The Apache Http components support for Spnego scheme
uses canonical name by default.
Also when resolving host name, on centos by default
there are other aliases so adding them to the
DelegationPermission.

Closes#32498
2018-08-06 23:51:43 +10:00