Commit Graph

50936 Commits

Author SHA1 Message Date
Dimitris Athanasiou 0049e9467b
[7.x][ML] Fix node serialization on GET df-nalytics stats without id (#54808) (#54812)
Previously, the id of the `GetDataFrameAnalyticsStatsAction.Request`
could be `null` which caused NPE on serialization as `writeString`
is used (it doesn't accept null values).

This commit ensures the id is never null.

Closes #54807

Backport of #54808
2020-04-06 18:13:16 +03:00
Nhat Nguyen 2fdbed7797 Broadcast cancellation to only nodes have outstanding child tasks (#54312)
Today when canceling a task we broadcast ban/unban requests to all nodes
in the cluster. This strategy does not scale well for hierarchical
cancellation. With this change, we will track outstanding child requests
and broadcast the cancellation to only nodes that have outstanding child
tasks. This change also prevents a parent task from sending child
requests once it got canceled.

Relates #50990
Supersedes #51157

Co-authored-by: Igor Motov <igor@motovs.org>
Co-authored-by: Yannick Welsch <yannick@welsch.lu>
2020-04-06 11:11:29 -04:00
István Zoltán Szabó 7dc1ba4273 [DOCS] Updates transform prerequisites (#54804) 2020-04-06 17:07:59 +02:00
Christoph Büscher def519ea70 [Docs] Correct date rounding example for `range` query (#51524)
Looking into #50237 I realized that two of the examples given in the
documentation around date math rounding for range queries on date fields using
`gt` and `lt` is slightly off by a nanosecond. This PR changes this to the
bounds that are currently parsed using these parameters.
2020-04-06 17:05:45 +02:00
Andrei Dan 816fec7187
Enable support for decompression of compressed response within RestHighLevelClient (#53533) (#54811)
Added decompression of gzip when gzip value is return as an header from Elasticsearch

(cherry picked from commit 4a195b573ab85d4e756669c953419ebdb3003442)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>

Co-authored-by: Hakky54 <hakangoudberg@hotmail.com>
2020-04-06 16:04:26 +01:00
István Zoltán Szabó 4cba1e6368 [DOCS] Changes kibana_user to kibana_admin in DFA API prerequisites. (#54806) 2020-04-06 15:46:18 +02:00
David Kyle 03bc368c14
Wait for ML templates after creating a new cluster in TooManyJobsIT (#54801) 2020-04-06 13:45:56 +01:00
David Turner 2b8a91b7be Ensure correct no-master block applied on restart (#54800)
This commit addresses a long-standing `// TODO` in the coordinator tests to
ensure that the correct no-master block is applied when a node restarts while
disconnected from the cluster.

It also strengthens this test to check that the no-master block is applied
correctly on all nodes, not just the previous master.
2020-04-06 13:25:51 +01:00
Dimitris Athanasiou ed4ef78330
[7.x][ML] Increase open job wait time in MlDistributedFailureIT (#54792) (#54798)
It seems the 20 seconds timeout is occasionally not enough.
We still get sporadic failures where the logs reveal the job
wasn't opened within 20 seconds. I'm increasing the wait time
to 30 seconds.

Closes #54448

Backport of #54792
2020-04-06 14:51:46 +03:00
David Turner 63de8c0730 Reinstate commented-out CoordinatorTest (#54784)
Test `testAckListenerReceivesNacksFromFollowerInHigherTerm` was suppressed as
when it was written it didn't work due the lack of proper term bumping. We
added term bumping but never got around to implementing this test. This commit
addresses this.
2020-04-06 10:27:13 +01:00
Armin Braun baff7bfa14
Rationalize some ThreadPool Use in Snapshot Transport Actions (#54772) (#54782)
Removing a few spots where we clearly don't have to fork to the generic or management
pool since either we only interpret the current cluster state or fork-off directly to
some other pool in the transport action logic anyway.
2020-04-06 09:57:41 +02:00
Tim Vernum 30b01fe00d
Resolve SSO roles by pattern (#54777)
This changes a SamlServiceProvider to have a function that maps
from an "action-name" to set of role-names instead of a Map that does
so.

The on-disk representation of this mapping is a set of Java Regexp
Patterns, for which the first matching group is the role name.

For example "sso:(\w+)" would map any action that started with "sso:"
to the corresponding role name (e.g. "sso:superuser" -> "superuser").

Backport of: #54440
2020-04-06 14:10:30 +10:00
Jason Tedor b939b47b77
Add wire tests for get autoscaling decision objects
This commit adds wire serializing tests for the get autoscaling decision
request and response objects.
2020-04-05 21:34:36 -04:00
Jason Tedor 8f520f0a9c
Add wire tests to delete autoscaling policy request
This commit adds some wire serializing tests for delete autoscaling
policy requests.
2020-04-05 21:34:35 -04:00
Jason Tedor 98c4165348
Add wire tests for put autoscaling policy request
This commit adds some wire serializing tests for put autoscaling policy
requests.
2020-04-05 21:34:34 -04:00
Tim Vernum cf442aae38
Resolve SERVICE_UNAVAILABLE in IdP IntegTest (#54700)
The SamlIdentityProviderTests IntegTests would sometimes encounter a
service unavailable exception when registering a new service provider.

This change ensure that there is a data node, and that the cluster
state is recovered before registering providers

Backport of: #54622
2020-04-06 11:23:08 +10:00
Jason Tedor b2cd858f29
Return 404s when autoscaling policies do not exist (#54774)
This commit updates the autoscaling get and delete policy APIs to return
404s when the named policy does not exist.
2020-04-05 21:05:11 -04:00
Costin Leau a7c31a7632 Test: Include ML ILM policy in EsRestTest (#54773)
This should avoid REST failures caused by the inability to delete said
policy
Fix #54759

(cherry picked from commit 3ba5e02b713c03b1bdf14e0367a2bce68c35dd30)
2020-04-05 20:10:17 +03:00
Jason Tedor 184c038f59
Add get autoscaling policy API (#54762)
This commit adds the get autoscaling policy API.
2020-04-04 18:04:25 -04:00
Nhat Nguyen 73d24203e7 Handle no such remote cluster exception in ccr (#53415)
A remote client can throw a NoSuchRemoteClusterException while fetching
the cluster state from the leader cluster. We also need to handle that
exception when retrying to add a retention lease to the leader shard.

Closes #53225
2020-04-04 13:55:06 -04:00
Nhat Nguyen 4ecc7dcca5 Avoid StackOverflowError if write circular reference exception (#54147)
We should never write a circular reference exception as we will fail a 
node with StackOverflowError. However, we have one in #53589. 
I tried but failed to find its location. With this commit, we will avoid 
StackOverflowError in production and detect circular exceptions in
tests.

Closes #53589
2020-04-04 13:42:27 -04:00
Jason Tedor 05c5529b2d
Clean up a few instances of "MetaData"
We recently cleaned up the use of the word "metadata" across the
codebase. A few additional uses have trickled in, likely from
in-progress work. This commit cleans up these last few instances.

Relates #54519
2020-04-04 10:55:09 -04:00
Jason Tedor 2a94672c32
Separate autoscaling REST test cases
A couple of the autoscaling REST tests combine multiple tests into a
single REST test. This commit separates them in to single tests.
2020-04-04 10:21:21 -04:00
Jason Tedor d5a195ab3d
Rename the policies in put autoscaling REST tests
The autoscaling REST tests use policies named "hot" in their test
cases. Instead, this commit changes the name of these policies to
"my_autoscaling_policy".
2020-04-04 10:14:11 -04:00
Jason Tedor 79c72cd398
Migrate common autoscaling test code to base class
This commit moves some code repeated in a few autoscaling tests related
to writeable and x-content registries to the autoscaling tests base
class.
2020-04-04 09:57:20 -04:00
Jason Tedor dd99e6d951
Simplify name of delete autoscaling policy handler
The name here is unnecessarily long, containing the word "action" when
it does not need to. This commit simplifies the name.
2020-04-03 21:46:48 -04:00
Lee Hinman 814c248819
[7.x] Use V2 index templates during index creation (#54669) (#54750)
* Use V2 index templates during index creation

This commit changes our index creation code to use (and favor!) V2 index templates during index
creation. The creation precedence goes like so, in order of precedence:

- Existing source `IndexMetadata` - for example, when recovering from a peer or a shrink/split/clone
  where index templates should not be applied
- A matching V2 index template, if one is found
  - When a V2 template is found, all component templates (in the `composed_of` field) are applied
    in the order that they appear, with the index template having the 2nd highest precedence (the
    create index request always has the top priority when it comes to index settings)
- All matching V1 templates (the old style)

This also adds index template validation when `PUT`-ing a new v2 index template (because this was
required) and ensures that all index and component templates specify *no* top-level mapping type (it
is automatically added when the template is added to the cluster state).

This does not yet implement fine-grained component template merging of mappings, where we favor
merging only a single field's configuration, that will be done in subsequent work.

This also keeps the existing hidden index behavior present for v1 templates, where a hidden index
will match v2 index templates unless they are global (`*`) templates.

Relates to #53101
2020-04-03 14:46:15 -06:00
James Baiera 548145f4a3
Cat tasks output should respect time display settings (#54536) (#54735) 2020-04-03 15:34:44 -04:00
James Baiera 705e46d5c1
[DOCS] Remove unused cat tasks request parameters (#54539) (#54741) 2020-04-03 15:33:28 -04:00
James Baiera 2fa0d4cf3c
Move Fixture to minimumRuntime source (#54732) 2020-04-03 15:08:25 -04:00
Lisa Cawley de91d2aeea [DOCS] Collapse nested objects in CCR APIs (#54697) 2020-04-03 12:04:33 -07:00
Ross Wolf 022f829d84
EQL: Add wildcard function (#54020)
* EQL: Add wildcard function
* EQL: Cleanup Wildcard.getArguments
* EQL: Cleanup Wildcard and rearrange methods
* EQL: Wildcard newline lint
* EQL: Make StringUtils function final
* EQL: Make Wildcard.asLikes return ScalarFunction
* QL: Restore BinaryLogic.java
* EQL: Add Wildcard PR feedback
* EQL: Add Wildcard verification tests
* EQL: Switch wildcard to isFoldable test
* EQL: Change wildcard test to numeric field
* EQL: Remove Wildcard.get_arguments
2020-04-03 10:15:43 -06:00
Rory Hunter 83e900e248 Tweak formatting config for exception lists (#54710)
Change how we format exceptions to only wrap them as necessary. While
the config's overall philosophy is to put items one-per-line when
wrapping, in practice this is a little cumbersome for exception lists.
2020-04-03 16:54:31 +01:00
Ioannis Kakavas 8e255337f8
Fix SamlServiceProviderDocumentTests (#54718) (#54723)
Don't assume byte for byte equality because internal structures
 do not guarantee order
2020-04-03 18:46:36 +03:00
Dan Hermann 18fef3de2a
Get data stream accepts single search parameter 2020-04-03 10:36:26 -05:00
Christoph Büscher 8c9ac14a98
Rename field name constants in AbstractBuilderTestCase (#53234)
Some field name constants were not updaten when we moved from "string" to "text"
and "keyword" fields. Renaming them makes it easier and faster to know which
field type is used in test subclassing this base test case.
2020-04-03 17:28:22 +02:00
David Roberts 470aa9a5f1 [TEST] Mute CategorizationIT.testNumMatchesAndCategoryPreference (#54717)
The test results are affected by the off-by-one error that is
fixed by https://github.com/elastic/ml-cpp/pull/1122

This test can be unmuted once that fix is merged and has been
built into ml-cpp snapshots.
2020-04-03 14:40:47 +01:00
Dimitris Athanasiou e8c0351fd8
[7.x][ML] Allow force stopping failed and stopping DF analytics (#54650) (#54712)
Force stopping a failed job used to work but it
now puts the job in `stopping` state and hangs.
In addition, force stopping a `stopping` job is
not handled.

This commit addresses those issues with force
stopping data frame analytics. It inlines the
approach with that followed for anomaly detection
jobs.

Backport of #54650
2020-04-03 16:08:06 +03:00
Bogdan Pintea 7cef89e084 ODBC: Document the new VarcharLimit and EarlyExecution params (#54632)
* Document VarcharLimit and EarlyExecution params

Add the documentation for the newly added VarcharLimit and
EarlyExecution DSN attributes.

* Remove obsolete VersionChecking param

This param had been removed already along the #53082 work.

* Update docs/reference/sql/endpoints/odbc/configuration.asciidoc

fix typo

Co-Authored-By: Stuart Cam <stuart@codebrain.co.uk>

* Update docs/reference/sql/endpoints/odbc/configuration.asciidoc

fix typo

Co-Authored-By: Stuart Cam <stuart@codebrain.co.uk>
(cherry picked from commit f38761631a12b38f7f075635f7ac61dc96656cd7)
2020-04-03 14:46:39 +02:00
Nik Everett 195345b09e
Fix InternalAutoDateHistogramTests (#54602) (#54687)
The test had errors around time units that have different length - think
leap years or months that aren't 30 days. This fixes those errors. In
the proces I've changed a bunch of things to debug the problem:

* Replace `currentTimeMillis` with a random time. Now the test fails
  randomly! Wonderful. Much better than on random days of the month.
* Generate buckets "closer together" to test random reduction. Without
  this we were super frequently getting stuck in the "year of century"
  rounding because *some* of the of the buckets we built were far apart.
  This generates a much greater variety of tests.
* Implement `toString` on `RoundingInfo` so I can debug without going
  crazy.
* Switch keys in the bucket assertions from epoch millis to `Instant`s
  so we can read the failures.

Closes #54540
Closes #39497
2020-04-03 08:22:08 -04:00
Maria Ralli aa697346c4 Remove Xlint exclusions from gradle files (part 2)
Backport of #54576.

This commit is part of issue #40366 to remove disabled Xlint warnings
from gradle files. Remove the Xlint exclusions from the following files:

- x-pack/plugin/rollup/build.gradle
- x-pack/plugin/monitoring/build.gradle
- x-pack/qa/rolling-upgrade-basic/build.gradle

Add type parameters to parameterized types. Add wildcard-type parameters
or bounded wildcard-type parameters. Suppress `unchecked` and `rawtypes`
warnings at method level.
2020-04-03 12:15:42 +01:00
markharwood 2da2305587
Backport of lowercase normalizer PR #53882
A pre-configured normalizer for lower-casing.
Closes #53872
2020-04-03 11:43:40 +01:00
Christoph Büscher 9f22c0d37c Fix Eclipse compile problem in ModelLoadingService (#54670)
Current Eclipse 4.14.0 cannot deal with the direct lambda notation, changing to
an exlicite one.
2020-04-03 11:56:30 +02:00
Martijn van Groningen ec0bbda52f
Changed itv2 and data streams feature flag naming (#54431) (#54500)
from `*_flag_registered` to `#_feature_enabled`.

This previous name indicated that a flag was registered,
whilst the feature flag actually controls whether a
feature is enabled.
2020-04-03 10:12:00 +02:00
István Zoltán Szabó d025b90cd1 [DOCS] Makes PUT inference API docs collapsible (#54653)
Co-authored-by: lcawl <lcawley@elastic.co>
2020-04-03 09:48:53 +02:00
Lisa Cawley 11afead21e [DOCS] Adds collapsible sections to rollup APIs (#54690) 2020-04-02 17:51:16 -07:00
Mark Vieira 7666276b09
Enable the remote build cache in BWC distribution builds (#54691) 2020-04-02 17:05:56 -07:00
Jason Tedor f2590b9984
Workaround JDK 14 compiler bug (#54689)
This commit workarounds a bug in the JDK 14 compiler. It is choking on a
method reference, so we substitute a lambda expression instead. The JDK
bug ID is 9064309.
2020-04-02 19:45:52 -04:00
Lisa Cawley b138dc4565
[DOCS] Add processing details to get transforms stats API (#54368) (#54595) 2020-04-02 16:12:57 -07:00
lcawl d3fa0ec730 [DOCS] Fixes typo in node settings 2020-04-02 16:01:54 -07:00