Commit Graph

51197 Commits

Author SHA1 Message Date
Nhat Nguyen 22be925f58 Add more assertion for testRecoverLocallyUpToGlobalCheckpoint
Tracked at #54829
2020-04-06 19:42:26 -04:00
Costin Leau 99846f47b7 QL: Introduce infrastructure for surrogate functions (#54795)
Some functions act as shortcuts for more verbose declarations (sometimes
with certain constraints). This PR removes the boilerplate around
declaring such functions as well as a dedicated rule for the optimizer
to perform the actual substitution.

Fix #54334

(cherry picked from commit 3231d01b0c583deb89252fafe84db48878da3246)
2020-04-07 00:46:50 +03:00
Mayya Sharipova 0013dd4528
Add checks for field collapse test failure (#54831)
There were some failures on 7.x of field collapse tests,
where total hits count was less then expected.
This adds an additional test to check total hits count
before field collapse queries to understand if the problem
is with field collapsing or with simply that writes have
not been finished yet

Relates to #52416
2020-04-06 17:14:17 -04:00
Lisa Cawley b3d5300968
[DOCS] Collapses sections in put snapshot lifecycle policy API (#54834) (#54840) 2020-04-06 13:46:56 -07:00
Costin Leau 36121117f0 EQL: Sequence/Join parsing and model (#54227)
Add parsing and (logical) domain model for sequence and join

(cherry picked from commit 9e9632d41a39877256c68634ab18e441f4b67fe8)
2020-04-06 23:15:35 +03:00
James Rodewig e9c3bfc8e5 [DOCS] Collapse nested objects in node stats API response (#54755)
Replaces dot notation with collapsed nested object formatting
per the [Elastic API reference template][0].

[0]:https://github.com/elastic/docs/blob/master/shared/api-ref-ex.asciidoc
2020-04-06 15:19:54 -04:00
Przemko Robakowski 416267c038
Remove RestController from tests where it's not needed (#53782) (#54833) 2020-04-06 21:12:05 +02:00
Przemko Robakowski 7b1bb9952a
[7.x] HLRC support for Component Templates APIs (#54635) (#54828)
* HLRC support for Component Templates APIs (#54635)
2020-04-06 20:24:23 +02:00
Igor Motov 1aa87cd4a9
EQL: Make EQL search task cancellable (#54598)
First step towards async search execution. At the moment we don't try to cancel
the underlying search requests, and just check if the task is canceled before
performing network operation (such as field caps and search)

Relates to #49638
2020-04-06 13:38:03 -04:00
James Rodewig 548ad03941 [DOCS] Collapse nested objects in cluster stats API response (#54739)
Replaces dot notation with collapsed nested object formatting
per the [Elastic API reference template][0].

[0]:https://github.com/elastic/docs/blob/master/shared/api-ref-ex.asciidoc
2020-04-06 13:11:46 -04:00
Igor Motov 2794572a35
[7.x] Add Student's t-test aggregation support (#54469) (#54737)
Adds t_test metric aggregation that can perform paired and unpaired two-sample
t-tests. In this PR support for filters in unpaired is still missing. It will
be added in a follow-up PR.

Relates to #53692
2020-04-06 11:36:47 -04:00
Dimitris Athanasiou 0049e9467b
[7.x][ML] Fix node serialization on GET df-nalytics stats without id (#54808) (#54812)
Previously, the id of the `GetDataFrameAnalyticsStatsAction.Request`
could be `null` which caused NPE on serialization as `writeString`
is used (it doesn't accept null values).

This commit ensures the id is never null.

Closes #54807

Backport of #54808
2020-04-06 18:13:16 +03:00
Nhat Nguyen 2fdbed7797 Broadcast cancellation to only nodes have outstanding child tasks (#54312)
Today when canceling a task we broadcast ban/unban requests to all nodes
in the cluster. This strategy does not scale well for hierarchical
cancellation. With this change, we will track outstanding child requests
and broadcast the cancellation to only nodes that have outstanding child
tasks. This change also prevents a parent task from sending child
requests once it got canceled.

Relates #50990
Supersedes #51157

Co-authored-by: Igor Motov <igor@motovs.org>
Co-authored-by: Yannick Welsch <yannick@welsch.lu>
2020-04-06 11:11:29 -04:00
István Zoltán Szabó 7dc1ba4273 [DOCS] Updates transform prerequisites (#54804) 2020-04-06 17:07:59 +02:00
Christoph Büscher def519ea70 [Docs] Correct date rounding example for `range` query (#51524)
Looking into #50237 I realized that two of the examples given in the
documentation around date math rounding for range queries on date fields using
`gt` and `lt` is slightly off by a nanosecond. This PR changes this to the
bounds that are currently parsed using these parameters.
2020-04-06 17:05:45 +02:00
Andrei Dan 816fec7187
Enable support for decompression of compressed response within RestHighLevelClient (#53533) (#54811)
Added decompression of gzip when gzip value is return as an header from Elasticsearch

(cherry picked from commit 4a195b573ab85d4e756669c953419ebdb3003442)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>

Co-authored-by: Hakky54 <hakangoudberg@hotmail.com>
2020-04-06 16:04:26 +01:00
István Zoltán Szabó 4cba1e6368 [DOCS] Changes kibana_user to kibana_admin in DFA API prerequisites. (#54806) 2020-04-06 15:46:18 +02:00
David Kyle 03bc368c14
Wait for ML templates after creating a new cluster in TooManyJobsIT (#54801) 2020-04-06 13:45:56 +01:00
David Turner 2b8a91b7be Ensure correct no-master block applied on restart (#54800)
This commit addresses a long-standing `// TODO` in the coordinator tests to
ensure that the correct no-master block is applied when a node restarts while
disconnected from the cluster.

It also strengthens this test to check that the no-master block is applied
correctly on all nodes, not just the previous master.
2020-04-06 13:25:51 +01:00
Dimitris Athanasiou ed4ef78330
[7.x][ML] Increase open job wait time in MlDistributedFailureIT (#54792) (#54798)
It seems the 20 seconds timeout is occasionally not enough.
We still get sporadic failures where the logs reveal the job
wasn't opened within 20 seconds. I'm increasing the wait time
to 30 seconds.

Closes #54448

Backport of #54792
2020-04-06 14:51:46 +03:00
David Turner 63de8c0730 Reinstate commented-out CoordinatorTest (#54784)
Test `testAckListenerReceivesNacksFromFollowerInHigherTerm` was suppressed as
when it was written it didn't work due the lack of proper term bumping. We
added term bumping but never got around to implementing this test. This commit
addresses this.
2020-04-06 10:27:13 +01:00
Armin Braun baff7bfa14
Rationalize some ThreadPool Use in Snapshot Transport Actions (#54772) (#54782)
Removing a few spots where we clearly don't have to fork to the generic or management
pool since either we only interpret the current cluster state or fork-off directly to
some other pool in the transport action logic anyway.
2020-04-06 09:57:41 +02:00
Tim Vernum 30b01fe00d
Resolve SSO roles by pattern (#54777)
This changes a SamlServiceProvider to have a function that maps
from an "action-name" to set of role-names instead of a Map that does
so.

The on-disk representation of this mapping is a set of Java Regexp
Patterns, for which the first matching group is the role name.

For example "sso:(\w+)" would map any action that started with "sso:"
to the corresponding role name (e.g. "sso:superuser" -> "superuser").

Backport of: #54440
2020-04-06 14:10:30 +10:00
Jason Tedor b939b47b77
Add wire tests for get autoscaling decision objects
This commit adds wire serializing tests for the get autoscaling decision
request and response objects.
2020-04-05 21:34:36 -04:00
Jason Tedor 8f520f0a9c
Add wire tests to delete autoscaling policy request
This commit adds some wire serializing tests for delete autoscaling
policy requests.
2020-04-05 21:34:35 -04:00
Jason Tedor 98c4165348
Add wire tests for put autoscaling policy request
This commit adds some wire serializing tests for put autoscaling policy
requests.
2020-04-05 21:34:34 -04:00
Tim Vernum cf442aae38
Resolve SERVICE_UNAVAILABLE in IdP IntegTest (#54700)
The SamlIdentityProviderTests IntegTests would sometimes encounter a
service unavailable exception when registering a new service provider.

This change ensure that there is a data node, and that the cluster
state is recovered before registering providers

Backport of: #54622
2020-04-06 11:23:08 +10:00
Jason Tedor b2cd858f29
Return 404s when autoscaling policies do not exist (#54774)
This commit updates the autoscaling get and delete policy APIs to return
404s when the named policy does not exist.
2020-04-05 21:05:11 -04:00
Costin Leau a7c31a7632 Test: Include ML ILM policy in EsRestTest (#54773)
This should avoid REST failures caused by the inability to delete said
policy
Fix #54759

(cherry picked from commit 3ba5e02b713c03b1bdf14e0367a2bce68c35dd30)
2020-04-05 20:10:17 +03:00
Jason Tedor 184c038f59
Add get autoscaling policy API (#54762)
This commit adds the get autoscaling policy API.
2020-04-04 18:04:25 -04:00
Nhat Nguyen 73d24203e7 Handle no such remote cluster exception in ccr (#53415)
A remote client can throw a NoSuchRemoteClusterException while fetching
the cluster state from the leader cluster. We also need to handle that
exception when retrying to add a retention lease to the leader shard.

Closes #53225
2020-04-04 13:55:06 -04:00
Nhat Nguyen 4ecc7dcca5 Avoid StackOverflowError if write circular reference exception (#54147)
We should never write a circular reference exception as we will fail a 
node with StackOverflowError. However, we have one in #53589. 
I tried but failed to find its location. With this commit, we will avoid 
StackOverflowError in production and detect circular exceptions in
tests.

Closes #53589
2020-04-04 13:42:27 -04:00
Jason Tedor 05c5529b2d
Clean up a few instances of "MetaData"
We recently cleaned up the use of the word "metadata" across the
codebase. A few additional uses have trickled in, likely from
in-progress work. This commit cleans up these last few instances.

Relates #54519
2020-04-04 10:55:09 -04:00
Jason Tedor 2a94672c32
Separate autoscaling REST test cases
A couple of the autoscaling REST tests combine multiple tests into a
single REST test. This commit separates them in to single tests.
2020-04-04 10:21:21 -04:00
Jason Tedor d5a195ab3d
Rename the policies in put autoscaling REST tests
The autoscaling REST tests use policies named "hot" in their test
cases. Instead, this commit changes the name of these policies to
"my_autoscaling_policy".
2020-04-04 10:14:11 -04:00
Jason Tedor 79c72cd398
Migrate common autoscaling test code to base class
This commit moves some code repeated in a few autoscaling tests related
to writeable and x-content registries to the autoscaling tests base
class.
2020-04-04 09:57:20 -04:00
Jason Tedor dd99e6d951
Simplify name of delete autoscaling policy handler
The name here is unnecessarily long, containing the word "action" when
it does not need to. This commit simplifies the name.
2020-04-03 21:46:48 -04:00
Lee Hinman 814c248819
[7.x] Use V2 index templates during index creation (#54669) (#54750)
* Use V2 index templates during index creation

This commit changes our index creation code to use (and favor!) V2 index templates during index
creation. The creation precedence goes like so, in order of precedence:

- Existing source `IndexMetadata` - for example, when recovering from a peer or a shrink/split/clone
  where index templates should not be applied
- A matching V2 index template, if one is found
  - When a V2 template is found, all component templates (in the `composed_of` field) are applied
    in the order that they appear, with the index template having the 2nd highest precedence (the
    create index request always has the top priority when it comes to index settings)
- All matching V1 templates (the old style)

This also adds index template validation when `PUT`-ing a new v2 index template (because this was
required) and ensures that all index and component templates specify *no* top-level mapping type (it
is automatically added when the template is added to the cluster state).

This does not yet implement fine-grained component template merging of mappings, where we favor
merging only a single field's configuration, that will be done in subsequent work.

This also keeps the existing hidden index behavior present for v1 templates, where a hidden index
will match v2 index templates unless they are global (`*`) templates.

Relates to #53101
2020-04-03 14:46:15 -06:00
James Baiera 548145f4a3
Cat tasks output should respect time display settings (#54536) (#54735) 2020-04-03 15:34:44 -04:00
James Baiera 705e46d5c1
[DOCS] Remove unused cat tasks request parameters (#54539) (#54741) 2020-04-03 15:33:28 -04:00
James Baiera 2fa0d4cf3c
Move Fixture to minimumRuntime source (#54732) 2020-04-03 15:08:25 -04:00
Lisa Cawley de91d2aeea [DOCS] Collapse nested objects in CCR APIs (#54697) 2020-04-03 12:04:33 -07:00
Ross Wolf 022f829d84
EQL: Add wildcard function (#54020)
* EQL: Add wildcard function
* EQL: Cleanup Wildcard.getArguments
* EQL: Cleanup Wildcard and rearrange methods
* EQL: Wildcard newline lint
* EQL: Make StringUtils function final
* EQL: Make Wildcard.asLikes return ScalarFunction
* QL: Restore BinaryLogic.java
* EQL: Add Wildcard PR feedback
* EQL: Add Wildcard verification tests
* EQL: Switch wildcard to isFoldable test
* EQL: Change wildcard test to numeric field
* EQL: Remove Wildcard.get_arguments
2020-04-03 10:15:43 -06:00
Rory Hunter 83e900e248 Tweak formatting config for exception lists (#54710)
Change how we format exceptions to only wrap them as necessary. While
the config's overall philosophy is to put items one-per-line when
wrapping, in practice this is a little cumbersome for exception lists.
2020-04-03 16:54:31 +01:00
Ioannis Kakavas 8e255337f8
Fix SamlServiceProviderDocumentTests (#54718) (#54723)
Don't assume byte for byte equality because internal structures
 do not guarantee order
2020-04-03 18:46:36 +03:00
Dan Hermann 18fef3de2a
Get data stream accepts single search parameter 2020-04-03 10:36:26 -05:00
Christoph Büscher 8c9ac14a98
Rename field name constants in AbstractBuilderTestCase (#53234)
Some field name constants were not updaten when we moved from "string" to "text"
and "keyword" fields. Renaming them makes it easier and faster to know which
field type is used in test subclassing this base test case.
2020-04-03 17:28:22 +02:00
David Roberts 470aa9a5f1 [TEST] Mute CategorizationIT.testNumMatchesAndCategoryPreference (#54717)
The test results are affected by the off-by-one error that is
fixed by https://github.com/elastic/ml-cpp/pull/1122

This test can be unmuted once that fix is merged and has been
built into ml-cpp snapshots.
2020-04-03 14:40:47 +01:00
Dimitris Athanasiou e8c0351fd8
[7.x][ML] Allow force stopping failed and stopping DF analytics (#54650) (#54712)
Force stopping a failed job used to work but it
now puts the job in `stopping` state and hangs.
In addition, force stopping a `stopping` job is
not handled.

This commit addresses those issues with force
stopping data frame analytics. It inlines the
approach with that followed for anomaly detection
jobs.

Backport of #54650
2020-04-03 16:08:06 +03:00
Bogdan Pintea 7cef89e084 ODBC: Document the new VarcharLimit and EarlyExecution params (#54632)
* Document VarcharLimit and EarlyExecution params

Add the documentation for the newly added VarcharLimit and
EarlyExecution DSN attributes.

* Remove obsolete VersionChecking param

This param had been removed already along the #53082 work.

* Update docs/reference/sql/endpoints/odbc/configuration.asciidoc

fix typo

Co-Authored-By: Stuart Cam <stuart@codebrain.co.uk>

* Update docs/reference/sql/endpoints/odbc/configuration.asciidoc

fix typo

Co-Authored-By: Stuart Cam <stuart@codebrain.co.uk>
(cherry picked from commit f38761631a12b38f7f075635f7ac61dc96656cd7)
2020-04-03 14:46:39 +02:00