Commit Graph

54363 Commits

Author SHA1 Message Date
Costin Leau 74fde15833 EQL: Allow null tiebreakers inside ordinals/sequences (#65033)
Align Ordinal comparator to consider nulls last (higher) in tiebreakers.
Add unit tests to Ordinal comparisons and criterion extraction.

Fix #64706

(cherry picked from commit 93dc883abd6b8855ff1618a574412b7f773b8ff5)
(cherry picked from commit 936e5f1a2cc29c1d5662cb8aa90c629af563a987)
2020-11-16 16:52:55 +02:00
James Rodewig 20bda8d23a
[DOCS] Document reloadable Watcher settings (#64998) (#65073) 2020-11-16 09:18:48 -05:00
James Rodewig e4054e4e92
[DOCS] Clarify diff between shards per node settings (#64875) (#65071)
Clarifies differences between the
`cluster.routing.allocation.total_shards_per_node` and
`cluster.max_shards_per_node` cluster settings.

Closes #51839

Co-authored-by: Gordon Brown <arcsech@gmail.com>
2020-11-16 09:18:11 -05:00
Przemysław Witek de668ab84b
[7.10] [ML] Extract dependent variable's mapping correctly in case of a multi-field (#63813) (#64287) 2020-11-16 10:34:58 +01:00
Costin Leau 9551cb3420 EQL: small improvements to the testing base class
Extract request settings into dedicated methods for easier adjustments

(cherry picked from commit 4f93591cc561c7f8ff7c2f070dd1180f209810b7)
(cherry picked from commit ff7e8427345c304f5a37612c870b48555484b692)
2020-11-14 16:40:48 +02:00
Costin Leau f7cc570c4f EQL: Re-enable correctness tests (#65041)
Enable previously disabled tests - only two type of queries remain
disabled: one that does pattern matching and another one for
case-insensitivity.

Fix #63742

(cherry picked from commit 20210cc43b34438c40b8b5aebf0aa2b8161c4104)
(cherry picked from commit 95d08f2c8d0aac52cc1ed470fa489c239ee25159)
2020-11-14 16:09:11 +02:00
Costin Leau 76e73fec79
EQL: Add option for returning results from the tail of the stream (#64869) (#65040)
Introduce option for specifying whether the results are returned from
the tail (end) of the stream or the head (beginning).
Improve sequencing algorithm by significantly eliminating the number
of in-flight sequences for spare datasets.
Refactor the sequence class by eliminating some of the redundant code.
Change matching behavior for tail sequences.
Return results based on their first entry ordinal instead of
insertion order (which was ordered on the last match ordinal).
Randomize results position inside test suite.

Close #58646

(cherry picked from commit e85d9d1bbee13ad408e789fd62efb30bc8d223f2)
(cherry picked from commit 452c674a10cdc16dced3cde7babf5d5a9d64a6d9)
2020-11-14 13:44:17 +02:00
Adam Locke 38b5b59862
Explicitly defining types for sources parameter (#65006) (#65022) 2020-11-12 17:08:42 -05:00
Alan Woodward 0e2a9b4ac7 Fix sparse vector test 2020-11-12 20:10:07 +00:00
Alan Woodward caf143f4a5
Unused boost parameter should not throw mapping exception (#64999) (#65014)
We were correctly dealing with boosts that had an effect, but mappers
that had a silently accepted but ignored boost parameter were throwing
an error instead of continuing to ignore the boost but emitting a
warning.

Fixes #64982
2020-11-12 19:28:32 +00:00
István Zoltán Szabó d9970fa764
[DOCS] Adds scroll_size maximum value to datafeeds API docs (#64986) (#65003) 2020-11-12 18:10:24 +01:00
Benjamin Trent b888f36388
[ML] fix custom feature processor extraction bugs around boolean fields and custom one_hot feature output order (#64937) (#65009)
This commit fixes two problems:

- When extracting a doc value, we allow boolean scalars to be used as input
- The output order of processed feature names is deterministic. Previous custom one hot fields used to be non-deterministic and thus could cause weird bugs.
2020-11-12 11:15:57 -05:00
Tanguy Leroux e40d7e02ea
Makes testCcrRepositoryFetchesSnapshotShardSizeFromIndexShardStoreStats more robust (#64976) (#64989)
Today this test fails because the sizes of the snapshot 
shards are only kept in a very short period of time in 
the InternalSnapshotsInfoService and are not 
guaranteed to exist once the shards are correctly 
assigned.

closes #64167
2020-11-12 15:38:38 +01:00
James Rodewig 5c5fd50f8d
[DOCS] Fix indentation (#64941) (#64964)
Co-authored-by: kemalizing <36174398+kemalizing@users.noreply.github.com>
2020-11-12 08:47:46 -05:00
Dimitris Athanasiou b5efaf6e3b
[7.10][ML] Protect against stack overflow while loading DFA data (#64947) (#64956)
If we encounter an exception during extracting data in a data
frame analytics job, we retry once. However, we were not catching
exceptions thrown from processing the search response. This may
result in an infinite loop that causes a stack overflow.

This commit fixes this problem.

Backport of #64947
2020-11-12 11:08:40 +02:00
debadair e4b77bcd38
[DOCS] Make system & hidden index info more prominent. (#64839) (#64960)
* [DOCS] Make system & hidden index info more prominent.

* Update docs/reference/api-conventions.asciidoc
2020-11-11 15:16:45 -08:00
Andras Palinkas a7301065d7
SQL: Fix the return type in the sign function (#64845) (#64968)
Fixes the inconsistency between the type of the object returned by the
`SIGN()/SIGNUM()` SQL functions and the specified `DataType`.

In the Class Sign, DataType is DataTypes.INTEGER. The source code is as
follows:

```
    public DataType dataType() {
        return DataTypes.INTEGER;
    }
```

But In the Class MathProcessor, the source code of SIGN((Object l),
Parameter and return value types are the same. Therefore, when using
double or float parameters to test, there is a little problem, the test
method is like the following curl :

```
curl -XPOST 127.0.0.1:9200/_sql -d "{\"query\":\"select  SIGN(1.0) \"}" \
  -H 'Content-Type: application/json'
```

The result is:

```
{"columns":[{"name":"SIGN(1.0)","type":"integer"}],"rows":[[1.0]]}
```

The result value is `1.0`, but the type is `integer`.

Signed-off-by: mantuliu <240951888@qq.com>
Co-authored-by: Marios Trivyzas <matriv@gmail.com>
(cherry picked from commits aa78301e71f, ced3c1281c7, 40e5b9b)
2020-11-11 17:07:17 -05:00
James Rodewig 4b1d66fc29
[DOCS] Add checksum links for plugin downloads (#64949) 2020-11-11 13:12:35 -05:00
debadair 45ec4d7f5b
[DOCS] Backport glossary additions (#64891) (#64893)
* [DOCS] Add system & hidden indices definitions. (#64834)

* [DOCS] Add system & hidden indices definitions.

* Update docs/reference/glossary.asciidoc

* Update docs/reference/glossary.asciidoc

Co-authored-by: Lisa Cawley <lcawley@elastic.co>

* Update docs/reference/glossary.asciidoc

* Update docs/reference/glossary.asciidoc

Co-authored-by: Lisa Cawley <lcawley@elastic.co>

* [DOCS] Add data tier defs (#64838)

* [DOCS] Add data tier defs

* Add hot tier def

Co-authored-by: Lisa Cawley <lcawley@elastic.co>

Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-11-11 10:06:47 -08:00
István Zoltán Szabó 5893d51d42
[DOCS] Fixes example aggregation syntax in datafeed aggregations. (#64936) (#64943) 2020-11-11 17:17:00 +01:00
James Rodewig 75b4af5833
[DOCS] Fix plugins service comment typo (#64902) (#64933)
Co-authored-by: Howard <danielhuang@tencent.com>
2020-11-11 10:30:44 -05:00
Daniel Mitterdorfer b8c9780c23
Mute multiple tests in ClassificationIT (#64930)
Relates #64926
2020-11-11 15:30:19 +01:00
James Rodewig 8bdd9f6ddc
[DOCS] Remove 7.10.0 coming tag (#64813) (#64929) 2020-11-11 09:28:54 -05:00
Daniel Mitterdorfer a6302d2169
Mute RolloverIT#testRolloverWithClosedIndexInAlias (#64925)
Relates #64921
2020-11-11 14:33:48 +01:00
Andrei Dan cd35122e48
Bump versions after 7.10 release (#64856) 2020-11-11 13:08:16 +00:00
Daniel Mitterdorfer 90af072277
Mute AsyncSearchActionIT.testRetryVersionConflict (#64919)
Relates #63948
2020-11-11 14:01:33 +01:00
Daniel Mitterdorfer 723e14ab72
Mute field collapsing tests in MixedClusterClientYamlTestSuiteIT (#64914)
Relates #52416
2020-11-11 11:48:18 +01:00
James Rodewig 9c5d28bac8
[DOCS] Add heading and anchor to reindex with ingest pipeline section (#64835) (#64896)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Mark Walkom <markw@elastic.co>
2020-11-10 17:25:07 -05:00
Tim Brooks f96dccd1ec
Propogate rejected execution during bulk actions (#64886)
Currently a rejected execution exception can be swallowed when async
actions return during transport bulk actions. This includes scenarios
where we went async to perform ingest pipelines or index creation. This
commit resolves the issue by propagating a rejected exception.
2020-11-10 12:16:40 -07:00
Nhat Nguyen aa0e3f85e6 Increase timeout in testCleanUpShardFollowTasksForDeletedIndices (#64562)
If the deleted index has N shards, then ShardFollowTaskCleaner can send 
N*(N-1)/2 requests to remove N shard-follow tasks. I think that's fine
as the implementation is straightforward. The test failed when the
deleted index has 8 shards. This commit increases the timeout in the
test.

Closes #64311
2020-11-10 11:51:41 -05:00
Nhat Nguyen 207e4b00f9 Busily assert in testCreateSearchContextFailure (#64243)
If a background refresh is running, then the refCount assertion will 
fail as Engine#refreshIsNeeded can increase the refCount by 2.

Closes #64052
2020-11-10 11:51:41 -05:00
Andrei Dan 9546d0d532
Update 7.10 release notes after respin (#64725)
Update 7.10 release notes after respin
Add upgrade section to release notes
2020-11-10 14:02:40 +00:00
Benjamin Trent f0ff673f82
[ML] Fix bug with data frame analytics classification test data sampling when using custom feature processors (#64727) (#64864)
When using custom processors, the field names extracted from the documents are not the
same as the feature names used for training.

Consequently, it is possible for the stratified sampler to have an incorrect view of the feature rows.
This can lead to the wrong column being read for the class label, and thus throw errors on training
row extraction.

This commit changes the training row feature names used by the stratified sampler so that it matches
the names (and their order) that are sent to the analytics process.
2020-11-10 08:47:07 -05:00
Benjamin Trent dafafd7ec6
[ML] fix edge case for data frame analytics where a field mapped as a keyword actually has boolean and string values in the _source (#64826) (#64862)
It is possible that a value mapped as a `keyword` has any scalar value type. This includes any numerical value, String, or boolean.

This commit allows `boolean` types to be considered as a part of the categorical feature collection when this is the case.
2020-11-10 08:46:52 -05:00
Tanguy Leroux 09ff421d4f
Fix TaskIT (#64724) (#64848)
Closes #64056
2020-11-10 11:06:25 +01:00
James Rodewig 21a6a11550
[DOCS] Document get pipeline API as multi-target (#64816) (#64828) 2020-11-09 16:28:23 -05:00
James Rodewig f9f4543f7c
[DOCS] Fix put repository API docs (#64811) (#64819) 2020-11-09 14:42:40 -05:00
Leaf-Lin 2f4016204e remove node.ingest setting in the documentation (#64456)
I'm not sure if this setting was left here deliberately? or by accident?
With all other node role definition has changed syntax from `node.xxx` to `node.roles: [ ]`, the ingest one is the only one left behind.
2020-11-09 12:19:43 -07:00
Adam Locke 8080f1ce87
Remove typo (#64760) (#64810)
* Consistency in writing style

Removing spaces before and after brackets for consistency.

* Remove typo

Remove one of two consecutive "the"s

Co-authored-by: Johannes Mahne <johannes.mahne@elastic.co>
2020-11-09 12:34:48 -05:00
James Rodewig 88634bc022
[DOCS] Improve docs for Windows DOS/UNC paths in `path.*` settings (#64668) (#64805) 2020-11-09 11:15:00 -05:00
James Rodewig 93987c8297
[DOCS] Fix typo (#64675) (#64802)
Co-authored-by: Ashish Jayan <58534490+chasexd@users.noreply.github.com>
2020-11-09 11:13:10 -05:00
James Rodewig 8a91dac9f0
[DOCS] Format the data tier allocation doc (#64722) (#64798)
Co-authored-by: bellengao <gbl_long@163.com>
2020-11-09 11:12:35 -05:00
Armin Braun d173ba6b2d
Fix NPE in toString of FailedShard (#64770) (#64779)
The concatenation took precedence over the null check, leading to an NPE
because `null` was passed to `ExceptionsHelper.stackTrace(failure))`.
2020-11-09 17:02:11 +01:00
James Rodewig 7360a58b49
[DOCS] Remove unneeded period (#64687) (#64792)
Co-authored-by: Johannes Mahne <johannes.mahne@elastic.co>
2020-11-09 10:30:51 -05:00
James Rodewig 9bffb5d4f1
[DOCS] Remove unneeded spaces (#64759) (#64783)
Co-authored-by: Johannes Mahne <johannes.mahne@elastic.co>
2020-11-09 10:29:26 -05:00
Ryan Ernst 6be4acd509
Use no-kpi subdomains for tests downloading artifacts (#64502)
This commit converts build code that downloads distributions or other
artifacts to use the new no-kpi subdomain, and removes the formerly used
no-kpi header.
2020-11-07 11:30:52 -08:00
Lisa Cawley e3f52d7f1b
[DOCS] Add custom feature processor example (#64681) (#64737) 2020-11-06 11:11:38 -08:00
Lee Hinman 6dbfafcff2
[7.10] Fix SetSingleNodeAllocateStep for data tier deployments (#64679) (#64730)
Backports the following commits to 7.10:

    Fix SetSingleNodeAllocateStep for data tier deployments (#64679)
2020-11-06 10:12:16 -07:00
Armin Braun 6de9198017
Increase Timeout for Waiting on Tasks in REST Tests (#64707) (#64718)
10s is pretty tight here considering that some x-pack related tests
might have a running create-index tasks in the backgroud after a test
that require multiple CS updates and can randomly take a few seconds on
slow IO etc.

closes #64580
2020-11-06 17:39:24 +01:00
James Rodewig f00484afa8
[DOCS] Document upgrade process for concurrency control (#64550) (#64711) 2020-11-06 09:45:49 -05:00