Commit Graph

54259 Commits

Author SHA1 Message Date
Armin Braun bd07e44c9a
Make Searchable Snapshot's CacheFile Lock less (#63911) (#64125)
Replacing the mechanism for eviction and listener references via a read-write lock by
a reference counting implementation.
This fixes a bug that caused test failure #63586 in which concurrently trying to acquire or release
an eviction listener while doing a file operation would sometimes lead to throwing an exception
since the `tryLock` call on the read lock would fail in this case.
Also this removes the possibility of blocking cluster state updates as a result of them waiting
on the write-lock which might take a long time if a slow read operation executes concurrently.

Closes #63586
2020-10-26 09:30:22 +01:00
Armin Braun bdea16301d
Fix testMasterFailoverDuringCloneStep1 (#63580) (#64127)
Assuming the clone failed when the request failed is not sufficient.
There are failure modes where the request fails but the clone still works out
because the data node resent the requeest after the first clone had already been
failed and removed from the cluster state when master was restarted.

Closes #63473
2020-10-26 09:30:09 +01:00
James Rodewig bec3eca234 [DOCS] Remove unneeded words in EQL docs 2020-10-24 20:28:23 -04:00
James Rodewig 8e726c3f14 [DOCS] Minor data stream docs fixes 2020-10-24 19:50:58 -04:00
James Rodewig 2cecbc8c83
[DOCS] Remove unneeded link 2020-10-24 17:06:30 -04:00
James Rodewig 361fa021fa
[DOCS] Tighten data streams copy (#64085) (#64111) 2020-10-24 14:39:13 -04:00
James Rodewig f19f170811
[DOCS] Tighten async EQL copy (#64106) (#64108) 2020-10-24 14:28:43 -04:00
James Rodewig d6143bb49d
[DOCS] Tighten EQL copy (#64081) (#64104) 2020-10-24 11:09:51 -04:00
debadair 0f5d98e40c
[DOCS] Add searchable snapshots topic. (#63040) (#64088)
* [DOCS] Add searchable snapshots topic.

* [DOCS] Add definitions & remove fully-remote storage.

* [DOCS] Fixed duplicate anchor.

* Expand conceptual docs for searchable snapshots

* Rewordings

* Glossary tidy-up

* Beta

* Reword

* More performance idea to a TIP

* use -> manage

* red -> not green

* Missing space?

* Update docs/reference/glossary.asciidoc

* Fix beta label

* Use more attributes, fix link titles

* Apply suggestions from code review

Co-authored-by: debadair <debadair@elastic.co>

* Reformat

* Minor rewordings

* More minor rewordings

* Address Henning's comments

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-10-22 15:00:15 -07:00
James Rodewig 6e61292096
[DOCS] Fix `ignore_unavailable` param in get index and get alias APIs (#64075) (#64079) 2020-10-22 16:21:14 -04:00
James Rodewig 044dfb4637
[DOCS] Fix cache setting name in 7.9 migration docs (#64063) (#64073)
The setting name is script.context.$CONTEXT.cache_max_size rather than
script.context.$CONTEXT.context_max_size.

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
2020-10-22 13:56:06 -04:00
Marios Trivyzas 9b8ea63cd2
[7.10] Bump version after 7.9.3 release (#63818) 2020-10-22 17:49:21 +02:00
Przemyslaw Gomulka bab426be2c
[7.10] add 6.8.14 version (#63824)
adding 6.8.14 after version 6.8.13 release
2020-10-22 16:51:01 +02:00
James Rodewig ce6b13e993
[DOCS] Remove 7.9.3 coming tag (#64057) (#64060) 2020-10-22 08:46:29 -04:00
David Roberts cb0c538b35 [ML] Fix rare ML daily maintenance test race condition (#64043)
Depending on thread scheduling the ML daily maintenance
tests could do one more iteration than expected, causing
rare failures.

Fixes #64036
2020-10-22 13:03:02 +01:00
James Rodewig fa8ad3abde
[DOCS] Add release notes for 7.9.3 (#63834) (#64021)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Marios Trivyzas <matriv@gmail.com>
2020-10-21 17:05:08 -04:00
Jay Modi 0fab675e95
Correct system index names in Kibana module (#64011)
This commit updates the list of system index names to be complete and
correct for Kibana and APM. The pattern `.kibana*` is too inclusive for
system indices and actually includes the
`.kibana-event-log-${version}-${int}` pattern for the Kibana event log,
which should only be hidden and not a system index. Additionally, the
`.apm-custom-link` index was not included in the list of system
indices. Finally, the reporting pattern has been updated to match that
of the permissions given to the kibana_system role.

Backport of #63950
2020-10-21 12:11:10 -06:00
Lisa Cawley 25be69ceaf [DOCS] Add tips for num_top_classes classification parameter (#63781) 2020-10-21 10:03:19 -07:00
Jay Modi f0e6684a95
Add APM configuration index to Kibana system indices (#64007)
* Add APM index to Kibana system indices, making it
accessible through the _kibana endpoint and giving it the
same access privileges as the other Kibana system indices.
* Parameterize kibana system index tests by index name

Backport of #63756

Co-authored-by: William Brafford <williamrandolphbrafford@gmail.com>
2020-10-21 10:55:23 -06:00
Rory Hunter bfd2cbed86
Remove deprecation indexing code from 7.10 (#63942)
The deprecation indexing code was writing to a regular data stream,
and it is not yet possible to hide a data stream or prefix it with
a period. This functionality we be re-added once it is possible to
mark a data stream as hidden, and also to not rely on the standard
logs template since that can be disabled.
2020-10-21 16:28:21 +01:00
James Rodewig 7551c4dc7f
[DOCS] Avoid trailing newline in apikey base64 encoding (#63720) (#64002)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Jurgen Braam <g.j.m.braam@gmail.com>
2020-10-21 09:44:48 -04:00
Armin Braun e0f73c96f7
Fix testStartCloneWithSuccessfulShardSnapshotPendingFinalization (#63966) (#64000)
We have to wait for no more operations here not for `1`. This mostly worked
because the test thread would add the listener quickly enough so that it sees the
state where either the snapshot or clone but not both have already finished
but randomly the test thread would be slow and time out on a state without snaphots in it.
2020-10-21 15:33:12 +02:00
Yang Wang c1071b1eb4
Update invalidate api key doc for the new ids field (#63860) (#63994)
Follow-up for #63224 and adds the missing doc update for the new ids field.
2020-10-22 00:19:41 +11:00
markharwood b933bd9f45
Search - make term/prefix/wildcard/regex query parsing more lenient (#63926)
Remove errors when case_insensitive flag set to false

Closes #63893
2020-10-21 13:33:19 +01:00
Henning Andersen ddd897f747 Fix test timeout for health on master failover (#63455)
testHealthOnMasterFailover could timeout on some of the health requests
in the case where an index is added, since the recovery leads to
extended test run time.

Closes #62690
2020-10-21 14:31:53 +02:00
Yang Wang 428fd7218c
Use asterisk instead of empty string to clear all cached entries (#63907) (#63989)
The officially supported way to clearing all entries from a cache is to use 
wildcard of either * or _all. Though empty string has the same effect, it was
never intended. Therefore the tests should not use empty string and this PR
changes them to use *.
2020-10-21 23:14:01 +11:00
Hendrik Muhs f2517678aa
[7.10][Transform] add support for unsigned_long data type (#63957)
add support for unsigned_long, which required a change in
writing out integer results properly, because coerce is not
supported for unsigned_long

fixes #63871
backport #63940
2020-10-20 21:05:46 +02:00
Nik Everett 8d30766a7d
Fix scripted metric BWC serialization (backport of #63821) (#63897)
We had and an error when serializing fully reduced scripted metrics.
Small typo and sever lack of tests..... Anyway, this fixed the one
character typo and adds a bunch more tests.
2020-10-20 13:15:26 -04:00
István Zoltán Szabó 6f977fc6f7
[7.10][DOCS] Adds ML related release highlights to What's new. (#63934)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-10-20 15:46:31 +02:00
Ignacio Vera d0f5066310
Upgrade to lucene-8.7.0-snapshot-72d8528c3a6 (#63912) (#63928) (#63933) 2020-10-20 15:08:06 +02:00
Adam Locke e064db27b1
Move clone snapshot API page. (#63902) (#63936) 2020-10-20 09:03:56 -04:00
Tanguy Leroux b2e07076a0
Add snapshot shard size based test in DiskThresholdDeciderTests (#63913)
This commit adds a test in DiskThresholdDeciderTests that verifies
 the allocation of a snapshot recovery source based shard in the 
situation where the snapshot shard size was successfully provided 
by the SnapshotInfoService introduced in #61906 and when the 
service failed to provide the size.

Relates #61906
2020-10-20 14:59:00 +02:00
Benjamin Trent eff7f06ca6
[ML] fix inference binary classification predication label and feature importance (#63688) (#63930)
When calculating feature importance, the leaf values directly correlate the value of the importance.

Consequently, positive leaf values -> positive feature importance

negative leaf values -> negative feature importance.

It follows that for binary classification, this is done such that the importance relates to the leaf values, which relate directly to the "probability of class 1".

So, the feature importance calculated is always for the importance as it relates to class 1.

The inverse is the importance as it relates to class 0.
2020-10-20 08:50:15 -04:00
Rene Groeschke b5448f07f3
Fix resolveAllDependencies by removing refs to non existing configs (#63409) (#63594) (#63715)
* Fix concurrent modification on task realization
* Use taskprovider instead of relying on tasks in distribution setup
* Port more task references in :distribution to task provider
* Fix nullpointer in distribution setup
2020-10-20 12:43:17 +02:00
Jim Ferenczi 3423f214dd Composite aggregation must check live docs when the index is sorted (#63864)
This change ensures that the live docs are checked in the composite aggregator
when the index is sorted.
2020-10-20 11:40:28 +02:00
Armin Braun 1880bcdc09
Add REST Test for Snapshot Clone API (#63863) (#63881)
Adds snapshot clone REST tests and HLRC support for the API.
2020-10-20 09:48:03 +02:00
Mayya Sharipova 1287df4074 Fix max/min aggs for unsigned_long (#63904)
Max and min aggs were producing wrong results for unsigned_long field
if field was indexed. If field is indexed for max/min aggs instead of
field data, we use values from indexed Points, values of which
are derived using method pointReaderIfPossible. Before
UnsignedLongFieldType#pointReaderIfPossible was incorrectly
producing values, as it failed to shift them back to original
values.

This patch fixes method pointReaderIfPossible to produce
correct original values.

Relates to #60050
2020-10-19 15:59:55 -04:00
James Rodewig 3369216087 [DOCS] Fix typo 2020-10-19 14:44:43 -04:00
Nik Everett 5583db5a73
Fix broken parent and child aggregator (backport #63811) (#63892)
In #57892 I broke *some* sub-aggregations inside of the `parent` and
`child` aggregator, specifically any sub-aggregations that do work in
the `postCollect` phase. This fixes it by delaying the post collect
phase of aggs under `parent` and `child` until `beforeBuildingBuckets`
because, well, we haven't done *any* collection until after that phase.
2020-10-19 13:05:22 -04:00
Adam Locke c28c3422bb
[DOCS] [7.10] Combining important config settings into a single page (#63849) (#63883)
* [DOCS] Combining important config settings into a single page (#63849)

* Combining important config settings into a single page.

* Updating ids for two pages causing link errors and implementing redirects.

* Updating links to use IDs instead of xrefs.
2020-10-19 12:59:44 -04:00
James Rodewig af9e96d681
[DOCS] EQL: Update `allow_no_indices` default (#63748) (#63891)
Co-authored-by: Adam Locke <adam.locke@elastic.co>
2020-10-19 12:31:22 -04:00
Rene Groeschke 34ccd3f0dc Explicitly use TLS 1.2 in Gradle (#63880) 2020-10-19 08:57:26 -07:00
Aref Razavi 2950818d44 Remove useless parentheses in bucket_key formula (#63868) 2020-10-19 11:53:33 +02:00
István Zoltán Szabó eef670f004
[DOCS] Clarifies destination index mappings for Transforms. (#63794) (#63804) 2020-10-19 10:03:08 +02:00
Julie Tibshirani f122b88bc5 Remove dependency from version plugin. 2020-10-18 14:09:32 -07:00
Julie Tibshirani ff312512c5 Mark the search 'fields' option as beta. (#63699)
We've identified two important enhancements that may affect the API. We expect
any API changes from these enhancements to be minor, but want to leave open the
possibility for small breaks. For example, we may end up returning unmapped
fields by default, or omitting nested fields from the root hit. The impact to
users should be quite small.

We're tracking the issues we need to resolve before removing the 'beta' label
here: #60985.
2020-10-18 14:09:27 -07:00
Lisa Cawley e6959d0d20
[7.10][DOCS] Add grant API key API and grant_api_key privilege (#63853) 2020-10-16 15:41:46 -07:00
Andrei Stefan 5f3c79d64b
Remove filter from QL's field_caps requests (#63840) (#63845)
(cherry picked from commit f009e6341d0fc0471f212d5a41c91e7aab77e006)
2020-10-17 01:36:26 +03:00
Jay Greenberg 3e5e54b884 [DOCS] Add monitoring note (#61793) 2020-10-16 13:36:28 -07:00
James Rodewig 07e4129328
[DOCS] Remove 7.9.2 coming tags (#62872) (#63830)
Removes coming tags from 7.9.2 Release Notes.

Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
2020-10-16 12:44:53 -04:00