7111 Commits

Author SHA1 Message Date
Hendrik Muhs
e177a38504
[7.x][Transform] add throttling (#56007) (#56184)
add throttling to transform, throttling will slow down search requests by
delaying the execution based on a documents per second metric.

fixes #54862
2020-05-05 13:09:02 +02:00
Marios Trivyzas
363e994171
SQL: Fix DATETIME_PARSE behaviour regarding timezones (#56158) (#56182)
Previously, when the timezone was missing from the datetime string
and the pattern, UTC was used, instead of the session defined timezone.
Moreover, if a timezone was included in the datetime string and the
pattern then this timezone was used. To have a consistent behaviour
the resulting datetime will always be converted to the session defined
timezone, e.g.:
```
SELECT DATETIME_PARSE('2020-05-04 10:20:30.123 +02:00', 'HH:mm:ss dd/MM/uuuu VV') AS datetime;
```
with `time_zone` set to `-03:00` will result in
```
2020-05-04T05:20:40.123-03:00
```

Follows: #54960
(cherry picked from commit 8810ed03a209cc8fe1bad309a81e85b56a39da27)
2020-05-05 12:08:39 +02:00
Théophile Helleboid - chtitux
8a23da429a Docs fix node_id spec for secure settings reload API (#55712)
Fix docs typo for the `node_id` parameter in the secure settings reload API.
2020-05-05 11:21:02 +03:00
István Zoltán Szabó
9bcc975bd1 [DOCS] Simplifies footnote text in DFA APIs (#56105)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-05-05 09:05:08 +02:00
Lee Hinman
8fa14b333d
[7.x] Validate non-negative priorities for V2 index templates (#56139) (#56163)
Backports the following commits to 7.x:
 - Validate non-negative priorities for V2 index templates (#56139)
2020-05-04 16:19:13 -06:00
James Rodewig
922a80c3f4
[DOCS] Add collapsible sections to search API response (#55887) 2020-05-04 16:57:10 -04:00
Dan Hermann
9892813842
[7.x] Delay warning about missing x-pack (#56142)
* Delay warning about missing x-pack (#54265)

Currently, when monitoring is enabled in a freshly-installed cluster,
the non-master nodes log a warning message indicating that master may
not have x-pack installed. The message is often printed even when the
master does have x-pack installed but takes some time to setup the local
exporter for monitoring. This commit adds the local exporter setting
`wait_master.timeout` which defaults to 30 seconds. The setting
configures the time that the non-master nodes should wait for master to
setup monitoring. After the time elapses, they log a message to the user
about possible missing x-pack installation on master.

The logging of this warning was moved from `resolveBulk()` to
`openBulk()` since `resolveBulk()` is called only on cluster updates and
the message might not be logged until a new cluster update occurs.

Closes #40898
2020-05-04 14:16:18 -05:00
Lisa Cawley
b816ab0c18
[DOCS] Synchs and links hyperparameter descriptions (#56131) 2020-05-04 10:37:26 -07:00
Julie Tibshirani
6b5cf1b031 For constant_keyword, make sure exists query handles missing values. (#55757)
It's possible for a constant_keyword to have a 'null' value before any documents
are seen that contain a value for the field. In this case, no documents have a
value for the field, and 'exists' queries should return no documents.
2020-05-04 09:41:52 -07:00
James Rodewig
4faf5a7916
[DOCS] Reformat porter_stem token filter (#56053)
Makes the following changes to the `porter_stem` token filter docs:

* Rewrites description and adds a Lucene link
* Adds detailed analyze example
* Adds an analyzer example
2020-05-04 10:39:17 -04:00
bellengao
722de7dd98 [Docs] Fix typo in match-bool-prefix-query doc (#56077) 2020-05-04 14:19:23 +02:00
bellengao
40f99119ae [Docs] Fix typo in getting-started-slm doc (#56075) 2020-05-04 14:18:00 +02:00
markharwood
e197b6c45b
Analysis enhancement - add preserve_original setting in ngram-token-filter (#55432) (#56100)
Authored-by: Amit Khandelwal <amitmbm87@gmail.com>
2020-05-04 11:31:28 +01:00
Christos Soulios
c65f828cb7
[7.x] Histogram field type support for ValueCount and Avg aggregations (#56099)
Backports #55933 to 7.x

Implements value_count and avg aggregations over Histogram fields as discussed in #53285

- value_count returns the sum of all counts array of the histograms
- avg computes a weighted average of the values array of the histogram by multiplying each value with its associated element in the counts array
2020-05-04 13:23:02 +03:00
Armin Braun
3a64ecb6bf
Allow Deleting Multiple Snapshots at Once (#55474) (#56083)
* Allow Deleting Multiple Snapshots at Once (#55474)

Adds deleting multiple snapshots in one go without significantly changing the mechanics of snapshot deletes otherwise.
This change does not yet allow mixing snapshot delete and abort. Abort is still only allowed for a single snapshot delete by exact name.
2020-05-03 20:30:58 +02:00
William Brafford
d53c941c41
Make xpack.monitoring.enabled setting a no-op (#55617) (#56061)
* Make xpack.monitoring.enabled setting a no-op

This commit turns xpack.monitoring.enabled into a no-op. Mostly, this involved
removing the setting from the setup for integration tests. Monitoring may
introduce some complexity for test setup and teardown, so we should keep an eye
out for turbulence and failures

* Docs for making deprecated setting a no-op
2020-05-01 16:42:11 -04:00
David Turner
69f50fe79f Improve same-shard allocation explanations (#56010)
I see occasional confusion about the explanations emitted by the same-shard
allocation decider, particularly amongst new users setting up a single-node
cluster and trying to determine why their cluster has `yellow` health. For
example:

    the shard cannot be allocated to the same node on which a copy of the shard
    already exists

This is technically correct but it's quite a complicated sentence. Also, by
starting with "the shard cannot be allocated" it makes it sound like this is
the problem, whereas in fact this message is a good thing and users should
typically focus their attention elsewhere.

This commit simplifies the wording of these messages and makes them sound more
positive, for example:

    a copy of this shard is already allocated to this node
2020-05-01 10:07:14 +01:00
Vanessa Bell
9b005f70ef [Docs] Grammar improvements in disk-usage.asciidoc (#55620) 2020-04-30 19:31:56 +02:00
Karel Minarik
81dcc9aef7 [DOCS] Add link to the Go client bulk helper (#55872) 2020-04-30 13:24:00 -04:00
James Rodewig
61cf646f17
[DOCS] EQL: Add advantages to overview (#53452) (#56052)
Adds a concise list of EQL advantages, based on the "EQL Advantages"
section in the [EQL for the masses][0] blog post.

The intent is to inform users how EQL could benefit at a high level.

[0]: https://www.elastic.co/blog/eql-for-the-masses

Co-Authored-By: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-04-30 13:19:31 -04:00
Igor Motov
d8f9df771d
Expose agg usage in Feature Usage API (#55732) (#56048)
Counts usage of the aggs and exposes them on the _nodes/usage/.

Closes #53746
2020-04-30 12:53:36 -04:00
James Rodewig
e4e02e133e
[DOCS] Remove approximate document counts example from term agg docs (#55442)
Removes an example from the "Document counts are approximate" section of the
terms agg documentation.

As #52377 details, the example was no longer accurate in 7.x or 6.8. Document
counts were more precise than the example presented.

We've opened issue #56025 to discuss re-adding an example later.

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

Co-authored-by: AB Prashanth <panuradh@buffalo.edu>
2020-04-30 10:11:50 -04:00
William Brafford
273ff6a105
Make xpack.ilm.enabled setting a no-op (#55592) (#55980)
* Make xpack.ilm.enabled setting a no-op

* Add watcher setting to not use ILM

* Update documentation for no-op setting

* Remove NO_ILM ml index templates

* Remove unneeded setting from test setup

* Inline variable definitions for ML templates

* Use identical parameter names in templates

* New ILM/watcher setting falls back to old setting

* Add fallback unit test for watcher/ilm setting
2020-04-30 09:50:18 -04:00
Lisa Cawley
006e00ed0a
[DOCS] Adds documentation for secondary authorization headers (#55365) (#55986) 2020-04-29 16:29:38 -07:00
James Rodewig
65b47d20a6 [DOCS] Update attribute for multi arg footnotes (#55860) 2020-04-29 10:25:36 -04:00
James Rodewig
1808a1f36b [DOCS] EQL: Correct cidrMatch function heading (#55935) 2020-04-29 10:02:06 -04:00
István Zoltán Szabó
337dc45f5b [DOCS] Adds missing space and a relevant link to the slm execute API page (#55917)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-04-29 15:50:06 +02:00
James Rodewig
bbf68de446 [DOCS] Correct Lucene link in kstem token filter docs 2020-04-29 09:30:37 -04:00
Luca Cavanna
8b05027bf0 [DOCS] Clarify async search response flags (#55574)
Relates to #55572
2020-04-29 15:22:05 +02:00
James Rodewig
767836c367
[DOCS] Reformat kstem token filter (#55823)
Makes the following changes to the `kstem` token filter docs:

* Rewrite description and adds a Lucene work
* Adds detailed analyze example
* Adds an analyzer example
2020-04-29 08:52:55 -04:00
Christos Soulios
02bf0c586a
[7.x] Histogram field type support for Sum aggregation (#55916)
Implements Sum aggregation over Histogram fields by summing the value of each bucket multiplied by their count as requested in #53285

Backports #55681 to 7.x
2020-04-29 15:06:12 +03:00
Henning Andersen
f679880b80 [DOCS] Create index name required (#55886)
The name of the new index to create is required.

Relates #45749
2020-04-29 13:35:49 +02:00
István Zoltán Szabó
e982cf4381 [DOCS] Makes the footnotes less verbose in configuring aggs page. (#55857) 2020-04-29 09:52:29 +02:00
debadair
8a662c7e62
ILM update backports (#55902)
* [DOCS] Rework conceptual info for ILM. (#52181)

* [DOCS] Rework conceptual info for ILM.

* Split the actions out of concepts.

* Added xpack role to actions.

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Apply suggestions from code review

* Edit actions for consistency and add action template. (#55632)

* Edit actions for consistency and add action template.

* Update docs/reference/ilm/actions/ilm-readonly.asciidoc

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Apply suggestions from code review
2020-04-28 16:38:01 -07:00
Lee Hinman
4315a55a1c
[7.x] Initial documentation for index templates V2 (#55755) (#55898)
Backports the following commits to 7.x:
 - Initial documentation for index templates V2 (#55755)
2020-04-28 16:10:50 -06:00
Henning Andersen
cab7bcc156
Disk decider respect watermarks for single data node (#55805) (#55847)
The disk decider had special handling for the single data node case,
allowing any allocation (skipping watermark checks) for such clusters.
This special handling can now be avoided via a setting.
2020-04-28 18:46:22 +02:00
Lee Hinman
777caf0725
[7.x] Add support for V2 index templates to /_cat/templates (#55829) (#55866)
Backports the following commits to 7.x:
 - Add support for V2 index templates to /_cat/templates (#55829)
2020-04-28 10:14:19 -06:00
Larry Gregory
47d252424b
Backport: Deprecate the kibana reserved user (#54967) (#55822) 2020-04-28 10:30:25 -04:00
James Rodewig
ddc7305ac9 [DOCS] Correct search API's timeout parm default (#55855) 2020-04-28 09:44:50 -04:00
James Rodewig
386fb16409 [DOCS] SQL: Update link for supported regex in RLIKE docs (#55830)
The`RLIKE` function docs points users to [Java’s Pattern class doc][0]
for regular expression syntax. However, these docs include shorthand
character classes, such as `[\d]`, `[\s]`, and `[\w]`. These character
classes are not supported in Elasticsearch, which may confuse users.

This updates the SQL `RLIKE` docs to refer to the ES [regular expression
syntax docs][1], which only documents supported syntax.

[0]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/regex/Pattern.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/regexp-syntax.html

Relates to #55231
2020-04-28 09:25:51 -04:00
James Rodewig
452be22a4d [DOCS] Warn about searching across all fields wt. query_string (#55853)
Warn about potential performance impact when a large number of fields
is used with query string query and no default field.

Re-adds content from #35570.
That content was erroneously removed in #45296.

Co-authored-by: Peter Dyson <peter.dyson@geekpete.com>
2020-04-28 09:20:21 -04:00
Adrien Grand
58c3bb5ae1
Repurpose ignore_throttled to be only about frozen indices. (#55047) (#55852)
This has no practical impact on users since frozen indices are the only
throttled indices today. However this has an impact on upcoming features
that would use search throttling.

Filtering out throttled indices made sense a couple years ago, but as
we're now improving support for slow requests with `_async_search` and
exploring ways to reduce storage costs, this feature has most likely
become a trap, that we'd like to not have with upcoming features that
would use search throttling.

Relates #54058
2020-04-28 14:31:54 +02:00
Amit Khandelwal
126e4acca8 Expose preserve_original in edge_ngram token filter (#55766)
The Lucene `preserve_original` setting is currently not supported in the `edge_ngram`
token filter. This change adds it with a default value of `false`.

Closes #55767
2020-04-28 10:24:27 +02:00
István Zoltán Szabó
a5cf4712e5 [DOCS] Changes feature importance links to point to the new page (#55531)
* [DOCS] Changes feature importance links to point to the new page.

* [DOCS] Fixes line breaks.
2020-04-28 09:03:43 +02:00
James Rodewig
c16b1edae0 [DOCS] EQL: Fix whitespace in stringContains docs 2020-04-27 15:53:59 -04:00
James Rodewig
8df5cff9c1 [DOCS] Correct stemmer token filters anchor 2020-04-27 14:57:59 -04:00
James Rodewig
5b8a18c756 [DOCS] Correct stemmer token filter anchor 2020-04-27 14:51:51 -04:00
David Roberts
3ba44a5af8
[ML] Adding failed_category_count to model_size_stats (#55761)
The failed_category_count statistic records the number of times
categorization wanted to create a new category but couldn't
because the job had reached its model_memory_limit.

Backport of #55716
2020-04-25 10:36:49 +01:00
James Rodewig
c1b0548db0
[DOCS] Document EQL search REST API (#52384) 2020-04-24 15:36:01 -04:00
James Rodewig
5981412bf7
[DOCS] EQL: Document stringContains function (#54968) 2020-04-24 15:09:05 -04:00