Commit Graph

50883 Commits

Author SHA1 Message Date
Nik Everett 609c61f75c
Formalize usage stats for analytics (backport of #52966) (#53077)
This moves the usage statistics gathering from the `AnalyticsPlugin`
into an `AnalyicsUsage`, removing the static state. It also checks the
license level when parsing all analytics aggregations. This is how we
were checking them before but we did it in an easy to forget way. This
way is slightly simpler, I think.
2020-03-04 10:29:11 -05:00
James Rodewig 801e50203e [DOCS] Add missing doc type to EQL search results 2020-03-04 10:26:11 -05:00
Martijn van Groningen 3fa5395ac8
Use correct issue number: #52453 2020-03-04 16:17:55 +01:00
Martijn van Groningen 2e325e24cb
Mute testMonitorClusterHealth test (#53109)
Relates to #36782
2020-03-04 16:08:19 +01:00
Yannick Welsch d1e7951e00 [DOCS] Add 7.6.1. release notes (#52874)
Adds the release notes for 7.6.1.
2020-03-04 15:47:54 +01:00
Martijn van Groningen b77f6746d1
unmute watcher single node test case
relates to #36782
2020-03-04 15:25:17 +01:00
James Rodewig e3d3c3400c [DOCS] Update EQL default event category and timestamp values (#53102)
Updates the documented default `event_category_field` and `timestamp_field`
values for the EQL search API. Also updates related guidance in the
EQL requirement docs.

Relates to #53073.
2020-03-04 09:17:37 -05:00
James Rodewig 0c4bf64095 [DOCS] Fix several Asciidoctor double arrow replacements (#52827)
Per the [Asciidoctor docs][0], Asciidoctor replaces the following
syntax with double arrows in the rendered HTML:

* => renders as ⇒
* <= renders as ⇐

This escapes several unintended replacements, such as in the Painless
docs.

Where appropriate, it also replaces some double arrow instances with
single arrows for consistency.

[0]: https://asciidoctor.org/docs/user-manual/#replacements
2020-03-04 08:43:19 -05:00
James Rodewig 2b59f8ac34 [DOCS] Correct `hits.total.relation` response parm def (#52847)
Fixes a partially completed definition for the `hits.total.relation`
response parameter in the search API docs.
2020-03-04 08:23:34 -05:00
Aleksandr Maus b47bffba24
EQL: consistent naming for event type vs event category (#53073) (#53090)
Related to https://github.com/elastic/elasticsearch/issues/52941
2020-03-04 08:02:38 -05:00
Marios Trivyzas e180e2738a
SQL: [Tests] Add tests for optimization of aliased expressions (#53048)
Add a unit test to verify that the optimization of expression
(e.g. COALESCE) is applied to all instances of the expression:
SELECT, WHERE, GROUP BY and HAVING.

Relates to #35270

(cherry picked from commit 2ceedc7f2019fad92cd86679af1a9c6fa594aa8d)
2020-03-04 11:48:06 +01:00
Marios Trivyzas 1d5c842700 SQL: Fix column size for IP data type (#53056)
Set size/displaySize to 45 which is the maximum string for
an IP (v6), since IPs are returned as strings.

Fixes: #52762

(cherry picked from commit 815f01747a4d54a274ca248af6fc08e5ea0728c1)
2020-03-04 10:36:44 +01:00
Mark Vieira 4b528d97ad
Consolidate duplication of BWC testing task setup in script plugin (#53079)
(cherry picked from commit 33fc8e7ebfac8d47a5f9f026b3836bb47bea141a)
2020-03-03 14:43:02 -08:00
Zachary Tong 3fcf598b92 Reduce deprecation log noise from DateIntervalWrapper (#52655)
Converts the deprecations to `deprecatedAndMaybeLog` to reduce the
number of times we log deprecations, since some of these could be called
at a high frequency (due to unconverted queries, aggs, etc)
2020-03-03 17:08:10 -05:00
Jay Modi c610e0893d
Introduce system index APIs for Kibana (#53035)
This commit introduces a module for Kibana that exposes REST APIs that
will be used by Kibana for access to its system indices. These APIs are wrapped
versions of the existing REST endpoints. A new setting is also introduced since
the Kibana system indices' names are allowed to be changed by a user in case
multiple instances of Kibana use the same instance of Elasticsearch.

Additionally, the ThreadContext has been extended to indicate that the use of
system indices may be allowed in a request. This will be built upon in the future
for the protection of system indices.

Backport of #52385
2020-03-03 14:11:36 -07:00
Nik Everett 7339427af5
Remove some deprecation warnings parsing aggs (backport of #53026) (#53072)
With #50871 aggrgations should now be parsed directly by an
`ObjectParser` or `ConstructingObjectParser` without the need for the
ceremonial `parse` method. This removes 10 of those `parse` methods and
parses the aggregation directly from their `ObjectParser`.
2020-03-03 15:27:49 -05:00
Lisa Cawley 892f0d5848 [DOCS] Adds link in datafeed indices_options (#53067) 2020-03-03 10:31:32 -08:00
Lisa Cawley bb4bcbc54c [DOCS] Adds ignore_throttled to index conventions (#53063) 2020-03-03 10:23:56 -08:00
James Rodewig cf87724ff6
[DOCS] Reformat `stop` token filter (#53059)
Makes the following changes to the `stop` token filter docs:

* Updates description
* Adds a link to the related Lucene filter
* Adds detailed analyze snippet
* Updates custom analyzer and custom filter snippets
* Adds a list of predefined stop words by language

Co-authored-by: ScottieL <36999642+ScottieL@users.noreply.github.com>
2020-03-03 13:22:52 -05:00
Andrei Stefan 9ad9ad7a6b
SQL: update SqlNodeSubclassTests list of min-two-parameters functions list (#53045) (#53058)
(cherry picked from commit c741e49d9f5e7b78c1a78e1af97eb19354fe6864)
2020-03-03 19:37:37 +02:00
William Brafford d3a8ac66c6
Handle special chars in JAVA_HOME in elasticsearch-service.bat (#52676) (#53057)
* Handle special chars in JAVA_HOME in elasticsearch-service.bat (#52676)

* Test case for windows service where JAVA_HOME path contains spaces (#53028)

Co-authored-by: Muhammad Shaheer Akram <41253927+shaheerakr@users.noreply.github.com>
2020-03-03 12:01:54 -05:00
István Zoltán Szabó 6cece3a709 [DOCS] Adds response body documentation to GET inference API (#53050) 2020-03-03 16:26:40 +01:00
Luca Cavanna 8a05b670ca
Address MinAndMax generics warnings (#52642)
`MinAndMax` encapsulates min and max values for a shard. It uses generics to make sure that the values are of the same type and are also comparable. Though there are warnings whenever this class is currently used, which are addressed with this commit.

Relates to #49092
2020-03-03 16:08:10 +01:00
Adrien Grand cb868d2f5e
Introduce a `constant_keyword` field. (#49713) (#53024)
This field is a specialization of the `keyword` field for the case when all
documents have the same value. It typically performs more efficiently than
keywords at query time by figuring out whether all or none of the documents
match at rewrite time, like `term` queries on `_index`.

The name is up for discussion. I liked including `keyword` in it, so that we
still have room for a `singleton_numeric` in the future. However I'm unsure
whether to call it `singleton`, `constant` or something else, any opinions?

For this field there is a choice between
 1. accepting values in `_source` when they are equal to the value configured
    in mappings, but rejecting mapping updates
 2. rejecting values in `_source` but then allowing updates to the value that
    is configured in the mapping
This commit implements option 1, so that it is possible to reindex from/to an
index that has the field mapped as a keyword with no changes to the source.

Backport of #49713
2020-03-03 16:01:47 +01:00
James Rodewig bcb68c860c [DOCS] Reorganize EQL requirements page 2020-03-03 07:02:30 -05:00
Yang Wang 70814daa86
Allow _rollup_search with read privilege (#52043) (#53047)
Currently _rollup_search requires manage privilege to access. It should really be
a read only operation. This PR changes the requirement to be read indices privilege.

Resolves: #50245
2020-03-03 22:29:54 +11:00
Alan Woodward 3759063d34 Allow specifying an exclusive set of fields on ObjectParser (#52893)
ObjectParser allows you to declare a set of required fields, such that at least one
of the set must appear in an xcontent object for it to be valid. This commit adds
the similar concept of a set of exclusive fields, such that at most one of the set
must be present. It also enables required fields on ConstructingObjectParser, and
re-implements PercolateQueryBuilder.fromXContent() to use object parsing as
an example of how this works.
2020-03-03 10:56:20 +00:00
Martijn van Groningen 510db25dd0
Simplify watcher indexing listener.(#53046)
Backport: #52627

Add watcher to trigger server after index operation has succeeded,
instead of adding a watch to trigger service before
the actual index operation has performed on the shard level.

This logic is simpler to reason about in the case that a failure
does occur during the execution of an index operation on
the shard level.

Relates to #52453, but I think doesn't fix it, but makes it easier
to debug.
2020-03-03 11:01:57 +01:00
Hendrik Muhs 844f350774 [Transform] restructure transform yaml tests (#52956)
restructure transform yaml tests to run cleanup in teardown phase

relates #52428
2020-03-03 10:31:22 +01:00
Hendrik Muhs d9258e210e [Transform] fix sporadic race condition in TransformUsageIT (#52946)
relax the test for trigger count

fixes #52931
2020-03-03 10:27:36 +01:00
Jason Tedor a154f9c657
Early return if no global checkpoint listeners (#53036)
When notifying global checkpoint listeners, we have an opportunity to
early return if there are not any registered listeners. This is
important since it saves some allocations, and also saves forking some
empty work to another thread. This commit adds an early return from
notifying listeners if there are not any registered.
2020-03-02 23:28:22 -05:00
James Rodewig 5cffa14f45 [DOCS] Fix typo in EQL docs 2020-03-02 16:09:03 -05:00
Costin Leau 712e0c05cd EQL: Add implicit ordering on timestamp (#53004)
QL: Move Sort base class from SQL to QL
(cherry picked from commit 798015b7bbd565e9c4222724614baeb432c7c2b3)
2020-03-02 22:41:36 +02:00
Mark Vieira f8396e8d15
Mute RunDataFrameAnalyticsIT.testStopOutlierDetectionWithEnoughDocumentsToScroll
Signed-off-by: Mark Vieira <portugee@gmail.com>
2020-03-02 09:21:55 -08:00
Mark Vieira 5b5e92c71d
Mute NodeSubclassTests.testReplaceChildren
Signed-off-by: Mark Vieira <portugee@gmail.com>
2020-03-02 09:21:54 -08:00
Lisa Cawley 5a293b29a1 [DOCS] Add missing anchors in cron expressions (#52960) 2020-03-02 09:07:22 -08:00
Orhan Toy ad2f630795 [DOCS] Fix formatting of simulate ingest pipeline API docs (#52754) 2020-03-02 11:46:27 -05:00
István Zoltán Szabó a267849def [DOCS] Adds transform-settings page and its subpage to redirects (#52944)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-03-02 16:36:40 +01:00
Lisa Cawley 4fbe1b0550
[DOCS] Adds cat anomaly detectors API (#52866) (#52970) 2020-03-02 07:28:55 -08:00
Hendrik Muhs a328a8eaf1
[7.x][Transform] implement node.transform to control where to… (#52998)
implement transform node attributes to disable transform on certain nodes and
test which nodes are allowed to do remote connections

closes #52200
closes #50033
closes #48734

backport #52712
2020-03-02 16:10:57 +01:00
James Rodewig db64029919
[7.x] [DOCS] Add parameter examples to EQL search tutorial (#52953)
Makes the following updates to the EQL search tutorial:

* Adds an API response to the basic tutorial
* Adds an example using the `event_type_field` parm
* Adds an example using the `timestamp_field`parm
* Adds an example using the `query` parm
* Updates example dataset to support more EQL query variety
2020-03-02 10:08:03 -05:00
Aleksandr Maus 89ed857c79
EQL: Change request parameter query to filter and rule to query (#52971) (#53006)
Related to https://github.com/elastic/elasticsearch/issues/52911
2020-03-02 09:26:23 -05:00
James Rodewig d336faa0b0 [DOCS] Reformat trim token filter docs (#51649)
Makes the following changes to the `trim` token filter docs:

* Updates description
* Adds a link to the related Lucene filter
* Adds tip about removing whitespace using tokenizers
* Adds detailed analyze snippets
* Adds custom analyzer snippet
2020-03-02 07:48:23 -05:00
James Rodewig f5bccad847 [DOCS] Correct guidance for `index_options` mapping parm (#52899)
Adds a warning admonition stating that the `index_options` mapping
parameter is intended only for `text` fields.

Removes an outdated statement regarding default values for numeric
and other datatypes.
2020-03-02 07:39:35 -05:00
rhymes 7eb4c07f1f [DOCS] Fix typo in index and search analysis docs (#52988) 2020-03-02 07:25:01 -05:00
Andrei Stefan 6fecc1db84
Issue a different error message in case an index doesn't have a mapping (#52967) (#53003)
(cherry picked from commit a0bd83a0579cf196a1d727de2a46b3b101d5a73b)
2020-03-02 14:04:49 +02:00
Andrei Stefan 69383acecf
Define list of Nodes that have minimum two children in tests (#52957) (#52994)
(cherry picked from commit c1e43e694f02edf3e197abbab7c21008c022b516)
2020-03-02 11:26:50 +02:00
Hendrik Muhs 49f41d127b [Transform] fix NPE in derive stats if shouldStopAtNextCheckpo… (#52940)
fixes a NPE in _stats in case shouldStopAtNextCheckpoint is set.
2020-03-02 08:11:01 +01:00
Martijn van Groningen d102158e6f
Improve closing mock webserver when failed to start (#52943)
Fix NPE when closing a webserver that hasn't started correctly.

This can happen when ssl context isn't initialized. The server instance is then never set,
which causes an NPE that masks the actual failure.

Example stacktrace that would mask an actual failure:

```
java.lang.NullPointerException
	at org.elasticsearch.test.http.MockWebServer.close(MockWebServer.java:271)
	at org.elasticsearch.xpack.watcher.test.integration.HttpSecretsIntegrationTests.cleanup(HttpSecretsIntegrationTests.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
```
2020-03-02 07:19:08 +01:00
Yang Wang 712191f2af
[DOCS] Fix typo and ensure asterisks render properly (#52991) (#52992)
Fix a typo and rendering issue for doc test example.
2020-03-02 11:56:42 +11:00