50622 Commits

Author SHA1 Message Date
Marios Trivyzas
1d5c842700 SQL: Fix column size for IP data type (#53056)
Set size/displaySize to 45 which is the maximum string for
an IP (v6), since IPs are returned as strings.

Fixes: #52762

(cherry picked from commit 815f01747a4d54a274ca248af6fc08e5ea0728c1)
2020-03-04 10:36:44 +01:00
Mark Vieira
4b528d97ad
Consolidate duplication of BWC testing task setup in script plugin (#53079)
(cherry picked from commit 33fc8e7ebfac8d47a5f9f026b3836bb47bea141a)
2020-03-03 14:43:02 -08:00
Zachary Tong
3fcf598b92 Reduce deprecation log noise from DateIntervalWrapper (#52655)
Converts the deprecations to `deprecatedAndMaybeLog` to reduce the
number of times we log deprecations, since some of these could be called
at a high frequency (due to unconverted queries, aggs, etc)
2020-03-03 17:08:10 -05:00
Jay Modi
c610e0893d
Introduce system index APIs for Kibana (#53035)
This commit introduces a module for Kibana that exposes REST APIs that
will be used by Kibana for access to its system indices. These APIs are wrapped
versions of the existing REST endpoints. A new setting is also introduced since
the Kibana system indices' names are allowed to be changed by a user in case
multiple instances of Kibana use the same instance of Elasticsearch.

Additionally, the ThreadContext has been extended to indicate that the use of
system indices may be allowed in a request. This will be built upon in the future
for the protection of system indices.

Backport of #52385
2020-03-03 14:11:36 -07:00
Nik Everett
7339427af5
Remove some deprecation warnings parsing aggs (backport of #53026) (#53072)
With #50871 aggrgations should now be parsed directly by an
`ObjectParser` or `ConstructingObjectParser` without the need for the
ceremonial `parse` method. This removes 10 of those `parse` methods and
parses the aggregation directly from their `ObjectParser`.
2020-03-03 15:27:49 -05:00
Lisa Cawley
892f0d5848 [DOCS] Adds link in datafeed indices_options (#53067) 2020-03-03 10:31:32 -08:00
Lisa Cawley
bb4bcbc54c [DOCS] Adds ignore_throttled to index conventions (#53063) 2020-03-03 10:23:56 -08:00
James Rodewig
cf87724ff6
[DOCS] Reformat stop token filter (#53059)
Makes the following changes to the `stop` token filter docs:

* Updates description
* Adds a link to the related Lucene filter
* Adds detailed analyze snippet
* Updates custom analyzer and custom filter snippets
* Adds a list of predefined stop words by language

Co-authored-by: ScottieL <36999642+ScottieL@users.noreply.github.com>
2020-03-03 13:22:52 -05:00
Andrei Stefan
9ad9ad7a6b
SQL: update SqlNodeSubclassTests list of min-two-parameters functions list (#53045) (#53058)
(cherry picked from commit c741e49d9f5e7b78c1a78e1af97eb19354fe6864)
2020-03-03 19:37:37 +02:00
William Brafford
d3a8ac66c6
Handle special chars in JAVA_HOME in elasticsearch-service.bat (#52676) (#53057)
* Handle special chars in JAVA_HOME in elasticsearch-service.bat (#52676)

* Test case for windows service where JAVA_HOME path contains spaces (#53028)

Co-authored-by: Muhammad Shaheer Akram <41253927+shaheerakr@users.noreply.github.com>
2020-03-03 12:01:54 -05:00
István Zoltán Szabó
6cece3a709 [DOCS] Adds response body documentation to GET inference API (#53050) 2020-03-03 16:26:40 +01:00
Luca Cavanna
8a05b670ca
Address MinAndMax generics warnings (#52642)
`MinAndMax` encapsulates min and max values for a shard. It uses generics to make sure that the values are of the same type and are also comparable. Though there are warnings whenever this class is currently used, which are addressed with this commit.

Relates to #49092
2020-03-03 16:08:10 +01:00
Adrien Grand
cb868d2f5e
Introduce a constant_keyword field. (#49713) (#53024)
This field is a specialization of the `keyword` field for the case when all
documents have the same value. It typically performs more efficiently than
keywords at query time by figuring out whether all or none of the documents
match at rewrite time, like `term` queries on `_index`.

The name is up for discussion. I liked including `keyword` in it, so that we
still have room for a `singleton_numeric` in the future. However I'm unsure
whether to call it `singleton`, `constant` or something else, any opinions?

For this field there is a choice between
 1. accepting values in `_source` when they are equal to the value configured
    in mappings, but rejecting mapping updates
 2. rejecting values in `_source` but then allowing updates to the value that
    is configured in the mapping
This commit implements option 1, so that it is possible to reindex from/to an
index that has the field mapped as a keyword with no changes to the source.

Backport of #49713
2020-03-03 16:01:47 +01:00
James Rodewig
bcb68c860c [DOCS] Reorganize EQL requirements page 2020-03-03 07:02:30 -05:00
Yang Wang
70814daa86
Allow _rollup_search with read privilege (#52043) (#53047)
Currently _rollup_search requires manage privilege to access. It should really be
a read only operation. This PR changes the requirement to be read indices privilege.

Resolves: #50245
2020-03-03 22:29:54 +11:00
Alan Woodward
3759063d34 Allow specifying an exclusive set of fields on ObjectParser (#52893)
ObjectParser allows you to declare a set of required fields, such that at least one
of the set must appear in an xcontent object for it to be valid. This commit adds
the similar concept of a set of exclusive fields, such that at most one of the set
must be present. It also enables required fields on ConstructingObjectParser, and
re-implements PercolateQueryBuilder.fromXContent() to use object parsing as
an example of how this works.
2020-03-03 10:56:20 +00:00
Martijn van Groningen
510db25dd0
Simplify watcher indexing listener.(#53046)
Backport: #52627

Add watcher to trigger server after index operation has succeeded,
instead of adding a watch to trigger service before
the actual index operation has performed on the shard level.

This logic is simpler to reason about in the case that a failure
does occur during the execution of an index operation on
the shard level.

Relates to #52453, but I think doesn't fix it, but makes it easier
to debug.
2020-03-03 11:01:57 +01:00
Hendrik Muhs
844f350774 [Transform] restructure transform yaml tests (#52956)
restructure transform yaml tests to run cleanup in teardown phase

relates #52428
2020-03-03 10:31:22 +01:00
Hendrik Muhs
d9258e210e [Transform] fix sporadic race condition in TransformUsageIT (#52946)
relax the test for trigger count

fixes #52931
2020-03-03 10:27:36 +01:00
Jason Tedor
a154f9c657
Early return if no global checkpoint listeners (#53036)
When notifying global checkpoint listeners, we have an opportunity to
early return if there are not any registered listeners. This is
important since it saves some allocations, and also saves forking some
empty work to another thread. This commit adds an early return from
notifying listeners if there are not any registered.
2020-03-02 23:28:22 -05:00
James Rodewig
5cffa14f45 [DOCS] Fix typo in EQL docs 2020-03-02 16:09:03 -05:00
Costin Leau
712e0c05cd EQL: Add implicit ordering on timestamp (#53004)
QL: Move Sort base class from SQL to QL
(cherry picked from commit 798015b7bbd565e9c4222724614baeb432c7c2b3)
2020-03-02 22:41:36 +02:00
Mark Vieira
f8396e8d15
Mute RunDataFrameAnalyticsIT.testStopOutlierDetectionWithEnoughDocumentsToScroll
Signed-off-by: Mark Vieira <portugee@gmail.com>
2020-03-02 09:21:55 -08:00
Mark Vieira
5b5e92c71d
Mute NodeSubclassTests.testReplaceChildren
Signed-off-by: Mark Vieira <portugee@gmail.com>
2020-03-02 09:21:54 -08:00
Lisa Cawley
5a293b29a1 [DOCS] Add missing anchors in cron expressions (#52960) 2020-03-02 09:07:22 -08:00
Orhan Toy
ad2f630795 [DOCS] Fix formatting of simulate ingest pipeline API docs (#52754) 2020-03-02 11:46:27 -05:00
István Zoltán Szabó
a267849def [DOCS] Adds transform-settings page and its subpage to redirects (#52944)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-03-02 16:36:40 +01:00
Lisa Cawley
4fbe1b0550
[DOCS] Adds cat anomaly detectors API (#52866) (#52970) 2020-03-02 07:28:55 -08:00
Hendrik Muhs
a328a8eaf1
[7.x][Transform] implement node.transform to control where to… (#52998)
implement transform node attributes to disable transform on certain nodes and
test which nodes are allowed to do remote connections

closes #52200
closes #50033
closes #48734

backport #52712
2020-03-02 16:10:57 +01:00
James Rodewig
db64029919
[7.x] [DOCS] Add parameter examples to EQL search tutorial (#52953)
Makes the following updates to the EQL search tutorial:

* Adds an API response to the basic tutorial
* Adds an example using the `event_type_field` parm
* Adds an example using the `timestamp_field`parm
* Adds an example using the `query` parm
* Updates example dataset to support more EQL query variety
2020-03-02 10:08:03 -05:00
Aleksandr Maus
89ed857c79
EQL: Change request parameter query to filter and rule to query (#52971) (#53006)
Related to https://github.com/elastic/elasticsearch/issues/52911
2020-03-02 09:26:23 -05:00
James Rodewig
d336faa0b0 [DOCS] Reformat trim token filter docs (#51649)
Makes the following changes to the `trim` token filter docs:

* Updates description
* Adds a link to the related Lucene filter
* Adds tip about removing whitespace using tokenizers
* Adds detailed analyze snippets
* Adds custom analyzer snippet
2020-03-02 07:48:23 -05:00
James Rodewig
f5bccad847 [DOCS] Correct guidance for index_options mapping parm (#52899)
Adds a warning admonition stating that the `index_options` mapping
parameter is intended only for `text` fields.

Removes an outdated statement regarding default values for numeric
and other datatypes.
2020-03-02 07:39:35 -05:00
rhymes
7eb4c07f1f [DOCS] Fix typo in index and search analysis docs (#52988) 2020-03-02 07:25:01 -05:00
Andrei Stefan
6fecc1db84
Issue a different error message in case an index doesn't have a mapping (#52967) (#53003)
(cherry picked from commit a0bd83a0579cf196a1d727de2a46b3b101d5a73b)
2020-03-02 14:04:49 +02:00
Andrei Stefan
69383acecf
Define list of Nodes that have minimum two children in tests (#52957) (#52994)
(cherry picked from commit c1e43e694f02edf3e197abbab7c21008c022b516)
2020-03-02 11:26:50 +02:00
Hendrik Muhs
49f41d127b [Transform] fix NPE in derive stats if shouldStopAtNextCheckpo… (#52940)
fixes a NPE in _stats in case shouldStopAtNextCheckpoint is set.
2020-03-02 08:11:01 +01:00
Martijn van Groningen
d102158e6f
Improve closing mock webserver when failed to start (#52943)
Fix NPE when closing a webserver that hasn't started correctly.

This can happen when ssl context isn't initialized. The server instance is then never set,
which causes an NPE that masks the actual failure.

Example stacktrace that would mask an actual failure:

```
java.lang.NullPointerException
	at org.elasticsearch.test.http.MockWebServer.close(MockWebServer.java:271)
	at org.elasticsearch.xpack.watcher.test.integration.HttpSecretsIntegrationTests.cleanup(HttpSecretsIntegrationTests.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
```
2020-03-02 07:19:08 +01:00
Yang Wang
712191f2af
[DOCS] Fix typo and ensure asterisks render properly (#52991) (#52992)
Fix a typo and rendering issue for doc test example.
2020-03-02 11:56:42 +11:00
Stuart Tettemer
210aab0935
Settings: AffixSettings as validator dependencies (#52973) (#52982)
Allow AffixSetting as validator dependencies.  If a validator
specifies AffixSettings as a dependency, then `validate(T, Map)`
will have the concrete setting in a map.

Backport of: #52973, 1e0ba70
Fixes: #52933
2020-02-29 09:38:46 -07:00
Nhat Nguyen
e6755afeeb
Upgrade to Lucene 8.5.0-snapshot-c4475920b08 (#52950) (#52977)
To give LUCENE-9228 more CI cycles
2020-02-29 09:29:16 -05:00
Dimitris Athanasiou
85b4e45093
[7.x]ML] Parse and report memory usage for DF Analytics (#52778) (#52980)
Adds reporting of memory usage for data frame analytics jobs.
This commit introduces a new index pattern `.ml-stats-*` whose
first concrete index will be `.ml-stats-000001`. This index serves
to store instrumentation information for those jobs.

Backport of #52778 and #52958
2020-02-29 13:03:40 +02:00
Mark Vieira
d9463983af
Enable parallel builds by default (#52972) 2020-02-28 15:09:40 -08:00
Rory Hunter
b1be7dcd2d Document how to change GC logging behaviour (#52879)
Closes #43990. Describe how to change the default GC settings without changing
the default `jvm.options`. Give examples using `jvm.options.d`, and
`ES_JAVA_OPTS` with Docker.
2020-02-28 21:27:45 +00:00
Jay Modi
1cd0eee723
Remove TODO in IndexNameExpressionResolver (#52969)
This commit removes a TODO in the IndexNameExpressionResolver that
indicated the API should use a Set instead of a List. However, this
TODO was not completely correct since the ordering of arguments matters
due to negations when evaluating wildcards and since we also allow
a list of patterns like `*,-foo,*`, which would have a different
meaning even when using a Set with insertion ordering.

Relates #52788
Backport of #52963
2020-02-28 13:56:28 -07:00
Rory Hunter
39de995740 Exclude generated source from benchmarks formatting (#52968)
IDEs can sometimes run annotation processors that leave files in
`src/main/generated/**/*.java`, causing Spotless to complain. Even
though this path ought not to exist, exclude it anyway in order to avoid
spurious failures.
2020-02-28 20:55:52 +00:00
Adrien Grand
331d4bb0af
HybridDirectory should mmap postings. (#52641) (#52873)
Since version 8.4, `MMapDirectory` has an optimization to read long[]
arrays directly in little endian order, which postings leverage. So it'd
be more efficient to open postings with `MMapDirectory`.

I refactored a bit the existing logic to better explain why every listed
file extension is open with `mmap`.
2020-02-28 18:45:46 +01:00
Luca Cavanna
090bdf69c0
Mute NodeSubclassTests#testReplaceChildren (#52952)
Relates #52951
2020-02-28 16:13:17 +01:00
Dan Hermann
dd44376d27
[7.x] Send the fields param in body instead of URL params (#52948) 2020-02-28 08:57:35 -06:00
Rory Hunter
d863c510da
Autoformat :qa:os and :benchmarks (#52816)
Add `:qa:os` and `:benchmarks` to the list of automatically formatted
projects, and apply some manual fix-ups to polish it up.

In particular, I noticed that `Files.write(...)` when passed a list will
automaticaly apply a UTF-8 encoding and write a newline after each line,
making it easier to use than FileUtils.append. It's even available from
1.8.

Also, in the Allocators class, a number of methods declared thrown exceptions that IntelliJ reported were never thrown, and as far as I could see this is true, so I removed the exceptions.
2020-02-28 14:48:04 +00:00