Commit Graph

53404 Commits

Author SHA1 Message Date
Przemysław Witek 11c2710e7f
[7.x] [ML] Do not mark the DFA job as FAILED when a failure occurs after the node is shutdown (#61331) (#61526) 2020-08-26 09:53:13 +02:00
lcawl 5fa839b906 [DOCS] Fix typo in update anomaly detection job API 2020-08-25 17:13:38 -07:00
Igor Motov f70a59971a
[7.x] Add rate aggregation (#61369) (#61554)
Adds a new rate aggregation that can calculate a document rate for buckets
of a date_histogram.

Closes #60674
2020-08-25 17:39:00 -04:00
debadair 82585107aa
updated shard limit doc (#56496) (#61509)
* updated shard limit doc

As the documentation was not so clear. I have updated saying this limit includes open indices with unassigned primaries and replicas count towards the limit.

* [DOCS] Incorporated edits.

Co-authored-by: Deb Adair <debadair@elastic.co>

Co-authored-by: gadekishore <50092970+gadekishore@users.noreply.github.com>
2020-08-25 14:24:47 -07:00
James Rodewig e0843571c4 [DOCS] Fix typo in search your data docs 2020-08-25 17:01:08 -04:00
Nik Everett 87cf81e179
Migrate some more mapper test cases (#61507) (#61552)
Migrate some more mapper test cases from `ESSingleNodeTestCase` to
`MapperTestCase`.
2020-08-25 15:27:26 -04:00
markharwood 8b56441d2b
Search - add case insensitive support for regex queries. (#59441) (#61532)
Backport to add case insensitive support for regex queries. 
Forks a copy of Lucene’s RegexpQuery and RegExp from Lucene master.
This can be removed when 8.7 Lucene is released.

Closes #59235
2020-08-25 17:18:59 +01:00
James Rodewig e3d23c34ab
[DOCS] Document static HTTP settings (#61429) (#61536) 2020-08-25 11:27:05 -04:00
James Rodewig 5ad0ce49e1
[DOCS] Remove response params for #61428 (#61524) (#61534) 2020-08-25 11:17:56 -04:00
Brandon Morelli fade7408cd [DOCS] Fix link to quartz crontrigger tutorial (#61531) 2020-08-25 10:49:00 -04:00
Przemyslaw Gomulka f3f7d25316
Header warning logging refactoring backport(#55941) (#61515)
Splitting DeprecationLogger into two. HeaderWarningLogger - responsible for adding a response warning headers and ThrottlingLogger - responsible for limiting the duplicated log entries for the same key (previously deprecateAndMaybeLog).
Introducing A ThrottlingAndHeaderWarningLogger which is a base for other common logging usages where both response warning header and logging throttling was needed.

relates #55699
relates #52369
backports #55941
2020-08-25 16:35:54 +02:00
Costin Leau bff3c7470e
EQL: Replace SearchHit in response with Event (#61428) (#61522)
The building block of the eql response is currently the SearchHit. This
is a problem since it is tied to an actual search, and thus has scoring,
highlighting, shard information and a lot of other things that are not
relevant for EQL.
This becomes a problem when doing sequence queries since the response is
not generated from one search query and thus there are no SearchHits to
speak of.
Emulating one is not just conceptually incorrect but also problematic
since most of the data is missed or made-up.

As such this PR introduces a simple class, Event, that maps nicely to
the terminology while hiding the ES internals (the use of SearchHit or
GetResult/GetResponse depending on the API used).

Fix #59764
Fix #59779

Co-authored-by: Igor Motov <igor@motovs.org>
(cherry picked from commit 997376fbe6ef2894038968842f5e0635731ede65)
2020-08-25 17:32:42 +03:00
Armin Braun f22ddf822e
Some Optimizations around BytesArray (#61183) (#61511)
* Faster `equals` for `BytesArray` which is nice since with this change we use it for the search cache
* Lighter `StreamInput` for `BytesArray` that should save memory and some indirection relative to the one on the abstract bytes reference
* Lighter `writeTo` implementation
* Build a `BytesArray` instead of a PagedBytesReference whenever possible to save indirection and memory
2020-08-25 07:13:39 +02:00
Armin Braun 806dfcfcf7
Speed up Compression Logic by Pooling Resources (#61358) (#61495)
This is mostly motivated by the performance issues we are seeing around the GET mappings
REST API which (in case of a large number of indices) will create decompressing streams in a hot loop
which takes a significant amount of time for the system calls involved in instantiating deflaters
and inflaters.
Also, this fixes a leaked deflater when deserializing cached repository data.
2020-08-25 04:01:55 +02:00
Armin Braun 16b932c1dc
Remove Potentially Expensive Use of BytesReference.toBytesRef (#61415) (#61503)
This method might have materialize all the bytes in a reference into a fresh `byte[]`.
Using the stream is much safer and only trivially more expensive + in most cases we now run the fast path via `BytesArray` anyway.
2020-08-24 23:58:21 +02:00
David Kyle 539cf914bc
[ML] handle new model metadata stream from native process (#59725) (#61251)
This adds the serialization handling for the new model_metadata object from the native process.

Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2020-08-24 15:52:13 -04:00
James Rodewig 2400098a52
[DOCS] Fix typo in profile API docs (#61445) (#61501)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: shashikumarec088 <shashikumarec088@gmail.com>
2020-08-24 15:30:18 -04:00
Nhat Nguyen baa685c2d9 Fix anchor doc for msearch cancellation paragraph
Relates #61418
2020-08-24 15:14:17 -04:00
Nhat Nguyen f34d3efae7 Add cancellation doc for multi search (#61418)
Relates #61337
2020-08-24 15:14:05 -04:00
Nhat Nguyen d47bbbafe0 Cancel multisearch when http connection closed (#61399)
Relates #61337
2020-08-24 15:12:54 -04:00
Nhat Nguyen 23a0f8b617 Detect and optimize noop of update index settings (#61348)
This optimization is more relevant in the context of CCR. When a node in
the follower cluster leaves, we reallocate the shard-follow tasks on 
that node to other nodes. The new tasks will overwhelm the follower
cluster with many put-mapping, update-settings requests, although most
of them are noop. This change detects and optimizes the noop
update-settings requests.
2020-08-24 15:08:53 -04:00
James Rodewig 439fa46735
[DOCS] Remove collapsible sections in EQL fn docs (#61498) (#61499) 2020-08-24 14:41:27 -04:00
Benjamin Trent 6ffcc02fb9
Muting test o.e.t.t.ESTestCaseTests.testRandomDateFormatterPattern (#61497) 2020-08-24 13:58:09 -04:00
Nik Everett f3b6d49ae1
Migrate server mapper tests to new MapperTestCase (#61378) (#61490)
This continues #61301, migrating all of the mappers in `server` to the
new `MapperTestCase` which is nicer than `FieldMapperTestCase` because
it doesn't depend on all of Elasticsearch.
2020-08-24 13:33:35 -04:00
James Rodewig 17b5a0d25e
[DOCS] Combine `Search your data` files (#61477) (#61486)
No-op changes to:

* Move `Search your data` source files into the same directory
* Rename `Search your data` source files based on page ID
* Remove unneeded includes
* Remove the `Request` dir
2020-08-24 13:08:00 -04:00
Benjamin Trent 1ae2923632
[7.x] [ML] adding docs + hlrc for data frame analysis feature_processors (#61149) (#61493)
* [ML] adding docs + hlrc for data frame analysis feature_processors (#61149)

Adds HLRC and some docs for the new feature_processors field in Data frame analytics.

Co-authored-by: Przemysław Witek <przemyslaw.witek@elastic.co>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-08-24 12:56:21 -04:00
Armin Braun d05649bfae
Fix PutPolicyRequestTests.testFromXContent (#61485) (#61494)
We only ever support `JSON` for the query source format in practice.
The reason this test worked before is a bug in xcontent parsing that parses
empty maps out of streams of the wrong format.

Closes #61483
2020-08-24 18:52:05 +02:00
Armin Braun bb4d97073c
Remove Favicon Special Path in RestController (#61460) (#61487)
It's unnecessary (and adds one string comparison to every request) to special
case the favicon so I added it as a normal REST handler to simplify the code.
2020-08-24 18:36:23 +02:00
James Rodewig 2b852388c5
[DOCS] Fix hyphenation for "time series" (#61472) (#61481) 2020-08-24 11:18:07 -04:00
Dimitris Athanasiou 618dd65d5f
[7.x][ML] Add debug logging for field caps request during DF Analytics (#61459) (#61478)
Adds debug logging for the request and the response that is getting
field capabilities during a data frame analytics job.

Backport of #61459
2020-08-24 18:01:30 +03:00
James Rodewig 5992bb0507
[DOCS] Fix ingest script compilation rate and cache size (#61468) (#61479) 2020-08-24 10:46:44 -04:00
Dimitris Athanasiou 18ca8a6be3
[7.x][ML] Remove redundant logging for creation of annotations index (#61461) (#61475)
This commit removes the log info message "Created ML annotations index and aliases".

The message comes in addition to elasticsearch's index creation logging and it does
not add to it. In addition, since #61107 that message may be logged multiple times.

Backport of #61461
2020-08-24 17:46:29 +03:00
Lisa Cawley 52b12a07c4 [DOCS] Document static machine learning settings (#61382) 2020-08-24 07:35:38 -07:00
Armin Braun af2e2782eb
Stop Needlessly Copying Bytes in XContent Parsing (#61447) (#61469)
Wrapping a `BytesArray` in a `StreamInput` for deserialization is inefficient.
This forces Jackson to internally buffer (i.e. copy) all bytes from the `BytesArray`
before deserializing, adding overhead for copying the bytes and managing the buffers.

This commit fixes a number of spots where `BytesArray` is the most common type of
`BytesReference` to special case this type and parse it more efficiently.
Also improves parsing `String`s to use the more efficient direct `String` parsing APIs.
2020-08-24 15:49:15 +02:00
James Rodewig 3373b1406a
[DOCS] Fix typo in CCR Put Follow API docs (#61392) (#61470)
Co-authored-by: Mark Laney <mark1@elastic.co>
2020-08-24 09:46:23 -04:00
James Rodewig 2100441ef8
[DOCS] Note the cluster settings API can override `elasticsearch.yml` (#61394) (#61464)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-08-24 09:32:26 -04:00
James Rodewig da89ff87bb
[DOCS] Prune `Search your data` content (#61303) (#61462)
Changes:
* Removes narrative around URI searches. These aren't commonly used in production. The `q` param is already covered in the search API docs: https://www.elastic.co/guide/en/elasticsearch/reference/master/search-search.html#search-api-query-params-q
* Adds a common options section that highlights narrative docs for query DSL, aggregations, multi-index search, search fields, pagination, sorting, and async search.
* Adds a `Search shard routing` page. Moves narrative docs for adaptive replica selection, preference, routing , and shard limits to that section.
* Moves search timeout and cancellation content to the `Search your data` page.
* Creates a `Search multiple data streams and indices` page. Moves related narrative docs for multi-target syntax searches and `indices_boost` to that page.
* Removes narrative examples for the `search_type` parameters. Moves documentation for this parameter to the search API docs.
2020-08-24 09:31:53 -04:00
Rory Hunter 0d8d0f423c Allow running the Docker image with a non-default group (#61194)
Closes #60864. Tweak the JDK directories' permissions in the ES
Docker image so that ES can run under a different user and group.

These changes assume that the image is being run with bind-mounted
config, data and logs directories, and reads and writes to these
locations will still fail when both the UID and GID are not the
default. Everything should be OK when running with the default GID
of zero, however.
2020-08-24 11:17:40 +01:00
Yang Wang f0615113b6
Report anonymous roles in authenticate response (#61355) (#61454)
Report anonymous roles in response to "GET _security/_authenticate" API call when:
* Anonymous role is enabled
* User is not the anonymous user
* Credentials is not an API Key
2020-08-24 14:51:44 +10:00
Ryan Ernst d1031fd928
Move versions convenience property to java plugin (#61438)
Access the common versions map is done in a lot of places. While it can
be access through an import of VersionProperties, the vast majority of
places use it through the provided convenience property added by
BuildPlugin. This commit moves that convenience property to the base
java plugin, so further reduce dependence on the BuildPlugin.
2020-08-21 16:43:02 -07:00
Qaiser Abbasi e2400dcd8e
Remove legacy debugger instructions (#60583)
In the past, the only way to run a local Elasticsearch build with a remote debugger was by extracting elasticsearch and passing ES_JAVA_OPTS. However, since switching to gradle, a convenience flag was added, `--debug-jvm` (which is documented elsewhere in the testings docs), when running a local elasticsearch build through gradle. This commit removes the old documentation.
2020-08-21 14:08:12 -07:00
Lisa Cawley 7c48a0fc8c [DOCS] Document static dynamic transform settings (#61384) 2020-08-21 13:04:54 -07:00
James Rodewig cbb5f18f81
[DOCS] Document `xpack.graph.enabled` setting (#60073) (#61433) 2020-08-21 15:13:13 -04:00
James Rodewig e92c62bdf8
[7.x] [DOCS] Fix query example for wildcard datatype (#61398) (#61431)
Co-authored-by: jessepeixoto <jessepeixoto@gmail.com>
2020-08-21 12:43:41 -04:00
Dan Hermann c53731a0cd
[7.x] Fix wrong pipeline name in debug log (#58817) (#61233) 2020-08-21 11:14:01 -05:00
James Rodewig cb5e9d3bee
[DOCS] Remove URI search examples from API reference (#61423) (#61425) 2020-08-21 11:19:11 -04:00
Yang Wang 0509465a9e
Warn about unlicensed realms if no auth token can be extracted (#61402) (#61419)
There are warnings about unlicense realms when user lookup fails. This PR adds
similar warnings for when no authentication token can be extracted from the request.
2020-08-22 00:04:45 +10:00
James Rodewig 1b3a002588
[DOCS] Fix ingest processor TOC sort (#61412) (#61416) 2020-08-21 09:21:41 -04:00
Yang Wang cd52233b94
Include authentication type for the authenticate response (#61247) (#61411)
Add a new "authentication_type" field to the response of "GET _security/_authenticate".
2020-08-21 22:59:43 +10:00
James Rodewig bba4220982
[DOCS] Fix `field` def for join processor (#61395) (#61413) 2020-08-21 08:53:38 -04:00