Commit Graph

53224 Commits

Author SHA1 Message Date
Rory Hunter e2e3fb12bc Fixes after adding UBI Docker support
The CertGenCliTests needed an update to ignore the UBI Docker
distribution as well as the regular Docker distribution.
2020-08-18 21:09:11 +01:00
Nhat Nguyen 08b0e78ef4 Log more info when search ops higher than expected (#61108)
We have seen a situation where the total search operations are higher 
than expected. Unfortunately, we did not have enough info to figure it
out. This commit adds the failures to the error to provide more context
and adjusts the log level in case of failure to debug.
2020-08-18 15:20:41 -04:00
Lisa Cawley b120368aee
[DOCS] Add security updates to release notes (#61288) (#61296) 2020-08-18 12:00:21 -07:00
David Roberts 96256bd4df
[DOCS] Add 7.9.0 known issue for problems with ML index mappings (#61289)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-08-18 11:46:08 -07:00
Lisa Cawley 2015d5f86a
[DOCS] Removes 7.9.0 coming tags (#61293) 2020-08-18 11:24:52 -07:00
Andrei Stefan 93abbb9057
Add data streams wildcard pattern yml test (#61269) (#61280)
(cherry picked from commit e13a365eeb6d8c6a7c9a91f94f0e8e78e3fe4773)
2020-08-18 19:38:07 +03:00
Andrei Stefan 5de0f19cc3
EQL: Return sequence join keys in the original type (#61268) (#61282)
(cherry picked from commit d54957d61faa0d502387656e3cace594017b6ea0)
2020-08-18 19:37:15 +03:00
István Zoltán Szabó 78d77ebed7
[DOCS] Replaces transform.node with node.roles: [ transform ] in transforms settings. (#61267) (#61271) 2020-08-18 18:00:06 +02:00
István Zoltán Szabó 7596bb7454
[DOCS] Clarifies node.roles settings (#61266) (#61274) 2020-08-18 17:59:34 +02:00
Pius d6ad247e07
[DOCS] Note max field expansions is not a hard limit (#61120)
Per #35284, it looks like we changed this from a max field expansions limit to a soft limit using the `indices.query.bool.max_clause_count` dynamic cluster settting.
2020-08-18 11:39:09 -04:00
Rory Hunter d0c57c3068 Fix PackagingTestCase for DOCKER_UBI variant 2020-08-18 16:12:03 +01:00
Rory Hunter bd7236cd65 Version bump for 7.9.0 release 2020-08-18 16:07:43 +01:00
Martijn van Groningen cbf60f6c5e
Add tests that simulate new indexing strategy upgrade procedure. (#61263)
Backport of #61082 to 7.x branch.

Closes #58251
2020-08-18 17:02:29 +02:00
Dimitrios Liappis c870640cbd
[7.x] Introduce 6.8.13 as a version (#61198)
Introduce version 6.8.13 to branch 7.x
2020-08-18 17:07:16 +03:00
Nik Everett 8a387d6df1 Redo experimental tag on vwh (#61065)
The docs didn't have the standard experimental text. This adds it.
2020-08-18 10:02:26 -04:00
James Rodewig e63c12f443
[DOCS] Fix typo in Java HLRC docs (#60863) (#61264)
Co-authored-by: bumjin <bumjin@gmail.com>
2020-08-18 09:09:10 -04:00
István Zoltán Szabó 06ba99413d
[DOCS] Replaces ml.node with node.roles: [ ml ] in ML settings (#61017) (#61257) 2020-08-18 12:06:17 +02:00
Andrei Stefan ad627c7eab
Introduce ordering in the constant_keyword test for better predictibility. (#61248) (#61252)
(cherry picked from commit 69193f9de8178dbaa1d8467f1686b100dd2b161c)
2020-08-18 12:17:15 +03:00
Armin Braun 58d07b2ffc
Remove Unused ByteBufferReference (#61116) (#61250)
We only work with heap byte buffers at this point and those we can and do unwrap the
`byte[]` ourselves and use `BytesArray` instead of a needless level of indirection via `ByteBuffer`.
2020-08-18 10:53:40 +02:00
Rory Hunter d8aacbdd31 Add UBI docker builds
Backport of #60742.

This PR resurrects support for building Docker images based on one of
Red Hat's UBI images. It also adds support for running the existing
Docker tests against the image. The image is named
`elasticsearch-ubi8:<version>`.

I also changed the Docker build file uses enums instead strings in a lot
of places, for added rigour.
2020-08-18 09:48:51 +01:00
Armin Braun 6ffa7f0737
Fix testConcurrentSnapshotDeleteAndDeleteIndex (#61228) (#61249)
There is a corner case here in which during partial snapshot the index is
deleted right between starting the snapshot in the CS and the data node getting to work
on it, causing the data node the fail that shard snapshot and making the snapshot `PARTIAL`.

Closes #61208
2020-08-18 10:45:30 +02:00
Adam Locke a0af82c213
[7.x] [DOCS] Update CCR docs to focus on Kibana. (#61237)
* First crack at rewriting the CCR introduction.

* Emphasizing Kibana in configuring CCR (part one).

* Many more edits, plus new files.

* Fixing test case.

* Removing overview page and consolidating that information in the main page.

* Adding redirects for moved and deleted pages.

* Removing, consolidating, and adding redirects.

* Fixing duplicate ID in redirects and removing outdated reference.

* Adding test case and steps for recreating a follower index.

* Adding steps for managing CCR tasks in Kibana.

* Adding tasks for managing auto-follow patterns.

* Fixing glossary link.

* Fixing glossary link, again.

* Updating the upgrade information and other stuff.

* Apply suggestions from code review

* Incorporating review feedback.

* Adding more edits.

* Fixing link reference.

* Adding use cases for #59812.

* Incorporating feedback from reviewers.

* Apply suggestions from code review

* Incorporating more review comments.

* Condensing some of the steps for accessing Kibana.

* Incorporating small changes from reviewers.
2020-08-17 16:58:13 -04:00
James Rodewig 06d3159125
[DOCS] Add usage tips to `top_hits` agg (#61215) (#61225) 2020-08-17 13:05:40 -04:00
Leaf-Lin e258c85b6a [DOCS] Update configuring-metricbeat.asciidoc (#60857)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2020-08-17 10:04:52 -07:00
James Rodewig 6a248aea07
[DOCS] Add admon for built-in index templates (#61063) (#61220)
Adds an important admonition for the built-in `metrics-*-*` and `logs-*-*` index
templates.

Updates several put index template snippets to include a priority.
2020-08-17 12:48:34 -04:00
Mark Tozzi db1df6cc30
[7.x] Remove a bunch of type boilerplate from Aggs (#60852) (#61031) 2020-08-17 12:13:05 -04:00
Mark Vieira d26a3e144e
Revert "Simplify distribution download and extraction (7.x backport) (#61184)"
This reverts commit 8b7a0a1f64.
2020-08-17 08:54:08 -07:00
Adam Locke a3f357c8a5
[DOCS] Update info about geo_shape bounding boxes (#61214) (#61216)
* Adding information about geo_shape bounding boxes.

* Fixing cross link and incorporating review feedback.
2020-08-17 11:44:46 -04:00
James Rodewig 60876a0e32
[DOCS] Replace Wikipedia links with attribute (#61171) (#61209) 2020-08-17 11:27:04 -04:00
Andrei Stefan db8788e5a2
QL: wildcard field type support (#58062) (#61205)
(cherry picked from commit c874e6cdd3e051ce599b50c18642de038b84105f)
2020-08-17 18:24:32 +03:00
Andrei Stefan 90e116738e
QL: add filtering query dsl support to IndexResolver (#60514) (#61200)
(cherry picked from commit 7b3635d796be26af9f87d19963a8ed4ab4bbf13f)
2020-08-17 17:59:58 +03:00
Nik Everett 1b7bbafd81
Add method to make random DateFormatter pattern (backport of #60613) (#61213)
Adds a method to make a random date `DateFormatter` pattern. We expect
this'll be useful for runtime fields to compate their formatting with
the standard date field.
2020-08-17 10:57:52 -04:00
Christoph Büscher 6866396e1d Improve 'ignore_malformed' handling for dates (#60211)
Currently we occasionally can get ArithmeticException from parsing bad input
values on 'date' fields that are passed on even if 'ignore_malformed' is set.
This change adds this exception to the ones we already catch for malformed
values.

Closes #52634
2020-08-17 16:18:08 +02:00
István Zoltán Szabó bc9170387a
[DOCS] Adds clarification to node roles (#61206) (#61211) 2020-08-17 16:15:53 +02:00
Nik Everett 1e6400285c
Some progress on failing runtime fields tests (bring #61098 to 7.x) (#61115)
* Some progress on failing runtime fields tests (bring #61098 to 7.x)

This breaks apart the a test for the `terms` aggregation into one that
work for runtime fields and one that doesn't.
2020-08-17 09:56:55 -04:00
Nik Everett 639782da12
Break up a test for with runtime fields (brings #60931 to master) (#61103) (#61114)
Breaks up an integration test into one that runtime fields can run and
one that runtime fields have to skip. This is because runtime fields
don't have global ords and we assert things *about* global ords in the
test we have to skip.
2020-08-17 09:56:33 -04:00
James Rodewig 81b8024d66
[DOCS] Fix typo in suggester docs (#61077) (#61202)
Co-authored-by: Arash Layeghi <arashlayeghi57@gmail.com>
2020-08-17 09:09:34 -04:00
David Turner b21cb7f466 Reduce allocations when persisting cluster state (#61159)
Today we allocate a new `byte[]` for each document written to the
cluster state. Some of these documents may be quite large. We need a
buffer that's at least as large as the largest document, but there's no
need to use a fresh buffer for each document.

With this commit we re-use the same `byte[]` much more, only allocating
it afresh if we need a larger one, and using the buffer needed for one
round of persistence as a hint for the size needed for the next one.
2020-08-17 13:45:31 +01:00
Dan Hermann 524247bbc0
[DOCS] write_index_only option for put mapping (#59610) (#61172) 2020-08-17 07:33:49 -05:00
David Kyle ba89af544f
[7.x] Respect ML upgrade mode in TrainedModelStatsService (#61143) (#61187)
When in upgrade mode the ml stats service should not write to the stats index.
2020-08-17 11:09:25 +01:00
David Turner f3e0c60896
Restrict testing of legacy discovery to tests (#61178)
The 7.x branch preserves the legacy discovery mechanism from 6.x purely
for running internal cluster tests; this mechanism is otherwise
completely untested and unsupported. However it is still technically
possible to use it outside of the test suite if you dig through the
source code to work out what settings need to be set. With this change
we make it impossible to use this mechanism in production.

Closes #61177
2020-08-17 11:05:27 +01:00
Rene Groeschke 8b7a0a1f64
Simplify distribution download and extraction (7.x backport) (#61184)
We leverage artifact transforms now when downloading and unpacking elasticsearch distributions.

This has the benefit of

- handcrafted extract tasks on the root project are not required. 
- the general tight coupling to the root project has been removed.
- the overall required configurations required to handle a distribution have been reduced
- ElasticsearchDistribution has been simplified by making Extracted an ordinary Configuration
downloaded and unpacked external distributions are reused in later builds by been cached
in the gradle user home.

DistributionDownloadPlugin functional tests have been extended and ported
to DistributionDownloadPluginFuncTest.

* Fix java8 compliant Path calculation
2020-08-17 10:10:32 +02:00
James Rodewig 1ffc983f98 [DOCS] Fix link in similarity module docs 2020-08-14 18:31:07 -04:00
James Rodewig 290adcd25e [DOCS] Reword in EQL threat detection example 2020-08-14 15:50:58 -04:00
James Rodewig d0810cca19
[DOCS] Add xref to multiplexer token filter docs (#60431) (#61168)
Co-authored-by: paiboon auengkongkatong <paiboon15721@gmail.com>
2020-08-14 15:05:07 -04:00
James Rodewig 8263ce79e9
[DOCS] Update ingest processor snippet for ECS (#61128) (#61164)
Co-authored-by: Nicole Albee <2642763+a03nikki@users.noreply.github.com>
2020-08-14 14:21:47 -04:00
Rene Groeschke 9ca052d702
Use native Gradle support for --release flag (#61093) (#61151)
With Gradle 6.6 we can now use the native support for the --release compile options. 
As Gradle by default resolves the compatibility version from the release property we needed to workaround this in order to keep our current setup. An issue was raised at gradle to track this at https://github.com/gradle/gradle/issues/14141
2020-08-14 20:01:24 +02:00
James Rodewig 3fef26bfb0
[DOCS] EQL: Add threat detection example (#59105) (#61161) 2020-08-14 13:40:44 -04:00
Benjamin Trent 43fc6c34bc
Muting analytics integration tests for change new native output model_metadata (#61158)
relates to elastic/ml-cpp#1456
2020-08-14 11:45:35 -04:00
Benjamin Trent 038cc26ac5
[ML] adjusts feature importance format for hlrc (#61150) (#61153)
related to PR https://github.com/elastic/elasticsearch/pull/61104
2020-08-14 11:33:41 -04:00