We need to either exclude null responses from the scroll search response
or always create a search context for every target shards, although that
scroll query can be written to match_no_docs. Otherwise, we won't find
search_context for subsequent scroll requests.
This commit implements the latter option as it's less error-prone.
Relates #51708
In the packaging tests, we added convenience methods for asserting file
existence and file non-existence. This commit replaces the remaining
uses of assertFalse/assertTrue in favor of these dedicated matchers.
Java 8 can not infer types as well as Java 11 does. This means a
backport of some code that relied on Java 11's superior abilities to
infer types caused Java 8's head to explode trying to infer the same
type in 7.x. This commit addresses this by giving Java 8 the types that
it needs.
This commit provides a path to set register the autoscaling feature flag
in release builds, and therefore enabling autoscaling in release
builds. The primary reason that we add this is so that our release docs
tests can pass. Our release docs tests do not have infrastructure in
place to only register snippets from included portions of the docs, they
instead include all docs snippets. Since autoscaling can not be enabled
in release builds, this meant that the autoscaling snippets would fail
in the release docs tests. To address then, we need the ability to
enable autoscaling in the release docs tests which we can now do with
the system property added here. This system property will be removed
when autoscaling is ready for release.
The packaging tests like to assert that files exist or do not exist. We
do this with assertFalse and assertTrue which leads to useless assertion
messages, especially when asserting a file does not exist, but it does
and it is a directory. This commit helps with this situation by adding
dedicated matchers.
There is some extraneous whitespace here, and every time I look at this
file my editor wants to make these changes and so my diffs end up having
this noise in it which I fight to exclude. This commit addresses this
issue by removing this extraneous whitespace.
* [ML] Add bwc serialization unit test scaffold (#51889)
Adds new `AbstractBWCSerializationTestCase` which provides easy scaffolding for BWC serialization unit tests.
These are no replacement for true BWC tests (which execute actual old code). These tests do provide some good coverage for the current code when serializing to/from old versions.
* removing unnecessary override for 7.series branch
* adding necessary import
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
in preparation for feature importance and split information gain, adding `number_samples` field to `TreeNode` definition.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Improve Painless compilation performance for nested conditionals (#52056)
This PR changes how conditional expression is handled in `PainlessParser`
in a way that avoids the need for backtracking, which led to exponential
compilation times in case of nested conditionals.
The test was added ensures that we can compile deeply nested conditionals.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Fix Map.of in Java8
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This change fixes flakiness in `CsvProcessorTests` where source field
can be the same as one of the headers used by tests which messes up
asserts when we check that field is not present after processor run.
Closes#50209
The related issue regarding aggregation queries where some literals
are also selected together with aggregate function has been fixed
with #49570. Add integration tests to verify the behavior.
Relates to: #41411
(cherry picked from commit 9f414a8d05c75e1a9f8250084f6dcd634d5d78d8)
The main purpose of this commit is to add a single autoscaling REST
endpoint skeleton, for the purpose of starting to build out the build
and testing infrastructure that will surround it. For example, rather
than commiting a fully-functioning autoscaling API, we introduce here
the skeleton so that we can start wiring up the build and testing
infrastructure, establish security roles/permissions, an so on. This
way, in a forthcoming PR that introduces actual functionality, that PR
will be smaller and have less distractions around that sort of
infrastructure.
Adds the ability to display docs on permanently unreleased branches,
such as `master` and `7.x`.
Also updates how the autoscaling and EQL docs are included.
Currently, these feature-flag docs would display on any unreleased
branches that contain the changes, such as 7.7.
If the configs are removed (by some horrific means), we should still allow tasks to be cleaned up easily.
Datafeeds and jobs with missing configs are now visible in their respective _stats calls and can be stopped/closed.
* Time parameter includes description
In option enumeration causing codegenerators to pick up the description
as a value to send.
* cat.shards missing ending quotes
(cherry picked from commit 1c3b341960e3b70555927bdbab325d26382f68b2)
Not all clients support this e.g if the java high level rest client were
to map this it would look like `client.cat().ml().api()` which hinders
discoverability.
(cherry picked from commit 21cdabf09dc8305ce2f5e3b6cb193f67137d8bdb)
SecurityIT.testGetUser creates a user for testing purposes, but did
not delete the user at the end of the test. This could leave the
cluster in an unexpected state for other tests.
This commit:
- Deletes the user at the end of `testGetUser`
- Adds the test-name as metadata to the users that are created in `SecurityIT`
so that their origin is clear if they do interfere with other tests
- Enables SecurityDocumentationIT.testGetUsers on the expectation that
the new cleanup step will resolve the unreliability of that test.
Relates: #48440
Co-authored-by: Tim Vernum <tim@adjective.org>
Backport of #51867.
Tweak the documentation around configuring the heap size when using
Docker, to state that:
- using `ES_JAVA_OPTS` is the preferred method
- Any `ES_JAVA_OPTS` overrides the defaults in `jvm.options`
- It's possible to bind-mount a custom `jvm.options`
This change ensures that the rewrite of the shard request is executed in the network thread or in the refresh listener when waiting for an active shard. This allows queries that rewrite to match_no_docs to bypass the search thread pool entirely even if the can_match phase was skipped (pre_filter_shard_size > number of shards). Coordinating nodes don't have the ability to create empty responses so this change also ensures that at least one shard creates a full empty response while the other can return null ones. This is needed since creating true empty responses on shards require to create concrete aggregators which would be too costly to build on a network thread. We should move this functionality to aggregation builders in a follow up but that would be a much bigger change.
This change is also important for #49601 since we want to add the ability to use the result of other shards to rewrite the request of subsequent ones. For instance if the first M shards have their top N computed, the top worst document in the global queue can be pass to subsequent shards that can then rewrite to match_no_docs if they can guarantee that they don't have any document better than the provided one.
QueryBuilders that throw exceptions on shards when building the Lucene query
returns the full serialization of the query builder in the exception message.
For large queries that fails to execute due to the max boolean clause, this means
that we keep a reference of these big messages for every shard that participate
in the request. In order to limit the memory needed to hold these query shard
exceptions in the coordinating node, this change removes the query builder
serialization from the shard exception. The query is known by the user so
there should be no need to repeat it on every shard exception. We could also
omit the entire stack trace for known bad request exception but it would deserve
a separate issue/pr.
Closes#51843Closes#48910
When clenaing a shard follow task after an index has been deleted, an
exception can occur submitting the complete persistent task
action. However, this exception message is not logged. This commit
addresses this by including the exception that led to the failure in the
log message.
* Add empty_value parameter to CSV processor
This change adds `empty_value` parameter to the CSV processor.
This value is used to fill empty fields. Fields will be skipped
if this parameter is ommited. This behavior is the same for both
quoted and unquoted fields.
* docs updated
* Fix compilation problem
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
When the `rare_terms` aggregation contained another aggregation it'd
break them. Most of the time. This happened because the process that it
uses to remove buckets that turn out not to be rare was incorrectly
merging results from multiple leaves. This'd cause array index out of
bounds issues. We didn't catch it in the test because the issue doesn't
happen on the very first bucket. And the tests generated data in such a
way that the first bucket always contained the rare terms. Randomizing
the order of the generated data fixed the test so it caught the issue.
Closes#51020
Costin Leau
Nhat Nguyen
Jason Tedor
Armin Braun
Benjamin Trent
Zachary Tong
Mark Vieira
Przemko Robakowski
debadair
Julie Tibshirani
Alpar Torok
Emanuele Sabellico
Tanguy Leroux
James Rodewig
Raidok
Albert Zaharovits
David Kyle
Andrei Stefan
Hendrik Muhs
Martijn Laarman
Ioannis Kakavas
Rory Hunter
Jim Ferenczi
Lisa Cawley
Nik Everett
Lee Hinman