We should never write a circular reference exception as we will fail a
node with StackOverflowError. However, we have one in #53589.
I tried but failed to find its location. With this commit, we will avoid
StackOverflowError in production and detect circular exceptions in
tests.
Closes#53589
We recently cleaned up the use of the word "metadata" across the
codebase. A few additional uses have trickled in, likely from
in-progress work. This commit cleans up these last few instances.
Relates #54519
The autoscaling REST tests use policies named "hot" in their test
cases. Instead, this commit changes the name of these policies to
"my_autoscaling_policy".
* Use V2 index templates during index creation
This commit changes our index creation code to use (and favor!) V2 index templates during index
creation. The creation precedence goes like so, in order of precedence:
- Existing source `IndexMetadata` - for example, when recovering from a peer or a shrink/split/clone
where index templates should not be applied
- A matching V2 index template, if one is found
- When a V2 template is found, all component templates (in the `composed_of` field) are applied
in the order that they appear, with the index template having the 2nd highest precedence (the
create index request always has the top priority when it comes to index settings)
- All matching V1 templates (the old style)
This also adds index template validation when `PUT`-ing a new v2 index template (because this was
required) and ensures that all index and component templates specify *no* top-level mapping type (it
is automatically added when the template is added to the cluster state).
This does not yet implement fine-grained component template merging of mappings, where we favor
merging only a single field's configuration, that will be done in subsequent work.
This also keeps the existing hidden index behavior present for v1 templates, where a hidden index
will match v2 index templates unless they are global (`*`) templates.
Relates to #53101
Change how we format exceptions to only wrap them as necessary. While
the config's overall philosophy is to put items one-per-line when
wrapping, in practice this is a little cumbersome for exception lists.
Some field name constants were not updaten when we moved from "string" to "text"
and "keyword" fields. Renaming them makes it easier and faster to know which
field type is used in test subclassing this base test case.
The test results are affected by the off-by-one error that is
fixed by https://github.com/elastic/ml-cpp/pull/1122
This test can be unmuted once that fix is merged and has been
built into ml-cpp snapshots.
Force stopping a failed job used to work but it
now puts the job in `stopping` state and hangs.
In addition, force stopping a `stopping` job is
not handled.
This commit addresses those issues with force
stopping data frame analytics. It inlines the
approach with that followed for anomaly detection
jobs.
Backport of #54650
* Document VarcharLimit and EarlyExecution params
Add the documentation for the newly added VarcharLimit and
EarlyExecution DSN attributes.
* Remove obsolete VersionChecking param
This param had been removed already along the #53082 work.
* Update docs/reference/sql/endpoints/odbc/configuration.asciidoc
fix typo
Co-Authored-By: Stuart Cam <stuart@codebrain.co.uk>
* Update docs/reference/sql/endpoints/odbc/configuration.asciidoc
fix typo
Co-Authored-By: Stuart Cam <stuart@codebrain.co.uk>
(cherry picked from commit f38761631a12b38f7f075635f7ac61dc96656cd7)
The test had errors around time units that have different length - think
leap years or months that aren't 30 days. This fixes those errors. In
the proces I've changed a bunch of things to debug the problem:
* Replace `currentTimeMillis` with a random time. Now the test fails
randomly! Wonderful. Much better than on random days of the month.
* Generate buckets "closer together" to test random reduction. Without
this we were super frequently getting stuck in the "year of century"
rounding because *some* of the of the buckets we built were far apart.
This generates a much greater variety of tests.
* Implement `toString` on `RoundingInfo` so I can debug without going
crazy.
* Switch keys in the bucket assertions from epoch millis to `Instant`s
so we can read the failures.
Closes#54540Closes#39497
Backport of #54576.
This commit is part of issue #40366 to remove disabled Xlint warnings
from gradle files. Remove the Xlint exclusions from the following files:
- x-pack/plugin/rollup/build.gradle
- x-pack/plugin/monitoring/build.gradle
- x-pack/qa/rolling-upgrade-basic/build.gradle
Add type parameters to parameterized types. Add wildcard-type parameters
or bounded wildcard-type parameters. Suppress `unchecked` and `rawtypes`
warnings at method level.
from `*_flag_registered` to `#_feature_enabled`.
This previous name indicated that a flag was registered,
whilst the feature flag actually controls whether a
feature is enabled.
This commit workarounds a bug in the JDK 14 compiler. It is choking on a
method reference, so we substitute a lambda expression instead. The JDK
bug ID is 9064309.
In #33933 we disallowed changing the `enabled` parameter in object mappings.
However, the fix didn't cover the root object mapper. This PR adjusts the change
to also include the root mapper and clarifies the error message.
This adds training_percent parameter to the analytics process for Classification and Regression. This parameter is then used to give more accurate memory estimations.
See native side pr: elastic/ml-cpp#1111
Removes pipeline aggregations from the aggregation result tree as they
are no longer used. This stops us from building the pipeline aggregators
at all on data nodes except for backwards compatibility serialization.
This will save a tiny bit of space in the aggregation tree which is
lovely, but the biggest benefit is that it is a step towards simplifying
pipeline aggregators.
This only does about half of the work to remove the pipeline aggs from
the tree. Removing all of it would, well, double the size of the change
and make it harder to review.
I derped out on a last minute bug fix when backporting #54282 and it
only causes the tests to fail about half the time. So I didn't catch
it until after merging. Great! This fixes it.
The `indices.recovery.max_bytes_per_sec` recovery bandwidth limit can differ
between nodes if it is not set dynamically, but today this is not obvious. This
commit adds a paragraph to its documentation clarifying how to set different
bandwidth limits on each node.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
* [ML] add new inference_config field to trained model config (#54421)
A new field called `inference_config` is now added to the trained model config object. This new field allows for default inference settings from analytics or some external model builder.
The inference processor can still override whatever is set as the default in the trained model config.
* fixing for backport
This commit addresses an issue with the autoscaling feature flag not
being registered in release builds of the internal cluster tests. This
commit addresses this by enabling the system property that is needed,
but only in release builds.
* [ML] prefer secondary authorization header for data[feed|frame] authz (#54121)
Secondary authorization headers are to be used to facilitate Kibana spaces support + ML jobs/datafeeds.
Now on PUT/Update/Preview datafeed, and PUT data frame analytics the secondary authorization is preferred over the primary (if provided).
closes https://github.com/elastic/elasticsearch/issues/53801
* fixing for backport
- Consolidates HDR/TDigest factories into a single factory
- Consolidates most HDR/TDigest builder into an abstract builder
- Deprecates method(), compression(), numSigFig() in favor of a new
unified PercentileConfig object
- Disallows setting algo options that don't apply to current algo
The unified config method carries both the method and algo-specific
setting. This provides a mechanism to reject settings that apply
to the wrong algorithm. For BWC the old methods are retained
but marked as deprecated, and can be removed in future versions.
Co-authored-by: Mark Tozzi <mark.tozzi@gmail.com>
Co-authored-by: Mark Tozzi <mark.tozzi@gmail.com>
This fixes pipeline aggregations used in cross cluster search from an older
version of Elasticsearch to a newer version of Elasticsearch. I broke
this in #53730 when I was too aggressive in shutting off serialization
of pipeline aggs. In particular, this comes up when the coordinating
node is pre-7.8.0 and the gateway node is on or after 7.8.0.
The fix is another step down the line to remove pipeline aggregators
from the aggregation tree. Sort of. It create a new
`List<PipelineAggregator>` member in `InternalAggregation` *but* it is
only used for bwc serialization and it is fed by the mechanism
established in #53730 to read the pipelines from the
These methods are not needed, we were only following a pattern in the
rest of the codebase, but it's legacy from the HLRC sharing
request/response objects with the server.
When one of ML's normalize processes fails to connect to the JVM
quickly enough and another normalize process for the same job
starts shortly afterwards it is possible that their named pipes
can get mixed up.
This change avoids the risk of that by adding an incrementing
counter value into the named pipe names used for normalize
processes.
Backport of #54636
* [ML] add num_matches and preferred_to_categories to category defintion objects (#54214)
This adds two new fields to category definitions.
- `num_matches` indicating how many documents have been seen by this category
- `preferred_to_categories` indicating which other categories this particular category supersedes when messages are categorized.
These fields are only guaranteed to be up to date after a `_flush` or `_close`
native change: https://github.com/elastic/ml-cpp/pull/1062
* adjusting for backport
Nhat Nguyen
Jason Tedor
Lee Hinman
James Baiera
Lisa Cawley
Ross Wolf
Rory Hunter
Ioannis Kakavas
Dan Hermann
Christoph Büscher
David Roberts
Dimitris Athanasiou
Bogdan Pintea
Nik Everett
Maria Ralli
markharwood
Martijn van Groningen
István Zoltán Szabó
Mark Vieira
Julie Tibshirani
Benjamin Trent
David Turner
Zachary Tong
qiye