We currently have two different native processes:
autodetect & normalizer. There are plans for introducing
a new process. All these share many things in common.
This commit refactors the processes to extend an
`AbstractNativeProcess` class that encapsulates those
commonalities with the purpose of reusing the code
for new processes in the future.
This commit adds a new ParentJoinAggregator that implements a join using global ordinals
in a way that can be reused by the `children` and the upcoming `parent` aggregation.
This new aggregator is a refactor of the existing ParentToChildrenAggregator with two main changes:
* It uses a dense bit array instead of a long array when the aggregation does not have any parent.
* It uses a single aggregator per bucket if it is nested under another aggregation.
For the latter case we use a `MultiBucketAggregatorWrapper` in the factory in order to ensure that each
instance of the aggregator handles a single bucket. This is more inlined with the strategy we use for other
aggregations like `terms` aggregation for instance since the number of buckets to handle should be low (thanks to the breadth_first strategy).
This change is also required for #34210 which adds the `parent` aggregation in the parent-join module.
Relates #34508
Index shard stats for the follower shard are fetched, when a shard follow task is started.
This is needed in order to bootstap the shard follow task with the follower global checkpoint.
Sometimes index shard stats are not available (e.g. during a restart) and
we fail now, while it is very likely that these stats will be available some time later.
Documents the new structured logfile format for auditing
that was introduced by #31931. Most changes herein
are for 6.x . In 7.0 the deprecated format is gone and a
follow-up PR is in order.
The pull request #34338 added strict deprecation mode to the REST tests
and adds the copy_settings param when testing the shrink of an index.
This parameter has been added in 6.4.0 and will be removed in 8.0, so
the test now needs to take care of the old cluster version when adding
the copy_settings param.
* Fix linelength suppressions in index.fielddata
* Some lines that were too long were dead code => Removed them and all code that became dead because of it
* Relates #34884
* Binding to `0` gives us free ports that are assigned sequentially by Linux making collisions much less likely compared to manually finding a free port in a range
* Closes#32208
testClusterFormsByScanningPorts is flaky: sometimes in CI it's not possible to
bind to any of the ports we need to in order for the port scanning to work.
This change removes this test, and #34809 describes a better way to test this
behaviour.
This change ensures the `message` field is always included
in the `field_stats` for the semi-structured text log file
file structure. Previously it was not, as it will almost
certainly contain all distinct values. However, for
consistency in the UI it's useful to include it.
For the assertions in the "testApiNamingConventions" test that check
the API contract, this change adds details of the method that was
being checked, and the intent of the assertion (the API contract)
After discussing on the team's FixItFriday, we concluded that
static final instance variables that are mutable should be lowercased.
Historically, DeprecationLogger was uppercased more frequently than lowercased.
ILM would return a resource-not-found exception when requesting policies
while the IndexLifecycleMetaData is not initialized. The behavior here
should not be as extreme since it is not the user's fault.
This commit changes the behavior so that it succeeds and returns no policies
when no policy names are explicitely specified, otherwise keep the same behavior
of throwing an exception
Adds checks for misbehaving parsers. The checks aren't perfect at all but
they are simple and fast enough that we can do them all the time so
they'll catch most badly behaving parsers.
Closes#34351
This is a forward port of a change made to clean up backwards
compatibility for the rollup cleanups. It makes the version of each node
available very early on in test execution. The 6.x version of the change
used those versions to control the cleanup backwards compatibility but
that isn't needed in this branch. But having the versions around *is*
useful. So this makes them available.
Closes#34629
Previously, for some queries the validation for ORDER BY
fields didn't kick in since a HAVING close or an ORDER BY
with scalar function would add `Filter` and `Project` plans
between the `OrderBy` and the `Aggregate`.
Now the LogicalPlan tree beneath `OrderBy` is traversed and
the ORDER BY fields are properly verified.
Fixes: #34590
This is related to #30876. The AbstractSimpleTransportTestCase initiates
many tcp connections. There are normally over 1,000 connections in
TIME_WAIT at the end of the test. This is because every test opens at
least two different transports that connect to each other with 13
channel connection profiles. This commit modifies the default
connection profile used by this test to 6. One connection for each
type, except for REG which gets 2 connections.
In an effort to encapsulate different clients, the request converters
are being shuffled around. This splits the Index Lifecycle Management
request converters.
* Check self references in metric agg after last doc collection (#33593)
* Revert 0aff5a30c5dbad9f476be14f34b81e2d1991bb0f (#33593)
* Check self refs in metric agg only once in post collection hook (#33593)
* Remove unnecessary mocking (#33593)
This change adds a section about the global search setting
`indices.query.bool.max_clause_count` that limits the number of boolean clauses
allowed in a Lucene BooleanQuery.
Closes#19858
The testCharsBeginsWith test has a check that a random prefix of length
2 is not the prefix of a char[]. However, there is no check that the
char[] is not randomly generated with the same two characters as the
prefix. This change ensures that the char[] does not begin with the
prefix.
Closes#34765
With the introduction of _ilm/stop and _ilm/start APIs, the
use cases where one would only target a select group
of indices to start/stop has been reduced. Since there is no
strong use-case for skipping specific indices, it is best to
remove this functionality and only adding if later desired, with the
hopes of keeping things more simple.
through randomization, there is a chance that the mutateInstance
for PolicyStatsTests does not actually mutate the original object.
This PR aims to fix this
- we already require Java 11 to build, yet we target the minimum
supported version in build-tools ( currently 8 )
- this is because we have some checks that are executed in a new JVM
which could be running the minimum version.
- For everything else it would be nice to be able to use new features,
like the new process API.
With this change, we selectively compile the few classes that need an
older target version and move everything over to Java 10.
Unfortunately the current Gradle version does not support 11 as a target
version yet.
* NETWORKING: Add SSL Handler before other Handlers
* The only way to run into the issue in #33998 is for `Netty4MessageChannelHandler`
to be in the pipeline while the SslHandler is not. Adding the SslHandler before any
other handlers should ensure correct ordering here even when we handle upstream events
in our own thread pool
* Ensure that channels that were closed concurrently don't trip the assertion
* Closes#33998
The contains syntax was added in #30874 but the skips were not properly
put in place.
The java runner has the feature so the tests will run as part of the
build, but language clients will be able to support it at their own
pace.
These were disabled because ingest attachement was not playing well with
JDK 11. We have addressed that by upgrading the Tika dependency, yet
forgot to reenable these. This commit enables these tests again.
Lee Hinman
Dimitris Athanasiou
Jim Ferenczi
Gordon Brown
markharwood
Martijn van Groningen
Albert Zaharovits
Tanguy Leroux
Armin Braun
David Turner
David Roberts
Tim Vernum
Tal Levy
Nik Everett
Senthil
Stéphane Campinas
Marios Trivyzas
Tim Brooks
Lee Hinman
Lisa Cawley
Christophe Bismuth
Christoph Büscher
Jay Modi
王振荟
lipsill
Colin Goodheart-Smithe
Costin Leau
Andrei Stefan
Nhat Nguyen
Alpar Torok
Jason Tedor