This commit adds the Log4j to SLF4J binding JAR to the repository-hdfs
plugin so that SLF4J can detect Log4j at runtime and therefore use the
server Log4j implementation for logging (and the usual Elasticsearch
APIs can be used for setting logging levels).
Relates #26514
The `_type` and `types` version of the current `type` parameter have been
deprecated since 5.0. We can remove support for them in 7.0 and also in 6.x and
6.0.
We currently have a weird relationship between Transport,
TransportService, and TransportServiceAdaptor. At some point I think
that we would like to collapse these all into one concept as we only
support TCP transports.
This commit moves in that direction by eliminating the adaptor and just
passing the transport service to the transport.
Improve testing around the GeoDistanceSortBuilder#build method, adding checks for correct
transfers of the sort order, mode, nested sorts and points validation and coercion.
Also changing the behaviour around the nested_path, nested_filter vs. nested parameter in
a similar way as in #26490 and deprecating the setters/getters for the old syntax.
Relates to #17286
Currently we allow both "old" and "new" way of setting nested sorts on the
FieldSortBuilder at the same time. This should throw an error, instead the user
should choose one of the two possible options.
Also adding testing for the now deprecated nestedPath/nestedFilter parameters,
inlcuding checks that they emmit warnings on parsing and that the new
NestetedSortBuilder overwrites the deprecated parameters when building the
SortField.
Relates to #17286
The `index.percolator.map_unmapped_fields_as_text` is a more better name, because unmapped fields are mapped to a text field with default settings
and string is no longer a field type (it is either keyword or text).
The definition of development vs. production mode has evolved slightly
over time (with the introduction of single-node) discovery. This commit
clarifies the documentation to better account for this adjustment.
Relates #26460
Adding a check to QueryStringQueryBuilderTests that checks the override
behaviour of `quote_analyzer`, also adding documentation explaining the use of
this parameter in `query_string` query.
Closes#25417
We are getting the default Object#toString implementation here, we need
more than this. This commit instead formats the primary response to JSON
so we can see into its soul.
There is a bug in Log4j on JDK 9 for walking the stack to find where a
log line is coming from. This bug is impacting some of our testing, so
this commit marks these tests as skippable only on JDK 9 until the bug
is fixed upstream.
Relates #26467
The new NestedSortBuilder currently is only tested via its use in the other
SortBuilder implementations it can be used in. This adds its own simple unit
test class that at first checks our usual fromXContent parsing, serialization
and hashCode/equals checks. It also adds tests for cases where NestedSortBuilder
is nested in itself and reuses the code for creating randomized instances in the
other SortBuilder tests.
In addition to the tests, this changes the `path` parameter in NestedSortBuilder
to be mandatory and removes the `read` method since it is not really needed.
The current script service has a script compilation limit for a one
minute window. This is set to a small default value of 15. Instead of
increasing that default value, this commit introduces a new setting
that allows to configure a rate per time unit, so that the script service can deal with bursts better.
The new setting is named `script.max_compilations_rate`,
requires a nonnegative number and a positive time value.
The default is `75/5m`, which is equivalent to the existing 15 per minute.
In some cases a request can already be aborted and retried. This means
the condition that aborting a request should only happen when an item
has not been processed yet is too strict. This commit allows for a
double abort. If we attempt to abort an operation that was previously
processed but not aborted, we treat that as a hard failure.
Relates #26434
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction.
This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
Currently we don't have much unit testing about the SortField that is created then
calling the SortBuilders `build` method. Most of this is covered by integration tests
somewhere but it would be good to have some basic checks in FieldSortBuilderTest
as well.
This adds testing for the sort order, mode, missing values and checks that `nested`
gets set in the XFieldComparatorSource when `nestedPath` and `nestedFilter` are
set on the builder.
Relates to #17286
* Implement adaptive replica selection
This implements the selection algorithm described in the C3 paper for
determining which copy of the data a query should be routed to.
By using the service time EWMA, response time EWMA, and queue size EWMA we
calculate the score of a node by piggybacking these metrics with each search
request.
Since Elasticsearch lacks the "broadcast to every copy" behavior that Cassandra
has (as mentioned in the C3 paper) to update metrics after a node has been
highly weighted, this implementation adjusts a node's response stats using the
average of the its own and the "best" node's metrics. This is so that a long GC
or other activity that may cause a node's rank to increase dramatically does not
permanently keep a node from having requests routed to it, instead it will
eventually lower its score back to the realm where it is a potential candidate
for new queries.
This feature is off by default and can be turned on with the dynamic setting
`cluster.routing.use_adaptive_replica_selection`.
Relates to #24915, however instead of `b=3` I used `b=4` (after benchmarking)
* Randomly use adaptive replica selection for internal test cluster
* Use an action name *prefix* for retrieving pending requests
* Add unit test for replica selection
* don't use adaptive replica selection in SearchPreferenceIT
* Track client connections in a SearchTransportService instead of TransportService
* Bind `entry` pieces in local variables
* Add javadoc link to C3 paper and javadocs for stat adjustments
* Bind entry's key and value to local variables
* Remove unneeded actionNamePrefix parameter
* Use conns.longValue() instead of cached Long
* Add comments about removing entries from the map
* Pull out bindings for `entry` in IndexShardRoutingTable
* Use .compareTo instead of manually comparing
* add assert for connections not being null and gte to 1
* Copy map for pending search connections instead of "live" map
* Increase the number of pending search requests used for calculating rank when chosen
When a node gets chosen, this increases the number of search counts for the
winning node so that it will not be as likely to be chosen again for
non-concurrent search requests.
* Remove unused HashMap import
* Rename rank -> rankShardsAndUpdateStats
* Rename rankedActiveInitializingShardsIt -> activeInitializingShardsRankedIt
* Instead of precalculating winning node, use "winning" shard from ranked list
* Sort null ranked nodes before nodes that have a rank
The output when building bwc versions is currently verbose, with git
warnings from doing git checkout of a hash. This commit changes this to
print the useful info before and after checking out. Note that due to
using LoggedExec, if the git task exits non-zero, the entire output will
still be dumped.
When changing how the config path is configured (from a command-line
flag to an environment variable) we had to add BWC code in the build so
that we could form clusters with 5.x nodes in them. Now that this branch
has moved to 7.x, we no longer need to be BWC with 5.x for starting
nodes. This commit removes this dead BWC code.
Relates #26446
When starting a node in standalone tests, we sometimes use a wrapper
script as opposed to starting Elasticsearch directly (this is used for
backgrounding). On Windows, the path to this wrapper script can be
exceptionally long, exceeding the length of a batch script that cmd.exe
will invoke without whining. This commit replaces using the full path
name for this wrapper script by the short name for the wrapper script.
Additionally, the data, configuration, and jvm.options paths can also
end up being too long so we shorten those too. Care has to be taken with
the data directory because we usually rely on the node creating it on
startup but doing that will not be compatible with getting the short
name as that requires the directory already existing. Therefore, we
create that directory on-demand immediately before actually resolving
the short name.
Relates #26444
Multi-level Nested Sort with Filters
Allow multiple levels of nested sorting where each level can have it's own filter.
Backward compatible with previous single-level nested sort.
This commit adds writing build metadata to the `check` command for each
bwc project. This ensures the files will be written if doing a general
`gradle check`, which is what CI intake jobs do. In later jobs like
bwcTest, the extra bwc-release-snapshot info is needed.
Note this commit also has a little cleanup of the output for the bwc
checkout, as it was plastering a git warning, instead of the real info
we care about (the refspec and commit that were used).
* Moves deferring code into its own subclass
This change moves the code that deals with deferring collection to a subclass of BucketAggregator called DeferringBucketAggregator. This means that the code in AggregatorBase is simplified and also means that the code for deferring colleciton is in one place and easier to maintain.
* Makes SIngleBucketAggregator an interface
This is so aggregators that extend BucketsAggregator directly and those that extend DeferringBucketAggregator can be a single bucket aggregator
* review comments
* More review comments
Martijn van Groningen
Jason Tedor
Christoph Büscher
Tim Brooks
marcocova
Boaz Leskes
Alexander Reelsen
shaulzorea
Tal Levy
Lee Hinman
Michael Basnight
Ryan Ernst
Tim Vernum
Adrien Grand
Lee Hinman
Matt Weber
Tanguy Leroux
Colin Goodheart-Smithe