This commits removes BytesValues/LongValues/DoubleValues/... and tries to use
Lucene's APIs such as NumericDocValues or RandomAccessOrds instead whenever
possible.
The next step would be to take advantage of the fact that APIs are the same in
Lucene and Elasticsearch in order to remove our custom comparators and use
Lucene's.
There are a few side-effects to this change:
- GeoDistanceComparator has been removed, DoubleValuesComparator is used instead
on top of dynamically computed values (was easier than migrating
GeoDistanceComparator).
- SortedNumericDocValues doesn't guarantee uniqueness so long/double terms
aggregators have been updated to make sure a document cannot fall twice in
the same bucket.
- Sorting by maximum value of a field or running a `max` aggregation is
potentially significantly faster thanks to the random-access API.
Our aggs and p/c aggregations benchmarks don't report differences with this
change on uninverted field data. However the fact that doc values don't need
to be wrapped anymore seems to help a lot. For example
TermsAggregationSearchBenchmark reports ~30% faster terms aggregations on doc
values on string fields with this change, which are now only ~18% slower than
uninverted field data although stored on disk.
Close#6908
The assertion on binary equality for streamable serialization
sometimes fails due to the usage of identify hashmaps inside
the InternalSearchHits serialization. This only happens if
the number of shards the result set is composed of is very high.
This commit makes the serialziation deterministic and removes
the need to serialize the ordinal due to in-order serialization.
This commit adds the ability to run bwc tests during the release
process to ensure the current release is backwards compatible with
the latest installed previous version. Installed means available
in the configured bwc test path.
Closes#6953
The change introduced in #6949 (do not serialize the cluster state) also means master now responds with an empty response rather then a JoinResponse. However, sendJoinRequestBlocking still expected a JoinRequest.
At the moment we serialize the cluster state in JoinResponse and ValidateJoinRequest. However this state is not used anywhere and can be removed to save on network overhead
Closes#6949
This test deletes and updates using upserts documents over several threads in a
tight loop. It counts the number of responses and verifies that the versions at
the end are correct.
today if a snapshot is corrupted the restore operation
never terminates. Yet, if the snapshot is corrupted there
is no way to restore it anyway. If such a snapshot is restored
today the only way to cancle it is to delete the entire index which
might cause dataloss. This commit also fixes an issue in InternalEngine
where a deadlock can occur if a corruption is detected during flush
since the InternalEngine#snapshotIndex aqcuires a topLevel read lock
which prevents closing the engine.
Closes#6938
The `index.fail_on_corruption` was not updateable via the index settings
API. This commit also fixed the setting prefix to be consistent with other
setting on the engine. Yet, this feature is unreleased so this won't break anything.
Closes#6941
Small refactorings to make the MessageChannelHandler more extensible.
Also allowed access to the different netty pipelines
This is the fix after the first version had problems with the HTTP
transport due to wrong reusing channel handlers, which is the reason
why tests failed.
Relates #6889Closes#6915
It's now possible to inject action filters from plugins via `ActionModule#registerFilter` through the following code:
```
public void onModule(ActionModule actionModule) {
actionModule.registerFilter(MyFilter.class);
}
```
Also made `TransportAction#execute` methods final to enforce the execution of the filter chain. By default the chain is empty though.
Note that the action filter chain is executed right after the request validation, as the filters might rely on a valid request to do their work.
Closes#6921
Today when we start a `TransportClient` we use the given transport
addresses and create a `DiscoveryNode` from it without knowing the
actual nodes version. We just use the `Version.CURRENT` which is an
upper bound. Yet, the other node might be a version less than the
currently running and serialisation of the nodes info might break. We
should rather use a lower bound here which is the version of the first
release with the same major version as `Version.CURRENT` since this is
what we officially support.
This commit moves to use the minimum major version or an RC / Snapshot
if the current version is a snapshot.
Closes#6894
Adds the ability to the Term Vector API to generate term vectors for some
chosen fields, even though they haven't been explicitely stored in the index.
Relates to #5184Closes#6567
The bulk processor tries to acquire all leases for the semaphore to wait
for all pending requests. Yet, we should release them afterwards again to
ensure we don't ever deadlock if there is a bug in the processor.
This commit also adds a testcase for this method
Brian Murphy
Adrien Grand
Nik Everett
Simon Willnauer
Clinton Gormley
Boaz Leskes
Shay Banon
Peter Johnson @insertcoffee
Martijn van Groningen
Alexander Reelsen
javanna
Mahesh Paolini-Subramanya
Medcl
Alex Ksikes
Peter Kim
matt-preston