Tests are more reproducible if documents are indexed in random order
with random flushes/refreshes etc. in order to catch corner cases
where doc Ids are used for tie-breaking and tests rely on a certain
order.
IndexGatewayTests rely on that all nodes have the same type of
storage to pass consistently. Random configs should be the same for all
nodes in the cluster.
If the score is equal in Lucene the order of the result depends on the
actual global doc ID such that due to background merges or concurrency
these test can return different result set orders.
Allow to update / merge in the dynamic flag on object or the root object mapper, meaning that it can be changed dynamically using the update mapping API.
closes#3384
For better integration with the Lucene Test Framework and the
availabilty of RandomizedRunner / Randommized Testing this commit
moves over from TestNG to JUnit.
This commit also moves relevant places over to RandomzedRunner for
reproduceability and removes copied classes from the Lucene Test
Framework.
if the index already exists, trace log it, since it might happen as a result of multiple index requests at the same time creating the index, all other ones, should be debug and not warn in the same spirit of other APIs
try and bulk as much as possible refresh and update mapping events, so they will all be processed at a single go, resulting in less cluster change events, and also reduce the load of multiple changes to the same index
also, change the prio for those to HIGH, since we want URGENT ones (like create index, delete index) to execute
Have a dedicated thread pool for explicit optimize calls (shard level optimize operations). By default, the size should be 1 to work the same with how things work currently allowing for only 1 shard level optimize on a node.
The change allows to see the optimize thread pool stats now, and potentially increase the thread pool size for beefy machines.
closes#3366
- master actions many times end up being executed on the cluster service, so there is no need to block them on the management thread pool to wait for a response, this remove the load on the management thread pool, and also simplifies the code implementing it
- cluster service state update exception handling was improved to include a callback when a failure happens during state update execution, this makes sure that we catch all relevant exceptions and invoke the callback, as well as simplify the code of cluster state update tasks, as they can throw failures from within the execute method and then handle them properly
FastVectorHighlighter fails at highlighting some complex queries such as
multi phrase queries which have two terms at the same position. This can be
easily triggered by running a `match_phrase` query with an analyzer which
outputs synonyms such as SynonymFilter or WordDelimiterFilter.
Close#3357
Currently, the master node might be processing too many cluster state events, and then be blocked on waiting for its respective even to be processed. We can use the new cluster state update timeout support to use the master_timeout value and respect it.
closes#3365
Moved alias action validation to IndicesAliasesRequest, so that Java api and RestIndexPutAliasAction can benefit from it too.
Added check in MetaDataIndexAliasesService too.
Closes#3363
Today, we have a low/med/high channel groups in our transport layer. High is used to publish cluster state and send ping requests. Sometimes, the overhead of publishing large cluster states can interfere with ping requests.
Introduce a new, dedicated ping channel (with size 1) to have a channel that only handles ping requests.
closes#3362
Master node cluster state events resulting in zen discovery (node gets added, removed, for example) should be processed with priority URGENT as its always better to process them as fast as possible, and not let other events get in the way.
closes#3361
Shay Banon
Luca Cavanna
Martijn van Groningen
Simon Willnauer
Lukas Vlcek
Boaz Leskes
Adrien Grand