Currently we sometimes see test failures that fail because not all replicase are
`searchable` which means they are not started yet or still recovering. Yet, the usual
situation is where two nodes have the same clusterstate but the one that acts as
the search target has not yet processed that clusterstate. The requester sees the
shard as started but it's not mark as such on the target node. For now the #ensureSearchable()
just delegates to #ensureYellow() to make sure the cluster is not red. In the future if we have
the possibilty to recover from situations like this in the search logic we can easily test
this by making the impl a no-op. Note: this problem only occurs if you have low number of docs
and the indexing is really quick such that first request are exectued but shards are not
fully `started`
The postings hl now uses a searcher that only encapsulate the view of segment the document being highlighted is in,
this should be better than using the top level engine searcher.
Closes#4385
Adding functionality to call cluster().transportClient() in tests in order
to get an arbitrary TransportClient object back, independently if the
transport client ratio in returning the normal clients is configured.
Also made sure, that if the normal client is already a transport client
(or a node client) we do not generate another one.
When searching with a query containing query_strings inside a bool query, the specified _name is randomly missing from the results due to caching.
Closes#4361.
Closes#4371.
Instead of processing all the bulk of update mappings we have per index/node, we can only update the last ordered one out of those (cause they are incremented on the node/index level). This will improve the processing time of an index that have large updates of mappings.
closes#4373
If a phrase query is wrapped in a filtered query due to type filtering
slop was not applied correctly. Also if the default field required a
type filter the filter was not applied.
Closes#4356
_all boosting used to rely on the fact that the TokenStream doesn't eagerly
consume the input java.io.Reader. This fixes the issue by using binary search
in order to find the right boost given a token's start offset.
Close#4315
This contribution is based on the feedback given in issue #4254 and
issue #4255, and should clear things up, when suggestions are being
removed and not displayed anymore after deletion of data.
When a node processed an index request, which caused it to update its own mapping, then it sends that mapping to the master. While the master process it, that node receives a state that includes an older version of the mapping. Now, there is a conflict, its not bad (i.e. the cluster state will eventually have the correct mapping), but we send for a refresh just in case form that node to the master.
With a system that has extreme cases of updates and frequent mapping changes, it might make sense to disable this feature. The indices.cluster.send_refresh_mapping setting can be introduced to support that (note, this setting need to be set on the data nodes)
Note, sending refresh mapping is more important when the reverse happens, and for some reason, the mapping in the master is ahead, or in conflict, with the actual parsing of it in the actual node the index exists on. In this case, the refresh mapping will result in warning being logged on the master node.
closes#4342
We have the situation that some tests fail since they don't handle
EsRejectedExecutionException which gets thrown when a node shuts
down. That is ok to ignore this exception and not fail.
We also suffer from OOMs that can't create native threads but don't
get threaddumps for those failures. This patch prints the thread
stacks once we catch a OOM which can' create native threads.
The newly added afterIndexShardPostRecovery method to InternalIndicesLifecycle, that the percolator now uses to trigger the loading of the registered queries will make sure that a shard doesn't go to started state before the queries have been loaded.
The percolate api (like other apis) will retry the execution on a different shard copy if a shard isn't in a started state preventing empty results if there registered queries to be loaded. Percolator tests fail sometimes for this reason.
The Fast Vector Highlighter can combine matches on multiple fields to
highlight a single field using `matched_fields`. This is most
intuitive for multifields that analyze the same string in different
ways. Example:
{
"query": {
"query_string": {
"query": "content.plain:running scissors",
"fields": ["content"]
}
},
"highlight": {
"order": "score",
"fields": {
"content": {
"matched_fields": ["content", "content.plain"],
"type" : "fvh"
}
}
}
}
Closes#3750
Simon Willnauer
Martijn van Groningen
Alexander Reelsen
David Pilato
Lee Hinman
Shay Banon
uboness
Boaz Leskes
Luca Cavanna
Adrien Grand
Andrew Raines
Markus Fischer
Nik Everett