Previously, collate feature would be executed on all shards of an index using the client,
this leads to a deadlock when concurrent collate requests are run from the _search API,
due to the fact that both the external request and internal collate requests use the
same search threadpool.
As phrase suggestions are generated from the terms of the local shard, in most cases the
generated suggestion, which does not yield a hit for the collate query on the local shard
would not yield a hit for collate query on non-local shards.
Instead of using the client for collating suggestions, collate query is executed against
the ContextIndexSearcher. This PR removes the ability to specify a preference for a collate
query, as the collate query is only run on the local shard.
closes#9377
This adds back the ability to disable _source, as well as set includes
and excludes. However, it also restricts these settings to not be
updateable. enabled was actually already not modifiable, but no
conflict was previously given if an attempt was made to change it.
This also adds a check that can be made on the source mapper to
know if the the source is "complete" and can be used for
purposes other than returning in search or get requests. There is
one example use here in highlighting, but more need to be added
in a follow up issue (eg in the update API).
closes#11116
Add methods to operate on multi-valued fields in the expressions language.
Note that users will still not be able to access individual values
within a multi-valued field.
The following methods will be included:
* min
* max
* avg
* median
* count
* sum
Additionally, changes have been made to MultiValueMode to support the
new median method.
closes#11105
- Renamed TranslogSnapshot to MultiSnapshot
- moved legacy logic for trucation into LegacyTranslogReaderBase
- made several methods private and pkg private where applicable
- renamed arguments for consistency
Today we barf if repositories are unregistered with a `*` pattern. This
happens on almost every test and adds weird log messages. I dont' think
we should barf in that case.
Closes#11113
We have some builders, specifically query builders, `SearchSourceBuilder`, `QuerySourceBuilder` and `SuggestBuilder`, that implement `ToXContent` and also allow to build their content as bytes by simply creating a `BytesReference` that holds their json (or yaml etc.) content (`buildAsBytes` methods). They can also print out their content through `toString`. Made sure that those common methods are in one single place and reused where needed.
Also, merged `QueryBuilder` and `BaseQueryBuilder` and made `QueryBuilder` an abstract class instead of an interface.
Closes#11063
We parse the rewrite field in FuzzyQueryParser but we don't allow to set it via FuzzyQueryBuilder for our java api users. Added missing field and setter.
Closes#11130Closes#11139
The esoteric classifier contains in particular maps that take bytes or doubles
as keys. In the byte case, we can just use integer, and in the double case we
can use their long bits instead.
Today we are almost intentionally corrupt the translog if we loose
a node due to powerloss or similary disasters. In the translog reading
code we simply read until we hit an EOF exception ignoring the rest of the
translog file once hit. There is no information stored how many records
we are expecting or what the last written offset was.
This commit restructures the translog to add checkpoints that are written
with every sync operation recording the number of synced operations as well
as the last synced offset. These checkpoints are also used to identify the actual
transaction log file to open instead of relying on directory traversal.
This change adds a significant amount of additional checks and pickyness to the translog
code. For instance is the translog now associated with a specific engine via a UUID that is
written to each translog file as part of it's header. If an engine opens a translog file it
was not associated with the operation will fail.
Closes to #10933
Relates to #11011
These clauses filter the document space without affecting scoring and map to
Lucene's BooleanClause.Occur.FILTER. The `filtered` query is now deprecated and
```json
{
"filtered": {
"query": { //query },
"filter": { //filter }
}
}
```
should be replaced with
```json
{
"bool": {
"must": { //query },
"filter": { //filter }
}
}
```
In case FieldNameAnalyzer does not find an explicit analyzer for a given
field name, it returns the default analyzer. This behaviour can hide bugs
where the analyzer fails to be propagated to FieldNameAnalyzer or an
analyzer is requested for a field which is not mapped.
We are using a a VLong to serialize the PercolateResponse#tookInMillis. This
can due to several `System.currentTimeMillis()` implemenation details be negative.
We should prevent the negavite value for being serialized as a VLong and make sure
we use a valid value for this in the first place
Closes#11138
A few meta fields can currently be set within a document's source.
However, the recommended way to set meta fields like this is through
the api, and setting within the document can be a performance trap
(e.g. needing to find _id in order to route the document).
This change removes the ability to set meta fields within
a document source for 2.0+ indexes.
closes#11051closes#11074
Several plugins (e.g. elasticsearch-cloud-aws, elasticsearch-cloud-azure, elasticsearch-cloud-gce)
have integration tests that run with actual credentials to a remote service, so test runs
need access to this file.
These all require the tester (or jenkins) to supply the file with -Dtests.config.
In `NodeEnvironment.deleteShardDirectoryUnderLock`, we will now attempt
to acquire, then release, the `write.lock` file for the Lucene index in
question to ensure that no other `IndexWriter` has the directory open
before deleting the data.
Note that the `write.lock` file must be released before the actual
deletion in order to allow the directory to be deleted.
Fixes#11097
If the user has set a shard_min_doc_count setting then avoid looking up background frequencies if the term fails to meet the foreground threshold on a shard.
Closes#11093
Areek Zillur
Simon Willnauer
Ryan Ernst
Jack Conradson
Robert Muir
Robert Muir
Adrien Grand
javanna
Dawid Weiss
Shay Banon
Igor Motov
Lee Hinman
Britta Weber
markharwood