Doc values can now only be enabled by setting `doc_values: true` in the
mappings. Removing this feature also means that we can now fail mapping updates
that try to disable doc values.
Parsing is currently very lenient, which has the bad side-effect that if you
have a typo and pass eg. `store: fasle` this will actually be interpreted as
`store: true`. Since mappings can't be changed after the fact, it is quite bad
if it happens on an index that already contains data.
Note that this does not cover all settings that accept a boolean, but since the
PR was quite hard to build and already covers some main settirgs like `store`
or `doc_values` this would already be a good incremental improvement.
Doc values currently default to `true` if the field is indexed and not analyzed.
So setting `index:no` automatically disables doc values, which is not explicit
in the documentation.
This commit makes doc values default to true for numerics, booleans regardless
of whether they are indexed. Not indexed strings still don't have doc values,
since we can't know whether it is rather a text or keyword field. This
potential source of confusion should go away when we split `string` into `text`
and `keyword`.
This commit adds handling on the master side for shard failure requests
for shards that do not exist at the time that they are processed on the
master node (whether it be from errant requests, duplicate requests, or
both the primary and replica notifying the master of a shard
failure). This change is made because such shard failure requests should
always be considered successful (the failed shard is not there anymore),
but could be marked as failed if batched with a shard failure request
that does in fact fail. This avoids the possibility of an unexpected
catastrophic failure while applying the failed shards from causing such
a request to also be marked as failed setting in motion additional
failures.
Closes#16089
RescoreBuilder: Add parsing and creating of RescoreSearchContext
Adding the ability to parse from xContent to the rescore builder. Also making RescoreBuilder an abstract base class that encapsulates the window_size setting, with QueryRescoreBuilder as its only implementation at the moment.
Relates to #15559
Merge feature/ingest branch into master branch.
This adds the ingest feature to ES that allows to preprocess document before indexing on an ingest node.
By default a node is an ingest node. Documents are preprocessed via a pipeline. A pipeline consists
out of one or more processors Each processor makes one or more modifications to a document processed.
There are many types of processors available out-of-the-box that are designed to make a specific change to a document being processed. In a cluster many pipeline can be configured via dedicated pipeline APIs. An new option on the bulk
and index APIs allows to control what pipeline is picked for preprocessing. If no pipeline is specified then the ingest
feature is skipped and no preprocessing takes place.
The current RescoreBaseBuilder only serves as a container
for a pair of the optional `window_size` parameter and
the actual rescorer. Instead of a wrapper object, this makes
it an abstract class that conrete implementations like
QueryRescoreBuilder can extend.
With this commit we deprecate the widely misunderstood
fuzzy query but will still allow the fuzziness
parameter in match queries and suggesters.
Relates to #15760
Boolean parsing is now strict. Also added isIngestNode methods to DiscoveryNode to align this setting with the existing node.data node.master and node.client. Removed NodeModule#isIngestEnabled methods that are not needed anymore.
This commit removes non-reproducible randomness from
SearchWhileCreatingIndexIT. The cause of the non-reproducible randomness
is the use of a random draws for the shard preference inside of a
non-deterministic while loop. Because the inner while loop executed a
non-deterministic number of times, the draws for the next iteration of
the outer loop would be impacted by this making the random draws
non-reproducible. The solution is to move the random draws outside of
the while loop (just make a single draw for the prefernce and increment
with a counter), and remove the outer loop iteration instead using an
annotation to get the desired repetitions.
Closes#16208
This commit amends the logging statement in two places in
ShardStateAction to log the full shard ID instead of just the numerical
shard ID (without the index name).
This commit migrates all the settings under network service to the new settings infra.
It also adds some chaining utils to make fall back settings slightly less verbose.
Breaking (but I think acceptable) - network.tcp.no_delay and network.tcp.keep_alive used to accept the value `default` which make us not set them at all on netty. Our default was true so we weren't using this feature. I removed it and now we only accept a true boolean.
* `test.cluster.node.seed` was only used in one place where it wasn't adding value and was now replaced with a constant
* `tests.portsfile` is now an official setting and has been renamed to `node.portsfile`
this commit also convert `search.default_keep_alive` and `search.keep_alive_interval` to the new settings infrastrucutre
This commit also uses a try/finally with success pattern instead of catching
and excpetion. TranslogTests reproduce with `-Dtests.seed=DF6A38BAE739227A` every
time.
Closes#16142
TranslogWriter.closeIntoReader transfers the file ownership from a writer to a reader and closes the writer. If the transfer fails, we need to make sure we closed the underlying channel as the writer is already closes.
See: http://build-us-00.elastic.co/job/es_core_master_regression/4355
Relates to #16142
Today we run the metadata upgrade only on the current major version
but this should run on every upgrade at least once to ensure we don't miss
an important check or upgrade.
