Cut over all index scope settings to the new setting infrastrucuture
This change moves all index.* settings over to the new infrastructure. This means in short that:
- every setting that has an index scope must be registered up-front
- index settings are validated on index creation, template creation, index settings update
- node level settings starting with index.* are validated on node startup
- settings that are private to ES like index.version.created can only be set by tests when they install a specific test plugin.
- all index settings can be reset by passing null as their value on update
- all index settings defaults can be listed via the settings APIs
Closes#12854Closes#6732Closes#16032Closes#12790
This commit adds handling of channel failures when starting a shard to
o.e.c.a.s.ShardStateAction. This means that shard started requests
that timeout or occur when there is no master or the master leaves
after the request is sent will now be retried from here. The listener
for a shard state request will now only be notified upon successful
completion of the shard state request, or when a catastrophic
non-channel failure occurs.
This commit also refactors the handling of shard failure requests so
that the two shard state actions of shard failure and shard started
now share the same channel-retry and notification logic.
This commit fixes an issue in the handling of TransportExceptions in
ShardStateAction. There were two cases not being handled correctly.
- when the local node is shutting down, handlers will be notified with
a TransportException with a message starting "transport stopped"
- when the remote node disconnects, handlers will be notified with a
NodeDisconnectedException
In both of these cases, the cause of the exception will be null and this
was incorrectly being handled. The first case can passed to the listener
like any other critical non-channel failure, and the second case can be
handled by modifying the logic for detecting master channel exceptions.
There was a third case of NodeNotConnectedException that was not being
treated as a master channel exception but should be.
This commit adds an integration test that simulates the handling of a
shard failure request during a network partition. By isolating the
master from the cluster while a shard failed request is in flight, this
test simulates that we wait until a new master is elected and then retry
sending that shard failed request to the newly elected master.
This commit adds methods to CapturingTransport to separate local and
remote transport exceptions. The motivation for this change is that
local transport exceptions are delivered to listeners (usually, but not
always) wrapped in SendRequestTransportException while remote transport
exceptions are delivered to listeners wrapped in
RemoteTransportException. By making this distinction clear in the
CapturingTransport, this makes it less likely that tests will make
incorrect assumptions about the exceptions coming out of the transport
layer to listeners.
Closes#16057
Currently we use ref counting to manage the life cycles of a translog file. This was done to allow the creation of view and snapshots, making sure that the underlying files are available. This commit takes a simpler route based on the observation that a snapshot doesn't need to have it's own life cycle but rather can lift on the lifecycle of it's parent (translog or view). If code failes to adhere to this assumption it will get a channel already closed exception. As such, each file is now owned by a single owner and there is no need for reference counting. As part of the rewrite TranslogReader is renamed to BaseTranslogReader and ImmutableTranslogReader to TranslogReader
Also, I took the opportunity to clean up legacy translog readers we don't need in master.
Closes#15898
Sometimes action callers might be interested in having an access to the task that they have just initiated. This changes allows callers to get access to the Task object of the actions that they just started if the action supports the task management.
We have two similar tests with the same name, ContextAndHeaderTransportTests.
They shared lots of common code so I extracted much of it into
ActionRecordingPlugin, a plugin which records all action requests for later
inspection.
I also removed all the warnings from both tests. That made lang-mustache
compile cleanly without any custom -Xlint so I removed those. To remove
the warnings I had to add type parameters to ActionFilter which seemed
like a good idea anyway.
1. Gets guice out of the business of building ScoreFunctionParsers and
QueryParsers.
2. Moves QueryParser registration to SearchModule
3. Moves NamedWriteableRegistry construction out of guice and into Node and
TransportClient.
4. Moves shape registration into SearchModule so now all named writeable
registration is done in the SearchModule.
This is breaking for plugin authors. Instead of declaring new QueryParser
like:
```java
public void onModule(IndicesModule module) {
module.registerQueryParser(NewQueryParser.class);
}
```
you do it like:
```java
public void onModule(SearchModule module) {
module.registerQueryParser(NewQueryParser::new);
}
```
The QueryParser's argument no longer come from @Inject, now they come from
the declaration in the plugin. The above example is for a no-arg QueryParser.
Most of the QueryParsers in Elasticsearch are no-arg.
ScoreFunctionParsers have a similar but slightly different change. This:
```java
public void onModule(SearchModule module) {
module.registerFunctionScoreParser(NewFunctionScoreParser.class);
}
```
becomes
```java
public void onModule(SearchModule module) {
module.registerFunctionScoreParser(new NewFunctionScoreParser());
}
```
Since all known ScoreFunctionParsers have no arg constructors its simpler to
just build them at registration time rather than specify a supplier that is
used to build them later.
This commit modifies the load_average in the node stats API response
to be an object containing the one-minute, five-minute and
fifteen-minute load averages as fields (if those values are
available). Additionally, this commit modifies the cat nodes API
response to format the one-minute, five-minute and fifteen-minute load
averages as null if any of the respective values are not available.
This would be useful in order to only perform some validations in the case of
a mapping update and in cases when a mapping is restored eg. after a restart,
such as discussed in #15989.
This replaces the current `applyDefault` parameter which can be derived from
the mapping merge reason: the default mapping should be applied only in case of
a mapping update, if the mapping does not exist yet and if this is not the
default mapping.
Setting realTime to false in the get term vector request ensures,
that after a refresh the document is not fetched from the translog,
which seems to yield different in rare test runs.
The change likely triggering this was introduced in #15933
From this commit on we also validate all settings starting with `index.` that are
node-level settings configured in yaml files or via commandline arguments or system properties.
This check happens on node startup before the actual node is started.
The old infa has been removed in this commit such that nothing uses `DynamicSettings` anymore
and all index-scoped settings require to be registered before the node has fully started up.
Jason Tedor
Isabel Drost-Fromm
Simon Willnauer
Simon Willnauer
Yannick Welsch
Isabel Drost-Fromm
Adrien Grand
Boaz Leskes
Igor Motov
Nik Everett
Christoph Büscher
Alexander Reelsen
Britta Weber