This commit terminates any controller processes plugins might have after
the node has been closed. This gives the plugins a chance to shut down their
controllers gracefully.
Previously there was a race condition where controller processes could be shut
down gracefully and terminated by two threads running in parallel, leading to
non-deterministic outcomes.
Additionally, controller processes that failed to shut down gracefully were
not forcibly terminated when running as a Windows service; there was a reliance
on the plugin to shut down its controller gracefully in this situation.
This commit also fixes this problem.
The max concurrent searches logic is complex and we shouldn't duplicate that in multi search template api,
so we should template each individual template search request and then delegate to multi search api.
The max concurrent searches logic is complex and we shouldn't duplicate that in multi search template api,
so we should template each individual template search request and then delegate to multi search api.
Adds a new "icu_collation" field type that exposes lucene's
ICUCollationDocValuesField. ICUCollationDocValuesField is the replacement
for ICUCollationKeyFilter which has been deprecated since Lucene 5.
This commit renames ScriptEngineService to ScriptEngine. It is often
confusing because we have the ScriptService, and then
ScriptEngineService implementations, but the latter are not services as
we see in other places in elasticsearch.
File scripts have 2 related settings: the path of file scripts, and
whether they can be dynamically reloaded. This commit deprecates those
settings.
relates #21798
A constraint on the global checkpoint was inadvertently committed from
the inlining global checkpoint work. Namely, the constraint prevents the
global checkpoint from advancing to no ops performed, a situation that
can occur when shards are started but empty.
Today we rely on background syncs to relay the global checkpoint under
the mandate of the primary to its replicas. This means that the global
checkpoint on a replica can lag far behind the primary. The commit moves
to inlining global checkpoints with replication requests. When a
replication operation is performed, the primary will send the latest
global checkpoint inline with the replica requests. This keeps the
replicas closer in-sync with the primary.
However, consider a replication request that is not followed by another
replication request for an indefinite period of time. When the replicas
respond to the primary with their local checkpoint, the primary will
advance its global checkpoint. During this indefinite period of time,
the replicas will not be notified of the advanced global
checkpoint. This necessitates a need for another sync. To achieve this,
we perform a global checkpoint sync when a shard falls idle.
Relates #24513
This changes the way we register pre-configured token filters so that
plugins can declare them and starts to move all of the pre-configured
token filters out of core. It doesn't finish the job because doing
so would make the change unreviewably large. So this PR includes
a shim that keeps the "old" way of registering pre-configured token
filters around.
The Lowercase token filter is special because there is a "special"
interaction between it and the lowercase tokenizer. I'm not sure
exactly what to do about it so for now I'm leaving it alone with
the intent of figuring out what to do with it in a followup.
This also renames these pre-configured token filters from
"pre-built" to "pre-configured" because that seemed like a more
descriptive name.
This is a part of #23658
This test waited 10 seconds for a refresh listener to appear in
the stats. It turns out that in our NFS testing infrastructure this can
take a lot longer than 10 seconds. The error reported here:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+nfs/257/consoleFull
has it taking something like 15 seconds. This bumps the timeout
to a solid minute.
Closes#24417
Now that indices have a single type by default, we can move to the next step
and identify documents using their `_id` rather than the `_uid`.
One notable change in this commit is that I made deletions implicitly create
types. This helps with the live version map in the case that documents are
deleted before the first type is introduced. Otherwise there would be no way
to differenciate `DELETE index/foo/1` followed by `PUT index/foo/1` from
`DELETE index/bar/1` followed by `PUT index/foo/1`, even though those are
different if versioning is involved.
If a node in version >= 5.3 acts as a coordinating node during a scroll request that targets a single shard, the scroll may return the same documents over and over iff the targeted shard is hosted by a node with a version <= 5.3.
The nodes in this version will advance the scroll only if the search_type has been set to `query_and_fetch` though this search type has been removed in 5.3.
This change handles this situation by adding the removed search_type in the request that targets a node in version <= 5.3.
Previously, Mustache would call `toString` on the `_ingest.timestamp`
field and return a date format that did not match Elasticsearch's
defaults for date-mapping parsing. The new ZonedDateTime class in Java 8
happens to do format itself in the same way ES is expecting.
This commit adds support for a feature flag that enables the usage of this new date format
that has more native behavior.
Fixes#23168.
This new fix can be found in the form of a cluster setting called
`ingest.new_date_format`. By default, in 5.x, the existing behavior
will remain the same. One will set this property to `true` in order to
take advantage of this update for ingest-pipeline convenience.
This commit fixes inefficient (worst case exponential) loading of
snapshot repository data when checking for incompatible snapshots,
that was introduced in #22267. When getting snapshot information,
getRepositoryData() was called on every snapshot, so if there are
a large number of snapshots in the repository and _all snapshots
were requested, the performance degraded exponentially. This
commit fixes the issue by only calling getRepositoryData once and
using the data from it in all subsequent calls to get snapshot
information.
Closes#24509
When starting a new replication group in an index level replication test
case, a started replica would not have a valid recovery state. This
violates simple assumptions as replicas always have to have recovered
before being started. This commit causes this to be the case that this
assumption is not violated too.
We previously removed this assertion because it could be violated in
races. This commit adds this assertion back with sampling done more
carefully to avoid failures solely due to race conditions.
When multiple bootstrap checks fail, it's not clear where one error
message begins and the next error message ends. This commit numbers the
bootstrap check error messages so they are easier to read.
Relates #24548
This starts breaking up the `UpdateHelper.prepare` method so that each piece can
be individually unit tested. No actual functionality has changed.
Note however, that I did add a TODO about `ctx.op` leniency, which I'd love to
remove as a separate PR if desired.
This commit fixes a bug in the cache expire after access
implementation. The bug is this: if you construct a cache with an expire
after access of T, put a key, and then touch the key at some time t > T,
the act of getting the key would update the access time for the entry
before checking if the entry was expired. There are situations in which
expire after access would be honored (e.g., if the cache needs to prune
the LRU list to keep the cache under a certain weight, or a manual
refresh was called) but this behavior is otherwise broken.
Relates #24546
In order to make MockLogAppender (utility to test logging) available outside
of es-core move MockLogAppender from test core-tests to test framework. As
package names do not change, no need to change clients.
Today when opening the engine we skip gaps in the history, advancing the
local checkpoint until it is equal to the maximum sequence number
contained in the commit. This allows history to advance, but it leaves
gaps. A previous change filled these gaps when recovering from store,
but since we were skipping the gaps while opening the engine, this
change had no effect. This commit removes the gap skipping when opening
the engine allowing the gap filling to do its job.
Relates #24535
When installing plugin permissions, we try to set the permissions on all
installed files ourselves because a umask from the user could violate
everything needed to get the permissions right. Sadly, directories were
not handled correctly at all and so we were still left with broken
installations with umasks like 0077. This commit fixes this issue, adds
a thorough unit test for the situation, and most importantly, adds a
test that sets the umask before installing the plugin.
Relates #24527
I stumbled on this code today and I hated it; I wrote it. I did not like
that you could not tell at a glance whether or not the method parameters
were correct. This commit fixes it.
Relates #24522
Previously you could set the system property tests.es.path.data and
start the run task with a custom data directory. A change in core to
detect duplicate settings broke this. That change should stay, but we
can work around this easily by only setting path.data to the data
directory if tests.es.path.data is not set.
Christoph Büscher
David Roberts
Clinton Gormley
Tanguy Leroux
Martijn van Groningen
Isabel Drost-Fromm
Matt Weber
Ryan Ernst
Tal Levy
Jason Tedor
Suhas Karanth
Nik Everett
Jim Ferenczi
Adrien Grand
joachimdraeger
Lee Hinman
Igor Motov
Hendrik Muhs
glefloch
javanna
Koen De Groote