FSync translog outside of the writers global lock
Today we aquire a write global lock that blocks all modification to the
translog file while we fsync / checkpoint the file. Yet, we don't necessarily
needt to block concurrent operations here. This can lead to a lot of blocked
threads if the machine has high concurrency (lot os CPUs) but uses slow disks
(spinning disks) which is absolutely unnecessary. We just need to protect from
fsyncing / checkpointing concurrently but we can fill buffers and write to the
underlying file in a concurrent fashion.
This change introduces an additional lock that we hold while fsyncing but moves
the checkpointing code outside of the writers global lock.
This commit fixes the inequality symbol used in a test assertion in
RepositoryS3SettingsTests#testInvalidChunkBufferSizeRepositorySettings. The
inequality symbol was previously backwards but fixed in commit
cad0608cdb but fixing the inequality
symbol here was missed in that commit.
Closes#18449
Currently rounding intervals obtained by nextRoundingValue() for hour, minute and
second units can include an extra hour when happening at DST transitions that add
an extra hour (eg CEST -> CET). This changes the rounding logic for time units
smaller or equal to an hour to fix this.
Closes#18326
This commit adds simple GC overhead logging. This logging captures
intervals where the JVM is spending a lot of time performing GC but it
is not necessarily the case that each GC is large. For a start, this
logging is simple and does not attempt to incorporate whether or not the
collections were efficient (that is, we are only capturing that a lot of
GC is happening, not that a lot of useless GC is happening).
Relates #18419
We are currently only parsing the array-syntax for the rescore part
in SearchSourceBuilder ("rescore" : [ {...}, {...} ]) . We also need
to support "rescore" : {...}
Closes#18439
- Moves recovery logic into IndexShard
- Simplifies logic to cancel peer recovery of shard where recovery source node changed
- Ensures routing entry is set on initialization of IndexShard
With this commit we clear all caches after testing the parent circuit breaker.
This is necessary as caches hold on to circuit breakers internally. Additionally,
due to usage of CircuitBreaker#addWithoutBreaking() in caches, it's even possible
to go above the limit. As a consequence, all subsequent requests fall victim to
the limit.
Hence, right after the parent circuit breaker tripped, we clear all caches to
reduce these circuit breakers to 0 again. We also exclude the clear caches
transport request from limit check in order to ensure it will succeed. As this is
typically a very small and low-volume request, it is deemed ok to exclude it.
Closes#18325
Two of the snippets in validate weren't working properly so they are
marked as skip and linked to this:
https://github.com/elastic/elasticsearch/issues/18254
We didn't properly handle empty parameter values. We were sending
them as the literal string "null". Now we do better and send them
as the empty string.
This commit adds a variety of real disk metrics for the block devices
that back Elasticsearch data paths. A collection of statistics are read
from /proc/diskstats and are used to report the raw metrics for
operations and read/write bytes.
Relates #15915
This uses the same backoff policy we use for bulk and just retries until
the request isn't rejected.
Instead of `{"retries": 12}` in the response to count retries this now
looks like `{"retries": {"bulk": 12", "search": 1}`.
Closes#18059
This commit refactors the JvmGcMonitorService so that it can be
tested. In particular, hooks are added to verify that the
JvmMonitorService correctly observes slow GC events, and that the
JvmGcMonitorService logs the correct messages.
Relates #18378
Instead of re-exposing index metadata and blocks in RoutingNodes (which is part of the cluster state before rerouting), expose it as part of the RoutingAllocation which is known to be only temporarily used during reroute.
Sorts an array of values in ascending or descending order. If all elements are numerics, they will be sorted numerically. If values are strings, or mixtures of strings/numbers, the elements will be sorted lexicographically.
Simon Willnauer
Simon Willnauer
Nik Everett
Isabel Drost-Fromm
Jason Tedor
Clinton Gormley
Christoph Büscher
markharwood
Tanguy Leroux
Daniel Mitterdorfer
Robert Muir
Daniel Mitterdorfer
Isabel Drost-Fromm
Yannick Welsch
Uwe Schindler
polyfractal
Jack Conradson
Lee Hinman
Zachary Tong