This adds a new boolean (index.merge.scheduler.auto_throttle) dynamic
setting, default true (matching Lucene), to adaptively set the IO rate
limit for merges over time.
This is more flexible than the previous fixed rate throttling because
it responds depending on the incoming merge rate, so search-heavy
applications that are not doing much indexing will see merges heavily
throttled while indexing-heavy cases will lighten the throttle so
merges can keep up within incoming indexing.
The fixed rate throttling is still available as a fallback if things
go horribly wrong.
Closes#9243Closes#9133
previous push was partial by mistake, we still need the wrapped dirs around after being closed for the test infra, for now, explicitly clear it in the leak test (which is still bad apple)
I ran the bad apple test for index memory leaks and still saw leaks, it seems like we don't properly clean the dirs from the static mock test dir wrapper
`cluster.routing.allocation.disk.include_relocations` and
`cluster.routing.allocation.disk.reroute_interval` are both boolean
settings, so they should have the `Validator.BOOLEAN` applied.
Fixes#9309
validation tests for constants
Currently the snapshot flag for Version constants is only set to true
for CURRENT. However, this means that the snapshot state changes from
branch to branch. Instead, snapshot should be "is this version
released?". This change also adds a validation test checking that
ID -> constant and vice versa are correct, and fixes one bug found there
(for an unreleased version).
- don't allow for soft references anymore in the recycler
- remove some abusive thread locals
- don't recycle independently float/double and int/long pages, they are the
same and just interpret bits differently.
Close#9272
BucketAggregationMode used to be part of the framework, now it's only an
implementation detail of the terms, histogram, geohash grid and scripted
aggregators.
Aggregator.estimatedBucketCount() was a complicated way to do the initial sizing
of the data structures, but it did not work very well in practice and was rather
likely to lead to over-sized data-structures causing OOMEs. It's removed now and
all data-structures start with a size of 1 and grow exponentially.
Aggregator.preCollection() is now symetric with postCollection(): it exists on
all aggregation objects where postCollection() also is and recursively calls its
children.
Fixed other minor issues related to generics and exceptions.
Close#9097
The query cache has a mechanism that disables it automatically when
SearchContext.nowInMillis() is used. One issue with that is that the date math
parser always evaluates the current timestamp when parsing a date, even if it
is not needed. As a consequence, whenever you use a date expression in your
queries, the query cache would not be used.
Close#9225
Some of our Java API requests have public setters but their corresponding getters are package private only. This commit makes those getters public as well.
Closes#9273
This change fixes _timestamp's serialization method to write out
`doc_values` and `doc_values_format`, which could already be set,
but would not be written out.
closes#8893closes#8967
Today we give the HTTP status back within the HTTP response itself and within the JSON response as well:
```sh
curl localhost:9200/
```
```js
{
"status" : 200,
"name" : "Red Wolf",
"version" : {
"number" : "2.0.0",
"build_hash" : "6837a61d8a646a2ac7dc8da1ab3c4ab85d60882d",
"build_timestamp" : "2014-08-19T13:55:56Z",
"build_snapshot" : true,
"lucene_version" : "4.9"
},
"tagline" : "You Know, for Search"
}
```
On Windows platforms when JAVA_HOME is not defined, a message is printed on standard output and the bat script is paused until the user press a key. This behavior is not compliant to automated processes where elasticsearch.bat can be executed by another script. This commit adds a new parameter --silent / -s that allow to skip the pause. Also, the error message is directed to standard and error outputs.
Closes#8913
This commits adds a test that simulate disconnecting nodes and dropping requests during the various stages of recovery and solves all the issues that were raised by it. In short:
1) On going recoveries will be scheduled for retry upon network disconnect. The default retry period is 5s (cross node connections are checked every 10s by default).
2) Sometimes the disconnect happens after the target engine has started (but the shard is still in recovery). For simplicity, I opted to restart the recovery from scratch (where little to no files will be copied again, because there were just synced).
3) To protected against dropped requests, a Recovery Monitor was added that fails a recovery if no progress has been made in the last 30m (by default), which is equivalent to the long time outs we use in recovery requests.
4) When a shard fails on a node, we try to assign it to another node. If no such node is available, the shard will remain unassigned, causing the target node to clean any in memory state for it (files on disk remain). At the moment the shard will remain unassigned until another cluster state change happens, which will re-assigned it to the node in question but if no such change happens the shard will remain stuck at unassigned. The commits adds an extra delayed reroute in such cases to make sure the shard will be reassinged
5) Moved all recovery related settings to the RecoverySettings.
Closes#8720
You can now specify `format` in the request definition for most numeric metric aggregations. The exceptions are Percentile_Ranks, Cardinality and Value_Count as the response type of these can be different from the field type so the formatter won't work.
Closes#6812
Michael McCandless
Dmitry Polushkin
Clinton Gormley
Boaz Leskes
Adrien Grand
Ryan Ernst
Robert Muir
Shay Banon
David Haney
eBuildy
Lee Hinman
Luca Cavanna
David Pilato
Paul Echeverri
Tomoya Hirano
tlrx
Martijn van Groningen
Simon Willnauer
Colin Goodheart-Smithe
Christoph Büscher