This commit rewrites the state controls in the RecoveryTarget family classes to make it easier to guarantee that:
- recovery resources are only cleared once there are no ongoing requests
- recovery is automatically canceled when the target shard is closed/removed
- canceled recoveries do not leave temp files behind when canceled.
Highlights of the change:
1) All temporary files are cleared upon failure/cancel (see #7315 )
2) All newly created files are always temporary
3) Doesn't list local files on the cluster state update thread (which throw unwanted exception)
4) Recoveries are canceled by a listener to IndicesLifecycle.beforeIndexShardClosed, so we don't need to explicitly call it.
5) Simplifies RecoveryListener to only notify when a recovery is done or failed. Removed subtleties like ignore and retry (they are dealt with internally)
Closes#8092 , Closes#7315
This commit adds the ability to enable / disable relocations
on an entire cluster or on individual indices for either:
* `primaries` - only primaries can rebalance
* `replica` - only replicas can rebalance
* `all` - everything can rebalance (default)
* `none` - all rebalances are disabled
similar to the allocation enable / disable functionality.
Relates to #7288
If `fielddata_fields` are passed as a simple value instead of an array
we end up in an infinite loop createing parsed elements with null
values.
This commit validates the incoming token
Closes#8203
Since we enabled the disk threshold decider by default, we need to
enable the cluster info service so that disk usages and shard sizes can
be gathered also.
Adds a test that checks that we are gathering information by default.
With this change, the elasticsearch script can be linked to another path
without having to set ES_INCLUDE to match the installation path.
Previously, the elasticsearch would find ES_HOME correctly even if linked
but could not find the include script, and finding it would be expected
behavior to me based on its current search path.
Closes#4958
This commit adds throttle stats to the indexing stats and uses a call back from InternalEngine to manage the stats.
Also includes updates the IndexStatsTests to test for these new stats.
Stats added :
```
throttle_time_in_millis
is_throttled
```
Closes#7861
We used to handle FNF exceptions in the store when reading a snapshot.
For instance if we can't open a segments file for a given commit point
we just return an empty metadata object and tracelog the even. This can
cause shards to be false marked as corrupted if a shard is forcefully
removed while a recovery started at the same time. We should in general
bubble up these exceptions and let the caller decided how to handle the
IOExceptions.
The issue with making it dynamic is that in the event a cluster is
switched from a noop to a concrete implementation, there may be
in-flight requests, once these requests complete we adjust the breaker
with a negative number and trip an assertion.
This also rarely uses noop breakers in InternalTestCluster
Previously, the leniency was on a per-query basis, with each query being
parsed into multiple queries, one for each field. If any one of these
queries failed, the entire query was discarded in the name of being
lenient.
Now query parts will only be discarded if they fail for a particular
field, the entire query is not discarded. This helps when performing a
query over a numeric and string field, as only the sub-queries that are
invalid due to format exceptions will be discarded.
Also moves the `simple_query_string` queries out of SimpleQueryTests and
into a dedicated SimpleQueryStringTests class.
Fixes#7967
The live docs that is passed down was ignored by the filter impl. Now the children filter gets wrapped with ApplyAcceptedDocsFilter, so live docs are actually applied.
Closes#8180
The IndicesWarmer gets set before the InternalIndexService gets set, which can lead to a small time window were InternalIndexService isn't set
Closes#8140Closes#8168
Also added a bwc test that runs a delete by query with a has_child query and verifies that only that operation is ignored when recovering from disk during a upgrade.
Closes#8031Closes#8177
Using the Groovy jar with the indy (short for `invokedynamic`) classifier enables usage of the `invokedynamic` instruction available in Java 7+. Due to buggy JVMs, it should only be used with Java 7u60 or later.
Query String query now supports a new `time_zone` option based on JODA time zones.
When using a range on date field, the time zone is applied.
```json
{
"query": {
"query_string": {
"text": "date:[2012 TO 2014]",
"timezone": "Europe/Paris"
}
}
}
```
Closes#7880.
Storing `_timestamp` by default means that under the default configuration, you
would have all the information you need in order to reindex into a different
index.
Close#8139
This adds a NoopCircuitBreaker, and then adds the settings
`indices.breaker.fielddata.type` and `indices.breaker.request.type`,
which can be set to "noop" in order to use a breaker that will never
break, and incurs no overhead during computation.
This also refactors the tests for the CircuitBreakerService to use
@Before and @After functions as well as adding settings in
ElasticsearchIntegrationTest to occasionally use NOOP breakers for all
tests.
This is functionally equivalent to before, so there should be no
user-visible impact, except I added a NOTE in the docs warning about
the interaction of pagination and rescoring.
Closes#6232Closes#7707
Boaz Leskes
Lee Hinman
Simon Willnauer
Alex Ksikes
Randy D. Wallace Jr
Brian Murphy
Ryan Ernst
Ron DuPlain
Clinton Gormley
Martijn van Groningen
Chris Earle
Brian Kim
Adrien Grand
Peter Dyson
Andrei Kolosok
Reuben Sutton
Mark J Greene
Erik Wyatt
David Pilato
Ryan Grimm
Michael McCandless