If a request comes in at the same moment the timeout handler for it runs, we may leak a timeoutInfoHolder and erroneously log "Transport response handler not found of id" . The same issue could cause the request tracer to fire a traceUnresolvedResponse call instead of traceReceivedResponse , causing a failure of testTracerLog ( see #10187 ) .
This commit makes sure timeoutInfoHolder is visible before removing the corresponding RequestHolder. It also unifies the TransportService.Adapter#remove(requestId) with TransportService.Adapter#onResponseReceived(requestId), as they are always called together to indicate a response was received.
Closes#10187Closes#10220
When a primary moves to another node, we cancel ongoing recoveries and retry from the primary's new home. At the moment this happens when the primary relocation *starts*. It's a shame as we cancel recoveries that may be close to completion and will finish before the primary has been fully relocated. This commit only triggers the cancelation once the primary relocation is completed.
Next to this, it fixes a race condition between recovery cancellation and the recovery completion. At the moment we may trigger remove a recovered shard just after it was completed. Instead, we should use the recovery cancellation logic to make sure only one code path is followed.
All of the above caused the recoverWhileUnderLoadWithNodeShutdown test to fail (see http://build-us-00.elastic.co/job/es_core_15_debian/32/ ). The test creates an index and then increasingly disallows nodes for it, until only 1 node is left in the allocation filtering rules. Normally, this means we stay in green, but the premature recovery cancellation plus the race condition mentioned above caused a shard to be failed and stay unassigned and the test asserts to fail. This happens due to the following sequence:
- The shard has finished recovering and sent the master a shard started command.
- The recovery is cancelled locally, removing the index shard.
- Master starts shard (deleting it's other copy).
- Local node gets a cluster state with the shard started in it, which cause it to send a shard failed (to make the master aware).
- Shard is failed and can't be re-assigned due to the allocation filter.
The recoverWhileUnderLoadWithNodeShutdown is also adapted a bit to fit the current behavior of allocation filtering (in the past it used to really shut down nodes). Last, all tests in that class are given better names to fit the current terminology.
Clsoes #10218
Today we simply fetch the shards metadata without verifying the
index UUID the shard belongs to. We recently added this UUID
to the shard state metadata. This commit adds verification
to the shard metadata fetching to prevent bringing shards
back into an index it doesn't belong to due to name collisions.
Note, Jackson 2.5 is less lenient when it comes to not starting an object before starting to add fields on a fresh builder, fixed where applicable.
closes#10210
Sometimes, when using transport client for example, through a load balancer, there is a need to send a scheduled ping message to keep each channel alive.
Add support for `transport.ping_schedule`, which controls the schedule (-1 for disabled) at which a ping message will be sent. For transport client case, it gets enabled automatically since almost always this is the desired behavior.
We use the same 6 bytes header format for the ping message, with ES header and -1 for data length for ping message, and simply continue to process the next messages once this is encountered.
closes#10189
The Update Settings API tries to merge the query_string params with the settings sent as body and excluding some "well known params" such as `pretty`, `timeout` etc. Those well known params do not include the params used by IndicesOptions though, so they get merged resulting in invalid settings that get rejected.
Closes#10030
`setConsistencyLevel` setter is already present in the base class `ShardReplicationOperationRequestBuilder`. It is not needed in `DeleteRequestBuilder` and `IndexRequestBuilder`.
Closes#10188
Relates to https://github.com/elastic/elasticsearch-ruby/issues/29, the parsing logic is incorrect as comma is used twice as a separator (e.g. indices_boost=index1,5,index,10 in expected but will never be parsed correctly). Anyways `indices_boost` makes more sense in the request body only where properly parsed and supported.
Closes#6281
Detect the worst-offenders, all IBM versions and several known hotspot
versions that can cause index corruption, and fail on startup.
Provide/detect compiler workarounds when they exist, but warn about
performance degradation.
In all cases the check can be bypassed completely with a safety
switch via undocumented system property (es.bypass.vm.check=true)
Closes#7580
CBOR has a special header that is optional, if exists, allows for exact detection. Also, since we know which formats we support in ES, we can support the object major type case.
closes#7640
I've been attempting to programatically verify that adding index templates via the `{path.conf}/templates/` directory works fine although I was never able to validate this via an API call to the `/_template/`. It seems that these templates do not appear in that API call, which I discovered in the following mail thread:
http://elasticsearch-users.115913.n3.nabble.com/Loading-of-index-settings-template-from-file-in-config-templates-td4024923.html#d1366317284000-912
My question is why wouldn't the `/_template/*` method return these templates? This tends to complicate things for those that want to perform automated tests to verify that they are in fact being recognized and used by Elasticsearch.
Today we leave the shard state behind even if a recovery is half finished
this causes in rare conditions shards to be recovered and promoted as
primaries that have never been fully recovered.
Closes#10053
This commit changes the behaviour of the delete api when processing a delete request that refers to a type that has routing set to required in the mapping, and the routing is missing in the request. Up until now the delete api sent a broadcast delete request to all of the shards that belong to the index, making sure that the document could be found although the routing value wasn't specified. This was probably not the best choice: if the routing is set to required, an error should be thrown instead.
A `RoutingMissingException` gets now thrown instead, like it happens in the same situation with every other api (index, update, get etc.). Last but not least, this change allows to get rid of a couple of `TransportAction`s, `Request`s and `Response`s and simplify the codebase.
Closes#9123Closes#10136
This makes the assertion a bit more flexible and removes the
`ensureGreen` in favor of `ensureYellow`, which is really all that is
needed to perform a search. On slow machines the relocations can take a
while and time out the `ensureGreen`.
This commit allows code to be executed before or after a shards content
is deleted from disk. This is only executed if the shard owns the
content ie. on a shard file system only a primary shard will execute
these operations.
Devin Chollak
Martijn van Groningen
Ryan Ernst
Boaz Leskes
Simon Willnauer
Britta Weber
Sylwester Lachiewicz
Colin Goodheart-Smithe
Shay Banon
Jun Ohtani
Tim Schlechter
javanna
olivier bourgain
Asimov4
popovae
Reuben Sutton
Corey Daley
Robert Muir
Florian Hopf
Al Lefebvre
Simon Willnauer
Michael McCandless
Lee Hinman