ElasticsearchRestTests extends now ElasticsearchIntegrationTest and makes use of our ordinary test infrastructure, in particular all randomized aspects now come for free instead of having to maintain a separate (custom) tests runner
We previously parsed only the tests that needed to be run given the version of the cluster the tests are running against. This doesn't happen anymore as it didn't buy much and it would be harder to support as the tests get now parsed before the test cluster gets started. Thus all the tests are now parsed regardless of their skip sections, afterwards the ones that don't need to be run will be skipped through assume directives.
Fixed REST tests that rely on a specific number of shards as this change introduces also random number of shards and replicas (through randomIndexTemplate)
Closes#5654
Some tests disable refresh and verify that documents cannot be found without an explicit refresh. With at least one replica and waiting for yellow though, it can happen that we start indexing docs when the replicas are not ready yet, and as soon as they become ready they get refreshed, causing the newly added docs to be unexpectedly found. Solution is to disable replicas for these specific tests.
REST tests get run against either 1 node or multiple nodes. Wait for yellow with replicas>0 is not enough when running against multiple nodes as replicas shard might get initialized during testing, which can cause timing issues.
This change make single shard requests fail when no routing is specified and routing has been configured to be required in the mapping. Thi
Closes#4506
javanna
Clinton Gormley
Boaz Leskes
Lee Hinman
Honza Král
Martijn van Groningen