honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

RecoveryIT should wait for green when in mixed cluster to avoid unassigned shards 

The test starts with two old nodes and creates indices (without waiting for green, which is fixed here too). Then it restarts one of the nodes and waits for it to join the cluster. This wait condition only uses wait for yellow as our generic infra doesn't how many nodes are there in total. Once the restarted node is part of the cluster (mixed mode) the second old node is restarted. If indices are not fully allocated when that happens, the shards will go into delayed unassigned mode. If the recovery of the replica never completed we may end up with corrupted / no secondary copy on the node. This will cause the shards to be delayed for 1m before being reassigned and the test will time out.
This commit is contained in:
Boaz Leskes 2017-09-24 22:27:32 +02:00
parent 2b6f75730e
commit cd2a4372b4
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
qa/rolling-upgrade/src/test/java/org/elasticsearch/upgrades/RecoveryIT.java
View File

@ -105,6 +105,7 @@ public class RecoveryIT extends ESRestTestCase {
.put(IndexMetaData.INDEX_NUMBER_OF_SHARDS_SETTING.getKey(), 1) .put(IndexMetaData.INDEX_NUMBER_OF_SHARDS_SETTING.getKey(), 1)
.put(IndexMetaData.INDEX_NUMBER_OF_REPLICAS_SETTING.getKey(), 1); .put(IndexMetaData.INDEX_NUMBER_OF_REPLICAS_SETTING.getKey(), 1);
createIndex(index, settings.build()); createIndex(index, settings.build());
ensureGreen();
} else if (clusterType == CLUSTER_TYPE.UPGRADED) { } else if (clusterType == CLUSTER_TYPE.UPGRADED) {
ensureGreen(); ensureGreen();
Response response = client().performRequest("GET", index + "/_stats", Collections.singletonMap("level", "shards")); Response response = client().performRequest("GET", index + "/_stats", Collections.singletonMap("level", "shards"));
@ -123,6 +124,11 @@ public class RecoveryIT extends ESRestTestCase {
assertThat("different history uuid found for shard on " + nodeID, historyUUID, equalTo(expectHistoryUUID)); assertThat("different history uuid found for shard on " + nodeID, historyUUID, equalTo(expectHistoryUUID));
} }
} }
} else {
// we are now in mixed cluster mode. we want to make sure the shard is fully allocated on the new node that was just
// started in order not to run into delayed unassigned shards when we bring down the old node (there must be a fully valid
// copy)
ensureGreen();
} }
} }