honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Pre-populate unicast hosts files (#35136) 

Today when ESIntegTestCase starts some nodes it writes out the unicast hosts
files each time a node starts its transport service. This does mean that a
number of nodes can start and perform their first pinging round without any
unicast hosts which, if the timing is unlucky and a lot of nodes are all
started at the same time, can lead to a split brain as in #35052.

Prior to #33554 this was unlikely to happen since the MockUncasedHostsProvider
would always have yielded the existing hosts, so the timing would have to have
been implausibly unlucky. Since #33554, however, it's more likely because the
race occurs between the start of the first round of pinging and the writing of
the unicast hosts file. It is realistic that new nodes will be configured with
the existing nodes from startup, so this change reinstates that behaviour.

Closes #35052.
This commit is contained in:
David Turner 2018-10-31 19:21:24 +00:00 committed by GitHub
parent 6492eaa84d
commit 0072c90e2a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
test/framework/src/main/java/org/elasticsearch/test/InternalTestCluster.java
View File

@ -1562,6 +1562,7 @@ public final class InternalTestCluster extends TestCluster {
// if we're adding too many master-eligible nodes at once, we can't update the min master setting before adding the nodes. // if we're adding too many master-eligible nodes at once, we can't update the min master setting before adding the nodes.
updateMinMasterNodes(currentMasters + newMasters); updateMinMasterNodes(currentMasters + newMasters);
} }
rebuildUnicastHostFiles(nodeAndClients); // ensure that new nodes can find the existing nodes when they start
List<Future<?>> futures = nodeAndClients.stream().map(node -> executor.submit(node::startNode)).collect(Collectors.toList()); List<Future<?>> futures = nodeAndClients.stream().map(node -> executor.submit(node::startNode)).collect(Collectors.toList());
try { try {

2
x-pack/plugin/ml/src/test/java/org/elasticsearch/xpack/ml/integration/NetworkDisruptionIT.java
View File

@ -25,7 +25,6 @@ import org.elasticsearch.xpack.core.ml.job.config.Job;
import org.elasticsearch.xpack.core.ml.job.persistence.AnomalyDetectorsIndex; import org.elasticsearch.xpack.core.ml.job.persistence.AnomalyDetectorsIndex;
import org.elasticsearch.xpack.core.ml.job.process.autodetect.state.Quantiles; import org.elasticsearch.xpack.core.ml.job.process.autodetect.state.Quantiles;
import org.elasticsearch.xpack.ml.support.BaseMlIntegTestCase; import org.elasticsearch.xpack.ml.support.BaseMlIntegTestCase;
import org.elasticsearch.test.junit.annotations.TestLogging;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays; import java.util.Arrays;
@ -54,7 +53,6 @@ public class NetworkDisruptionIT extends BaseMlIntegTestCase {
return plugins; return plugins;
} }
@TestLogging("org.elasticsearch.discovery.zen:TRACE")
public void testJobRelocation() throws Exception { public void testJobRelocation() throws Exception {
internalCluster().ensureAtLeastNumDataNodes(5); internalCluster().ensureAtLeastNumDataNodes(5);
ensureStableCluster(5); ensureStableCluster(5);