honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Prevent in-place downgrades and invalid upgrades (#41731) 

Downgrading an Elasticsearch node to an earlier version is unsupported, because
we do not make any attempt to guarantee that a node can read any of the on-disk
data written by a future version. Yet today we do not actively prevent
downgrades, and sometimes users will attempt to roll back a failed upgrade with
an in-place downgrade and get into an unrecoverable state.

This change adds the current version of the node to the node metadata file, and
checks the version found in this file against the current version at startup.
If the node cannot be sure of its ability to read the on-disk data then it
refuses to start, preserving any on-disk data in its upgraded state.

This change also adds a command-line tool to overwrite the node metadata file
without performing any version checks, to unsafely bypass these checks and
recover the historical and lenient behaviour.
This commit is contained in:
David Turner 2019-05-21 07:52:01 +01:00
parent ec63160243
commit 7abeaba8bb
12 changed files with 556 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
docs/reference/commands/node-tool.asciidoc
View File

@ -4,14 +4,15 @@
The `elasticsearch-node` command enables you to perform certain unsafe
operations on a node that are only possible while it is shut down. This command
allows you to adjust the <<modules-node,role>> of a node and may be able to
recover some data after a disaster.
recover some data after a disaster or start a node even if it is incompatible
with the data on disk.
[float]
=== Synopsis
[source,shell]
--------------------------------------------------
bin/elasticsearch-node repurpose|unsafe-bootstrap|detach-cluster
bin/elasticsearch-node repurpose|unsafe-bootstrap|detach-cluster|override-version
[--ordinal <Integer>] [-E <KeyValuePair>]
[-h, --help] ([-s, --silent] | [-v, --verbose])
--------------------------------------------------
@ -19,7 +20,7 @@ bin/elasticsearch-node repurpose|unsafe-bootstrap|detach-cluster
[float]
=== Description
This tool has three modes:
This tool has four modes:
* `elasticsearch-node repurpose` can be used to delete unwanted data from a
node if it used to be a <<data-node,data node>> or a
@ -36,6 +37,11 @@ This tool has three modes:
cluster bootstrapping was not possible, it also enables you to move nodes
into a brand-new cluster.
* `elasticsearch-node override-version` enables you to start up a node
even if the data in the data path was written by an incompatible version of
{es}. This may sometimes allow you to downgrade to an earlier version of
{es}.
[[node-tool-repurpose]]
[float]
==== Changing the role of a node
@ -109,6 +115,25 @@ way forward that does not risk data loss, but it may be possible to use the
`elasticsearch-node` tool to construct a new cluster that contains some of the
data from the failed cluster.
[[node-tool-override-version]]
[float]
==== Bypassing version checks
The data that {es} writes to disk is designed to be read by the current version
and a limited set of future versions. It cannot generally be read by older
versions, nor by versions that are more than one major version newer. The data
stored on disk includes the version of the node that wrote it, and {es} checks
that it is compatible with this version when starting up.
In rare circumstances it may be desirable to bypass this check and start up an
{es} node using data that was written by an incompatible version. This may not
work if the format of the stored data has changed, and it is a risky process
because it is possible for the format to change in ways that {es} may
misinterpret, silently leading to data loss.
To bypass this check, you can use the `elasticsearch-node override-version`
tool to overwrite the version number stored in the data path with the current
version, causing {es} to believe that it is compatible with the on-disk data.
[[node-tool-unsafe-bootstrap]]
[float]
@ -262,6 +287,9 @@ one-node cluster.
`detach-cluster`:: Specifies to unsafely detach this node from its cluster so
it can join a different cluster.
`override-version`:: Overwrites the version number stored in the data path so
that a node can start despite being incompatible with the on-disk data.
`--ordinal <Integer>`:: If there is <<max-local-storage-nodes,more than one
node sharing a data path>> then this specifies which node to target. Defaults
to `0`, meaning to use the first node in the data path.
@ -423,3 +451,32 @@ Do you want to proceed?
Confirm [y/N] y
Node was successfully detached from the cluster
----
[float]
==== Bypassing version checks
Run the `elasticsearch-node override-version` command to overwrite the version
stored in the data path so that a node can start despite being incompatible
with the data stored in the data path:
[source, txt]
----
node$ ./bin/elasticsearch-node override-version
WARNING: Elasticsearch MUST be stopped before running this tool.
This data path was last written by Elasticsearch version [x.x.x] and may no
longer be compatible with Elasticsearch version [y.y.y]. This tool will bypass
this compatibility check, allowing a version [y.y.y] node to start on this data
path, but a version [y.y.y] node may not be able to read this data or may read
it incorrectly leading to data loss.
You should not use this tool. Instead, continue to use a version [x.x.x] node
on this data path. If necessary, you can use reindex-from-remote to copy the
data from here into an older cluster.
Do you want to proceed?
Confirm [y/N] y
Successfully overwrote this node's metadata to bypass its version compatibility checks.
----

19
server/src/main/java/org/elasticsearch/cluster/coordination/ElasticsearchNodeCommand.java
View File

@ -44,7 +44,7 @@ import java.util.Objects;
public abstract class ElasticsearchNodeCommand extends EnvironmentAwareCommand {
private static final Logger logger = LogManager.getLogger(ElasticsearchNodeCommand.class);
protected final NamedXContentRegistry namedXContentRegistry;
static final String DELIMITER = "------------------------------------------------------------------------\n";
protected static final String DELIMITER = "------------------------------------------------------------------------\n";
static final String STOP_WARNING_MSG =
DELIMITER +
@ -81,9 +81,8 @@ public abstract class ElasticsearchNodeCommand extends EnvironmentAwareCommand {
throw new ElasticsearchException(NO_NODE_FOLDER_FOUND_MSG);
}
processNodePaths(terminal, dataPaths, env);
} catch (LockObtainFailedException ex) {
throw new ElasticsearchException(
FAILED_TO_OBTAIN_NODE_LOCK_MSG + " [" + ex.getMessage() + "]");
} catch (LockObtainFailedException e) {
throw new ElasticsearchException(FAILED_TO_OBTAIN_NODE_LOCK_MSG, e);
}
}
@ -166,6 +165,18 @@ public abstract class ElasticsearchNodeCommand extends EnvironmentAwareCommand {
}
}
protected NodeEnvironment.NodePath[] toNodePaths(Path[] dataPaths) {
return Arrays.stream(dataPaths).map(ElasticsearchNodeCommand::createNodePath).toArray(NodeEnvironment.NodePath[]::new);
}
private static NodeEnvironment.NodePath createNodePath(Path path) {
try {
return new NodeEnvironment.NodePath(path);
} catch (IOException e) {
throw new ElasticsearchException("Unable to investigate path [" + path + "]", e);
}
}
//package-private for testing
OptionParser getParser() {
return parser;

2
server/src/main/java/org/elasticsearch/cluster/coordination/NodeToolCli.java
View File

@ -22,6 +22,7 @@ import org.elasticsearch.cli.CommandLoggingConfigurator;
import org.elasticsearch.cli.MultiCommand;
import org.elasticsearch.cli.Terminal;
import org.elasticsearch.env.NodeRepurposeCommand;
import org.elasticsearch.env.OverrideNodeVersionCommand;
// NodeToolCli does not extend LoggingAwareCommand, because LoggingAwareCommand performs logging initialization
// after LoggingAwareCommand instance is constructed.
@ -39,6 +40,7 @@ public class NodeToolCli extends MultiCommand {
subcommands.put("repurpose", new NodeRepurposeCommand());
subcommands.put("unsafe-bootstrap", new UnsafeBootstrapMasterCommand());
subcommands.put("detach-cluster", new DetachClusterCommand());
subcommands.put("override-version", new OverrideNodeVersionCommand());
}
public static void main(String[] args) throws Exception {

11
server/src/main/java/org/elasticsearch/env/NodeEnvironment.java vendored
View File

@ -31,6 +31,7 @@ import org.apache.lucene.store.LockObtainFailedException;
import org.apache.lucene.store.NativeFSLockFactory;
import org.apache.lucene.store.SimpleFSDirectory;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.Version;
import org.elasticsearch.cluster.metadata.IndexMetaData;
import org.elasticsearch.cluster.node.DiscoveryNode;
import org.elasticsearch.common.CheckedFunction;
@ -250,7 +251,7 @@ public final class NodeEnvironment implements Closeable {
sharedDataPath = null;
locks = null;
nodeLockId = -1;
nodeMetaData = new NodeMetaData(generateNodeId(settings));
nodeMetaData = new NodeMetaData(generateNodeId(settings), Version.CURRENT);
return;
}
boolean success = false;
@ -395,7 +396,6 @@ public final class NodeEnvironment implements Closeable {
logger.info("heap size [{}], compressed ordinary object pointers [{}]", maxHeapSize, useCompressedOops);
}
/**
* scans the node paths and loads existing metaData file. If not found a new meta data will be generated
* and persisted into the nodePaths
@ -405,10 +405,15 @@ public final class NodeEnvironment implements Closeable {
final Path[] paths = Arrays.stream(nodePaths).map(np -> np.path).toArray(Path[]::new);
NodeMetaData metaData = NodeMetaData.FORMAT.loadLatestState(logger, NamedXContentRegistry.EMPTY, paths);
if (metaData == null) {
metaData = new NodeMetaData(generateNodeId(settings));
metaData = new NodeMetaData(generateNodeId(settings), Version.CURRENT);
} else {
metaData = metaData.upgradeToCurrentVersion();
}
// we write again to make sure all paths have the latest state file
assert metaData.nodeVersion().equals(Version.CURRENT) : metaData.nodeVersion() + " != " + Version.CURRENT;
NodeMetaData.FORMAT.writeAndCleanup(metaData, paths);
return metaData;
}

74
server/src/main/java/org/elasticsearch/env/NodeMetaData.java vendored
View File

@ -19,6 +19,7 @@
package org.elasticsearch.env;
import org.elasticsearch.Version;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.xcontent.ObjectParser;
import org.elasticsearch.common.xcontent.XContentBuilder;
@ -31,65 +32,103 @@ import java.io.OutputStream;
import java.util.Objects;
/**
* Metadata associated with this node. Currently only contains the unique uuid describing this node.
* Metadata associated with this node: its persistent node ID and its version.
* The metadata is persisted in the data folder of this node and is reused across restarts.
*/
public final class NodeMetaData {
private static final String NODE_ID_KEY = "node_id";
private static final String NODE_VERSION_KEY = "node_version";
private final String nodeId;
public NodeMetaData(final String nodeId) {
private final Version nodeVersion;
public NodeMetaData(final String nodeId, final Version nodeVersion) {
this.nodeId = Objects.requireNonNull(nodeId);
this.nodeVersion = Objects.requireNonNull(nodeVersion);
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
NodeMetaData that = (NodeMetaData) o;
return Objects.equals(this.nodeId, that.nodeId);
return nodeId.equals(that.nodeId) &&
nodeVersion.equals(that.nodeVersion);
}
@Override
public int hashCode() {
return this.nodeId.hashCode();
return Objects.hash(nodeId, nodeVersion);
}
@Override
public String toString() {
return "node_id [" + nodeId + "]";
return "NodeMetaData{" +
"nodeId='" + nodeId + '\'' +
", nodeVersion=" + nodeVersion +
'}';
}
private static ObjectParser<Builder, Void> PARSER = new ObjectParser<>("node_meta_data", Builder::new);
static {
PARSER.declareString(Builder::setNodeId, new ParseField(NODE_ID_KEY));
PARSER.declareInt(Builder::setNodeVersionId, new ParseField(NODE_VERSION_KEY));
}
public String nodeId() {
return nodeId;
}
public Version nodeVersion() {
return nodeVersion;
}
public NodeMetaData upgradeToCurrentVersion() {
if (nodeVersion.equals(Version.V_EMPTY)) {
assert Version.CURRENT.major <= Version.V_7_0_0.major + 1 : "version is required in the node metadata from v9 onwards";
return new NodeMetaData(nodeId, Version.CURRENT);
}
if (nodeVersion.before(Version.CURRENT.minimumIndexCompatibilityVersion())) {
throw new IllegalStateException(
"cannot upgrade a node from version [" + nodeVersion + "] directly to version [" + Version.CURRENT + "]");
}
if (nodeVersion.after(Version.CURRENT)) {
throw new IllegalStateException(
"cannot downgrade a node from version [" + nodeVersion + "] to version [" + Version.CURRENT + "]");
}
return nodeVersion.equals(Version.CURRENT) ? this : new NodeMetaData(nodeId, Version.CURRENT);
}
private static class Builder {
String nodeId;
Version nodeVersion;
public void setNodeId(String nodeId) {
this.nodeId = nodeId;
}
public NodeMetaData build() {
return new NodeMetaData(nodeId);
}
public void setNodeVersionId(int nodeVersionId) {
this.nodeVersion = Version.fromId(nodeVersionId);
}
public NodeMetaData build() {
final Version nodeVersion;
if (this.nodeVersion == null) {
assert Version.CURRENT.major <= Version.V_7_0_0.major + 1 : "version is required in the node metadata from v9 onwards";
nodeVersion = Version.V_EMPTY;
} else {
nodeVersion = this.nodeVersion;
}
return new NodeMetaData(nodeId, nodeVersion);
}
}
public static final MetaDataStateFormat<NodeMetaData> FORMAT = new MetaDataStateFormat<NodeMetaData>("node-") {
@ -103,10 +142,11 @@ public final class NodeMetaData {
@Override
public void toXContent(XContentBuilder builder, NodeMetaData nodeMetaData) throws IOException {
builder.field(NODE_ID_KEY, nodeMetaData.nodeId);
builder.field(NODE_VERSION_KEY, nodeMetaData.nodeVersion.id);
}
@Override
public NodeMetaData fromXContent(XContentParser parser) throws IOException {
public NodeMetaData fromXContent(XContentParser parser) {
return PARSER.apply(parser, null).build();
}
};

12
server/src/main/java/org/elasticsearch/env/NodeRepurposeCommand.java vendored
View File

@ -172,10 +172,6 @@ public class NodeRepurposeCommand extends ElasticsearchNodeCommand {
}
}
private NodeEnvironment.NodePath[] toNodePaths(Path[] dataPaths) {
return Arrays.stream(dataPaths).map(NodeRepurposeCommand::createNodePath).toArray(NodeEnvironment.NodePath[]::new);
}
private Set<String> indexUUIDsFor(Set<Path> indexPaths) {
return indexPaths.stream().map(Path::getFileName).map(Path::toString).collect(Collectors.toSet());
}
@ -226,14 +222,6 @@ public class NodeRepurposeCommand extends ElasticsearchNodeCommand {
return Arrays.stream(paths).flatMap(Collection::stream).map(Path::getParent).collect(Collectors.toSet());
}
private static NodeEnvironment.NodePath createNodePath(Path path) {
try {
return new NodeEnvironment.NodePath(path);
} catch (IOException e) {
throw new ElasticsearchException("Unable to investigate path: " + path + ": " + e.getMessage());
}
}
//package-private for testing
OptionParser getParser() {
return parser;

103
server/src/main/java/org/elasticsearch/env/OverrideNodeVersionCommand.java vendored Normal file
View File

@ -0,0 +1,103 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.env;
import joptsimple.OptionParser;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.Version;
import org.elasticsearch.cli.Terminal;
import org.elasticsearch.cluster.coordination.ElasticsearchNodeCommand;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Arrays;
public class OverrideNodeVersionCommand extends ElasticsearchNodeCommand {
private static final Logger logger = LogManager.getLogger(OverrideNodeVersionCommand.class);
private static final String TOO_NEW_MESSAGE =
DELIMITER +
"\n" +
"This data path was last written by Elasticsearch version [V_NEW] and may no\n" +
"longer be compatible with Elasticsearch version [V_CUR]. This tool will bypass\n" +
"this compatibility check, allowing a version [V_CUR] node to start on this data\n" +
"path, but a version [V_CUR] node may not be able to read this data or may read\n" +
"it incorrectly leading to data loss.\n" +
"\n" +
"You should not use this tool. Instead, continue to use a version [V_NEW] node\n" +
"on this data path. If necessary, you can use reindex-from-remote to copy the\n" +
"data from here into an older cluster.\n" +
"\n" +
"Do you want to proceed?\n";
private static final String TOO_OLD_MESSAGE =
DELIMITER +
"\n" +
"This data path was last written by Elasticsearch version [V_OLD] which may be\n" +
"too old to be readable by Elasticsearch version [V_CUR]. This tool will bypass\n" +
"this compatibility check, allowing a version [V_CUR] node to start on this data\n" +
"path, but this version [V_CUR] node may not be able to read this data or may\n" +
"read it incorrectly leading to data loss.\n" +
"\n" +
"You should not use this tool. Instead, upgrade this data path from [V_OLD] to\n" +
"[V_CUR] using one or more intermediate versions of Elasticsearch.\n" +
"\n" +
"Do you want to proceed?\n";
static final String NO_METADATA_MESSAGE = "no node metadata found, so there is no version to override";
static final String SUCCESS_MESSAGE = "Successfully overwrote this node's metadata to bypass its version compatibility checks.";
public OverrideNodeVersionCommand() {
super("Overwrite the version stored in this node's data path with [" + Version.CURRENT +
"] to bypass the version compatibility checks");
}
@Override
protected void processNodePaths(Terminal terminal, Path[] dataPaths, Environment env) throws IOException {
final Path[] nodePaths = Arrays.stream(toNodePaths(dataPaths)).map(p -> p.path).toArray(Path[]::new);
final NodeMetaData nodeMetaData = NodeMetaData.FORMAT.loadLatestState(logger, namedXContentRegistry, nodePaths);
if (nodeMetaData == null) {
throw new ElasticsearchException(NO_METADATA_MESSAGE);
}
try {
nodeMetaData.upgradeToCurrentVersion();
throw new ElasticsearchException("found [" + nodeMetaData + "] which is compatible with current version [" + Version.CURRENT
+ "], so there is no need to override the version checks");
} catch (IllegalStateException e) {
// ok, means the version change is not supported
}
confirm(terminal, (nodeMetaData.nodeVersion().before(Version.CURRENT) ? TOO_OLD_MESSAGE : TOO_NEW_MESSAGE)
.replace("V_OLD", nodeMetaData.nodeVersion().toString())
.replace("V_NEW", nodeMetaData.nodeVersion().toString())
.replace("V_CUR", Version.CURRENT.toString()));
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(nodeMetaData.nodeId(), Version.CURRENT), nodePaths);
terminal.println(SUCCESS_MESSAGE);
}
//package-private for testing
OptionParser getParser() {
return parser;
}
}

4
server/src/main/java/org/elasticsearch/gateway/MetaDataStateFormat.java
View File

@ -382,7 +382,7 @@ public abstract class MetaDataStateFormat<T> {
return files;
}
private String getStateFileName(long generation) {
public String getStateFileName(long generation) {
return prefix + generation + STATE_FILE_EXTENSION;
}
@ -466,7 +466,7 @@ public abstract class MetaDataStateFormat<T> {
IOUtils.rm(stateDirectories);
}
String getPrefix() {
public String getPrefix() {
return prefix;
}
}

37
server/src/test/java/org/elasticsearch/env/NodeEnvironmentIT.java vendored
View File

@ -19,12 +19,18 @@
package org.elasticsearch.env;
import org.elasticsearch.Version;
import org.elasticsearch.common.CheckedConsumer;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.Node;
import org.elasticsearch.test.ESIntegTestCase;
import org.elasticsearch.test.InternalTestCluster;
import java.nio.file.Path;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.endsWith;
import static org.hamcrest.Matchers.startsWith;
@ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.TEST, numDataNodes = 0)
@ -86,4 +92,35 @@ public class NodeEnvironmentIT extends ESIntegTestCase {
+ Node.NODE_DATA_SETTING.getKey()
+ "=false, but has shard data"));
}
private IllegalStateException expectThrowsOnRestart(CheckedConsumer<Path[], Exception> onNodeStopped) {
internalCluster().startNode();
final Path[] dataPaths = internalCluster().getInstance(NodeEnvironment.class).nodeDataPaths();
return expectThrows(IllegalStateException.class,
() -> internalCluster().restartRandomDataNode(new InternalTestCluster.RestartCallback() {
@Override
public Settings onNodeStopped(String nodeName) {
try {
onNodeStopped.accept(dataPaths);
} catch (Exception e) {
throw new AssertionError(e);
}
return Settings.EMPTY;
}
}));
}
public void testFailsToStartIfDowngraded() {
final IllegalStateException illegalStateException = expectThrowsOnRestart(dataPaths ->
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(randomAlphaOfLength(10), NodeMetaDataTests.tooNewVersion()), dataPaths));
assertThat(illegalStateException.getMessage(),
allOf(startsWith("cannot downgrade a node from version ["), endsWith("] to version [" + Version.CURRENT + "]")));
}
public void testFailsToStartIfUpgradedTooFar() {
final IllegalStateException illegalStateException = expectThrowsOnRestart(dataPaths ->
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(randomAlphaOfLength(10), NodeMetaDataTests.tooOldVersion()), dataPaths));
assertThat(illegalStateException.getMessage(),
allOf(startsWith("cannot upgrade a node from version ["), endsWith("] directly to version [" + Version.CURRENT + "]")));
}
}

118
server/src/test/java/org/elasticsearch/env/NodeMetaDataTests.java vendored Normal file
View File

@ -0,0 +1,118 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.env;
import org.elasticsearch.Version;
import org.elasticsearch.common.collect.Tuple;
import org.elasticsearch.gateway.MetaDataStateFormat;
import org.elasticsearch.test.ESTestCase;
import org.elasticsearch.test.EqualsHashCodeTestUtils;
import org.elasticsearch.test.VersionUtils;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.endsWith;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.notNullValue;
import static org.hamcrest.Matchers.startsWith;
public class NodeMetaDataTests extends ESTestCase {
private Version randomVersion() {
// VersionUtils.randomVersion() only returns known versions, which are necessarily no later than Version.CURRENT; however we want
// also to consider our behaviour with all versions, so occasionally pick up a truly random version.
return rarely() ? Version.fromId(randomInt()) : VersionUtils.randomVersion(random());
}
public void testEqualsHashcodeSerialization() {
final Path tempDir = createTempDir();
EqualsHashCodeTestUtils.checkEqualsAndHashCode(new NodeMetaData(randomAlphaOfLength(10), randomVersion()),
nodeMetaData -> {
final long generation = NodeMetaData.FORMAT.writeAndCleanup(nodeMetaData, tempDir);
final Tuple<NodeMetaData, Long> nodeMetaDataLongTuple
= NodeMetaData.FORMAT.loadLatestStateWithGeneration(logger, xContentRegistry(), tempDir);
assertThat(nodeMetaDataLongTuple.v2(), equalTo(generation));
return nodeMetaDataLongTuple.v1();
}, nodeMetaData -> {
if (randomBoolean()) {
return new NodeMetaData(randomAlphaOfLength(21 - nodeMetaData.nodeId().length()), nodeMetaData.nodeVersion());
} else {
return new NodeMetaData(nodeMetaData.nodeId(), randomValueOtherThan(nodeMetaData.nodeVersion(), this::randomVersion));
}
});
}
public void testReadsFormatWithoutVersion() throws IOException {
// the behaviour tested here is only appropriate if the current version is compatible with versions 7 and earlier
assertTrue(Version.CURRENT.minimumIndexCompatibilityVersion().onOrBefore(Version.V_7_0_0));
// when the current version is incompatible with version 7, the behaviour should change to reject files like the given resource
// which do not have the version field
final Path tempDir = createTempDir();
final Path stateDir = Files.createDirectory(tempDir.resolve(MetaDataStateFormat.STATE_DIR_NAME));
final InputStream resource = this.getClass().getResourceAsStream("testReadsFormatWithoutVersion.binary");
assertThat(resource, notNullValue());
Files.copy(resource, stateDir.resolve(NodeMetaData.FORMAT.getStateFileName(between(0, Integer.MAX_VALUE))));
final NodeMetaData nodeMetaData = NodeMetaData.FORMAT.loadLatestState(logger, xContentRegistry(), tempDir);
assertThat(nodeMetaData.nodeId(), equalTo("y6VUVMSaStO4Tz-B5BxcOw"));
assertThat(nodeMetaData.nodeVersion(), equalTo(Version.V_EMPTY));
}
public void testUpgradesLegitimateVersions() {
final String nodeId = randomAlphaOfLength(10);
final NodeMetaData nodeMetaData = new NodeMetaData(nodeId,
randomValueOtherThanMany(v -> v.after(Version.CURRENT) || v.before(Version.CURRENT.minimumIndexCompatibilityVersion()),
this::randomVersion)).upgradeToCurrentVersion();
assertThat(nodeMetaData.nodeVersion(), equalTo(Version.CURRENT));
assertThat(nodeMetaData.nodeId(), equalTo(nodeId));
}
public void testUpgradesMissingVersion() {
final String nodeId = randomAlphaOfLength(10);
final NodeMetaData nodeMetaData = new NodeMetaData(nodeId, Version.V_EMPTY).upgradeToCurrentVersion();
assertThat(nodeMetaData.nodeVersion(), equalTo(Version.CURRENT));
assertThat(nodeMetaData.nodeId(), equalTo(nodeId));
}
public void testDoesNotUpgradeFutureVersion() {
final IllegalStateException illegalStateException = expectThrows(IllegalStateException.class,
() -> new NodeMetaData(randomAlphaOfLength(10), tooNewVersion())
.upgradeToCurrentVersion());
assertThat(illegalStateException.getMessage(),
allOf(startsWith("cannot downgrade a node from version ["), endsWith("] to version [" + Version.CURRENT + "]")));
}
public void testDoesNotUpgradeAncientVersion() {
final IllegalStateException illegalStateException = expectThrows(IllegalStateException.class,
() -> new NodeMetaData(randomAlphaOfLength(10), tooOldVersion()).upgradeToCurrentVersion());
assertThat(illegalStateException.getMessage(),
allOf(startsWith("cannot upgrade a node from version ["), endsWith("] directly to version [" + Version.CURRENT + "]")));
}
public static Version tooNewVersion() {
return Version.fromId(between(Version.CURRENT.id + 1, 99999999));
}
public static Version tooOldVersion() {
return Version.fromId(between(1, Version.CURRENT.minimumIndexCompatibilityVersion().id - 1));
}
}

155
server/src/test/java/org/elasticsearch/env/OverrideNodeVersionCommandTests.java vendored Normal file
View File

@ -0,0 +1,155 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.env;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.Version;
import org.elasticsearch.cli.MockTerminal;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.gateway.WriteStateException;
import org.elasticsearch.test.ESTestCase;
import org.junit.Before;
import java.io.IOException;
import java.nio.file.Path;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
public class OverrideNodeVersionCommandTests extends ESTestCase {
private Environment environment;
private Path[] nodePaths;
@Before
public void createNodePaths() throws IOException {
final Settings settings = buildEnvSettings(Settings.EMPTY);
environment = TestEnvironment.newEnvironment(settings);
try (NodeEnvironment nodeEnvironment = new NodeEnvironment(settings, environment)) {
nodePaths = nodeEnvironment.nodeDataPaths();
}
}
public void testFailsOnEmptyPath() {
final Path emptyPath = createTempDir();
final MockTerminal mockTerminal = new MockTerminal();
final ElasticsearchException elasticsearchException = expectThrows(ElasticsearchException.class, () ->
new OverrideNodeVersionCommand().processNodePaths(mockTerminal, new Path[]{emptyPath}, environment));
assertThat(elasticsearchException.getMessage(), equalTo(OverrideNodeVersionCommand.NO_METADATA_MESSAGE));
expectThrows(IllegalStateException.class, () -> mockTerminal.readText(""));
}
public void testFailsIfUnnecessary() throws WriteStateException {
final Version nodeVersion = Version.fromId(between(Version.CURRENT.minimumIndexCompatibilityVersion().id, Version.CURRENT.id));
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(randomAlphaOfLength(10), nodeVersion), nodePaths);
final MockTerminal mockTerminal = new MockTerminal();
final ElasticsearchException elasticsearchException = expectThrows(ElasticsearchException.class, () ->
new OverrideNodeVersionCommand().processNodePaths(mockTerminal, nodePaths, environment));
assertThat(elasticsearchException.getMessage(), allOf(
containsString("compatible with current version"),
containsString(Version.CURRENT.toString()),
containsString(nodeVersion.toString())));
expectThrows(IllegalStateException.class, () -> mockTerminal.readText(""));
}
public void testWarnsIfTooOld() throws Exception {
final String nodeId = randomAlphaOfLength(10);
final Version nodeVersion = NodeMetaDataTests.tooOldVersion();
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(nodeId, nodeVersion), nodePaths);
final MockTerminal mockTerminal = new MockTerminal();
mockTerminal.addTextInput("n\n");
final ElasticsearchException elasticsearchException = expectThrows(ElasticsearchException.class, () ->
new OverrideNodeVersionCommand().processNodePaths(mockTerminal, nodePaths, environment));
assertThat(elasticsearchException.getMessage(), equalTo("aborted by user"));
assertThat(mockTerminal.getOutput(), allOf(
containsString("too old"),
containsString("data loss"),
containsString("You should not use this tool"),
containsString(Version.CURRENT.toString()),
containsString(nodeVersion.toString())));
expectThrows(IllegalStateException.class, () -> mockTerminal.readText(""));
final NodeMetaData nodeMetaData = NodeMetaData.FORMAT.loadLatestState(logger, xContentRegistry(), nodePaths);
assertThat(nodeMetaData.nodeId(), equalTo(nodeId));
assertThat(nodeMetaData.nodeVersion(), equalTo(nodeVersion));
}
public void testWarnsIfTooNew() throws Exception {
final String nodeId = randomAlphaOfLength(10);
final Version nodeVersion = NodeMetaDataTests.tooNewVersion();
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(nodeId, nodeVersion), nodePaths);
final MockTerminal mockTerminal = new MockTerminal();
mockTerminal.addTextInput(randomFrom("yy", "Yy", "n", "yes", "true", "N", "no"));
final ElasticsearchException elasticsearchException = expectThrows(ElasticsearchException.class, () ->
new OverrideNodeVersionCommand().processNodePaths(mockTerminal, nodePaths, environment));
assertThat(elasticsearchException.getMessage(), equalTo("aborted by user"));
assertThat(mockTerminal.getOutput(), allOf(
containsString("data loss"),
containsString("You should not use this tool"),
containsString(Version.CURRENT.toString()),
containsString(nodeVersion.toString())));
expectThrows(IllegalStateException.class, () -> mockTerminal.readText(""));
final NodeMetaData nodeMetaData = NodeMetaData.FORMAT.loadLatestState(logger, xContentRegistry(), nodePaths);
assertThat(nodeMetaData.nodeId(), equalTo(nodeId));
assertThat(nodeMetaData.nodeVersion(), equalTo(nodeVersion));
}
public void testOverwritesIfTooOld() throws Exception {
final String nodeId = randomAlphaOfLength(10);
final Version nodeVersion = NodeMetaDataTests.tooOldVersion();
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(nodeId, nodeVersion), nodePaths);
final MockTerminal mockTerminal = new MockTerminal();
mockTerminal.addTextInput(randomFrom("y", "Y"));
new OverrideNodeVersionCommand().processNodePaths(mockTerminal, nodePaths, environment);
assertThat(mockTerminal.getOutput(), allOf(
containsString("too old"),
containsString("data loss"),
containsString("You should not use this tool"),
containsString(Version.CURRENT.toString()),
containsString(nodeVersion.toString()),
containsString(OverrideNodeVersionCommand.SUCCESS_MESSAGE)));
expectThrows(IllegalStateException.class, () -> mockTerminal.readText(""));
final NodeMetaData nodeMetaData = NodeMetaData.FORMAT.loadLatestState(logger, xContentRegistry(), nodePaths);
assertThat(nodeMetaData.nodeId(), equalTo(nodeId));
assertThat(nodeMetaData.nodeVersion(), equalTo(Version.CURRENT));
}
public void testOverwritesIfTooNew() throws Exception {
final String nodeId = randomAlphaOfLength(10);
final Version nodeVersion = NodeMetaDataTests.tooNewVersion();
NodeMetaData.FORMAT.writeAndCleanup(new NodeMetaData(nodeId, nodeVersion), nodePaths);
final MockTerminal mockTerminal = new MockTerminal();
mockTerminal.addTextInput(randomFrom("y", "Y"));
new OverrideNodeVersionCommand().processNodePaths(mockTerminal, nodePaths, environment);
assertThat(mockTerminal.getOutput(), allOf(
containsString("data loss"),
containsString("You should not use this tool"),
containsString(Version.CURRENT.toString()),
containsString(nodeVersion.toString()),
containsString(OverrideNodeVersionCommand.SUCCESS_MESSAGE)));
expectThrows(IllegalStateException.class, () -> mockTerminal.readText(""));
final NodeMetaData nodeMetaData = NodeMetaData.FORMAT.loadLatestState(logger, xContentRegistry(), nodePaths);
assertThat(nodeMetaData.nodeId(), equalTo(nodeId));
assertThat(nodeMetaData.nodeVersion(), equalTo(Version.CURRENT));
}
}

BIN
server/src/test/resources/org/elasticsearch/env/testReadsFormatWithoutVersion.binary vendored Normal file
View File

Binary file not shown.