honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Make BlobStoreRepository Validation Read master.dat (#45546) (#45578) 

* Fixing this for two reasons:
   1. Why not verify that the seed we wrote is actually there when we can
   2. The AWS S3 SDK started to log a bunch of WARN messages about not fully reading the stream now that we started to abuse the read blob as an `exists` check after removing that method from the blob container
This commit is contained in:
Armin Braun 2019-08-15 07:07:52 +02:00 committed by GitHub
parent ad3d8c59bd
commit 1beea3588b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 14 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
server/src/main/java/org/elasticsearch/repositories/blobstore/BlobStoreRepository.java
View File

@ -1059,15 +1059,20 @@ public abstract class BlobStoreRepository extends AbstractLifecycleComponent imp
}
} else {
BlobContainer testBlobContainer = blobStore().blobContainer(basePath().add(testBlobPrefix(seed)));
try (InputStream ignored = testBlobContainer.readBlob("master.dat")) {
try {
BytesArray bytes = new BytesArray(seed);
try (InputStream stream = bytes.streamInput()) {
testBlobContainer.writeBlob("data-" + localNode.getId() + ".dat", stream, bytes.length(), true);
}
} catch (IOException exp) {
throw new RepositoryVerificationException(metadata.name(), "store location [" + blobStore() +
"] is not accessible on the node [" + localNode + "]", exp);
try {
BytesArray bytes = new BytesArray(seed);
try (InputStream stream = bytes.streamInput()) {
testBlobContainer.writeBlob("data-" + localNode.getId() + ".dat", stream, bytes.length(), true);
}
} catch (IOException exp) {
throw new RepositoryVerificationException(metadata.name(), "store location [" + blobStore() +
"] is not accessible on the node [" + localNode + "]", exp);
}
try (InputStream masterDat = testBlobContainer.readBlob("master.dat")) {
final String seedRead = Streams.readFully(masterDat).utf8ToString();
if (seedRead.equals(seed) == false) {
throw new RepositoryVerificationException(metadata.name(), "Seed read from master.dat was [" + seedRead +
"] but expected seed [" + seed + "]");
}
} catch (NoSuchFileException e) {
throw new RepositoryVerificationException(metadata.name(), "a file written by master to the store [" + blobStore() +