honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Third-party S3-compatible repos aren't unsupported (#63238) 

Today in the `repository-s3` docs we say

> Other S3-compatible storage systems may also work with Elasticsearch,
> but these are not tested or supported.

Saying that they are explicitly not supported is a very strong
statement, implying that it is positively irresponsible to use anything
except S3 or Minio, even after extensive testing. S3-compatibility in
third-party systems has matured in recent years and users today report
success with a good number of them. In contrast, we effectively claim
support for any old NFS implementation when used with the
shared-filesystem repository.

This commit weakens this statement, removing the absolute claim of
unsupportedness and instead spelling out that the user is responsible
for ironing out any incompatibilities with the storage supplier.
This commit is contained in:
David Turner 2020-10-12 09:37:52 +01:00
parent 56092b1a9f
commit 6ddeaa4791
1 changed files with 16 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
docs/plugins/repository-s3.asciidoc
View File

@ -215,8 +215,19 @@ include a port. For example, the endpoint may be `172.17.0.2` or
https://minio.io[Minio] is an example of a storage system that provides an https://minio.io[Minio] is an example of a storage system that provides an
S3-compatible API. The `repository-s3` plugin allows {es} to work with S3-compatible API. The `repository-s3` plugin allows {es} to work with
Minio-backed repositories as well as repositories stored on AWS S3. Other Minio-backed repositories as well as repositories stored on AWS S3. Other
S3-compatible storage systems may also work with {es}, but these are not tested S3-compatible storage systems may also work with {es}, but these are not
or supported. covered by the {es} test suite.
Note that some storage systems claim to be S3-compatible without correctly
supporting the full S3 API. The `repository-s3` plugin requires full
compatibility with S3. In particular it must support the same set of API
endpoints, return the same errors in case of failures, and offer a consistency
model no weaker than S3's when accessed concurrently by multiple nodes. If you
wish to use another storage system with the `repository-s3` plugin then you
will need to work with the supplier of the storage system to address any
incompatibilities you encounter. Incompatible error codes and consistency
models may be particularly hard to track down since errors and consistency
failures are usually rare and hard to reproduce.
[[repository-s3-repository]] [[repository-s3-repository]]
==== Repository Settings ==== Repository Settings
@ -444,8 +455,9 @@ go through the VPC's NAT instance. If your VPC's NAT instance is a smaller
instance size (e.g. a t2.micro) or is handling a high volume of network traffic instance size (e.g. a t2.micro) or is handling a high volume of network traffic
your bandwidth to S3 may be limited by that NAT instance's networking bandwidth your bandwidth to S3 may be limited by that NAT instance's networking bandwidth
limitations. Instead we recommend creating a https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html[VPC endpoint] limitations. Instead we recommend creating a https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html[VPC endpoint]
that enables connecting to S3 in instances that reside in a private subnet in an that enables connecting to S3 in instances that reside in a private subnet in
AWS VPC. This will eliminate any limitations imposed by the network bandwidth of your VPC's NAT instance. an AWS VPC. This will eliminate any limitations imposed by the network
bandwidth of your VPC's NAT instance.
Instances residing in a public subnet in an AWS VPC will connect to S3 via the Instances residing in a public subnet in an AWS VPC will connect to S3 via the
VPC's internet gateway and not be bandwidth limited by the VPC's NAT instance. VPC's internet gateway and not be bandwidth limited by the VPC's NAT instance.