From 5dc0de04fb2fac772b08c2b439cc5ce8489ad47a Mon Sep 17 00:00:00 2001 From: Adam Locke Date: Tue, 15 Sep 2020 12:02:43 -0400 Subject: [PATCH] [DOCS] Clarifying remote clusters based on feedback from Support (#62335) (#62394) * Clarifying remote clusters based on feedback from Support. * Apply suggestions from code review * Making additional editorial changes. --- .../modules/remote-clusters.asciidoc | 191 +++++++++--------- 1 file changed, 97 insertions(+), 94 deletions(-) diff --git a/docs/reference/modules/remote-clusters.asciidoc b/docs/reference/modules/remote-clusters.asciidoc index 78a8cfd3a2a..c44e81678b3 100644 --- a/docs/reference/modules/remote-clusters.asciidoc +++ b/docs/reference/modules/remote-clusters.asciidoc @@ -1,23 +1,29 @@ [[modules-remote-clusters]] == Remote clusters -The _remote clusters_ functionality enables you to establish unidirectional -connections to a remote cluster. Remote clusters are required for -<> and <>. +You can connect a local cluster to other {es} clusters, known as _remote +clusters_. Once connected, you can search remote clusters using +<>. You can also sync data between clusters +using <>. -Remote cluster connections work by configuring a remote cluster and connecting -to a limited number of nodes in that remote cluster. There are two modes for -remote cluster connections: <> and -<>. +To register a remote cluster, connect the local cluster to nodes in the +remote cluster using one of two connection modes: -Communication with a remote cluster uses the <> to establish a number of <> TCP -connections from the coordinating nodes of the local cluster to the chosen -nodes in the remote cluster. +* <> +* <> + +Your local cluster uses the <> to establish +communication with remote clusters. The coordinating nodes in the local cluster +establish <> TCP connections with specific +nodes in the remote cluster. {es} requires these connections to remain open, +even if the connections are idle for an extended period. + +You can use the <> to get +information about registered remote clusters. -[discrete] [[sniff-mode]] -=== Sniff mode +[discrete] +==== Sniff mode In sniff mode, a cluster is created using a name and a list of seed nodes. When a remote cluster is registered, its cluster state is retrieved from one of the @@ -27,22 +33,25 @@ are accessible by the local cluster. Sniff mode is the default connection mode. -[discrete] [[gateway-nodes-selection]] -==== Gateway nodes selection - The _gateway nodes_ selection depends on the following criteria: -- *version*: Remote nodes must be compatible with the cluster they are -registered to. This is subject to rules that are similar to those for -<>. Any node can communicate with any other node on the same -major version (e.g. 7.0 can talk to any 7.x node). Only nodes on the last minor -version of a certain major version can communicate with nodes on the following -major version. Note that in the 6.x series, 6.8 can communicate with any 7.x -node, while 6.7 can only communicate with 7.0. Version compatibility is +* *version*: Remote nodes must be compatible with the cluster they are +registered to, similar to the rules for +<>: +** Any node can communicate with another node on the same +major version. For example, 7.0 can talk to any 7.x node. +** Only nodes on the last minor version of a certain major version can +communicate with nodes on the following major version. In the 6.x series, 6.8 +can communicate with any 7.x node, while 6.7 can only communicate with 7.0. +** Version compatibility is symmetric, meaning that if 6.7 can communicate with 7.0, 7.0 can also -communicate with 6.7. The matrix below summarizes compatibility as described above. - +communicate with 6.7. The following table depicts version compatibility between +local and remote nodes. ++ +[%collapsible] +.Version compatibility table +==== // tag::remote-cluster-compatibility-matrix[] [cols="^,^,^,^,^,^,^,^"] |==== @@ -57,21 +66,22 @@ h| Remote cluster | 5.0->5.5 | 5.6 | 6.0->6.6 | 6.7 | 6.8 | 7.0 | 7.1->7.x | 7.1->7.x | {no-icon} | {no-icon} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} |==== // end::remote-cluster-compatibility-matrix[] +==== -- *role*: Dedicated master nodes never get selected. -- *attributes*: You can tag which nodes should be selected +* *role*: Dedicated master nodes are never selected as gateway nodes. +* *attributes*: You can tag which nodes should be selected (see <>), though such tagged nodes still have to satisfy the two above requirements. -[discrete] [[proxy-mode]] -=== Proxy mode +[discrete] +==== Proxy mode -In proxy mode, a cluster is created using a name and a single proxy address. When -a remote cluster is registered, a configurable number of socket connections are -opened to the proxy address. The proxy is required to route those connections to -the remote cluster. Proxy mode does not require remote cluster nodes to have -accessible publish addresses. +In proxy mode, a cluster is created using a name and a single proxy address. +When you register a remote cluster, a configurable number of socket connections +are opened to the proxy address. The proxy is required to route those +connections to the remote cluster. Proxy mode does not require remote cluster +nodes to have accessible publish addresses. The proxy mode is not the default connection mode and must be configured. Similar to the sniff <>, the remote @@ -80,55 +90,17 @@ connections are subject to the same version compatibility rules as [discrete] [[configuring-remote-clusters]] -==== Configuring remote clusters +=== Configuring remote clusters -You can configure remote clusters globally by using -<>, which you can update dynamically. -Alternatively, you can configure them locally on individual nodes by using the - `elasticsearch.yml` file. +You can configure remote clusters settings <>, or configure +settings <> in the +`elasticsearch.yml` file. -If you specify the settings in `elasticsearch.yml` files, only the nodes with -those settings can connect to the remote cluster. In other words, functionality -that relies on remote cluster requests must be driven specifically from those -nodes. For example: - -[source,yaml] --------------------------------- -cluster: - remote: - cluster_one: <1> - seeds: 127.0.0.1:9300 <2> - transport.ping_schedule: 30s <3> - cluster_two: <1> - mode: sniff <4> - seeds: 127.0.0.1:9301 <2> - transport.compress: true <5> - skip_unavailable: true <6> - cluster_three: <1> - mode: proxy <4> - proxy_address: 127.0.0.1:9302 <7> - --------------------------------- -<1> `cluster_one`, `cluster_two`, and `cluster_three` are arbitrary _cluster aliases_ -representing the connection to each cluster. These names are subsequently used to -distinguish between local and remote indices. -<2> The hostname and <> port (default: 9300) of a -seed node in the remote cluster. -<3> A keep-alive ping is configured for `cluster_one`. -<4> The configured connection mode. By default, this is <>, so -the mode is implicit for `cluster_one`. However, it can be explicitly configured -as demonstrated by `cluster_two` and must be explicitly configured for -<> as demonstrated by `cluster_three`. -<5> Compression is explicitly enabled for requests to `cluster_two`. -<6> Disconnected remote clusters are optional for `cluster_two`. -<7> The address for the proxy endpoint used to connect to `cluster_three`. - -For more information about the optional transport settings, see -<>. - - -If you use <>, the remote clusters -are available on every node in the cluster. For example: +[discrete] +[[configure-remote-clusters-dynamic]] +===== Dynamically configure remote clusters +Use the <> to dynamically +configure remote settings on every node in the cluster. For example: [source,console] -------------------------------- @@ -203,7 +175,8 @@ NOTE: When the compression or ping schedule settings change, all the existing node connections must close and re-open, which can cause in-flight requests to fail. -A remote cluster can be deleted from the cluster settings by setting its settings to `null` : +You can delete a remote cluster from the cluster settings by passing `null` +values for each remote cluster setting: [source,console] -------------------------------- @@ -230,13 +203,51 @@ PUT _cluster/settings <1> `cluster_two` would be removed from the cluster settings, leaving `cluster_one` and `cluster_three` intact. +[discrete] +[[configure-remote-clusters-static]] +===== Statically configure remote clusters +If you specify settings in `elasticsearch.yml` files, only the nodes with +those settings can connect to the remote cluster and serve remote cluster requests. For example: + +[source,yaml] +-------------------------------- +cluster: + remote: + cluster_one: <1> + seeds: 127.0.0.1:9300 <2> + transport.ping_schedule: 30s <3> + cluster_two: <1> + mode: sniff <4> + seeds: 127.0.0.1:9301 <2> + transport.compress: true <5> + skip_unavailable: true <6> + cluster_three: <1> + mode: proxy <4> + proxy_address: 127.0.0.1:9302 <7> + +-------------------------------- +<1> `cluster_one`, `cluster_two`, and `cluster_three` are arbitrary _cluster aliases_ +representing the connection to each cluster. These names are subsequently used to +distinguish between local and remote indices. +<2> The hostname and <> port (default: 9300) of a +seed node in the remote cluster. +<3> A keep-alive ping is configured for `cluster_one`. +<4> The configured connection mode. By default, this is <>, so +the mode is implicit for `cluster_one`. However, it can be explicitly configured +as demonstrated by `cluster_two` and must be explicitly configured for +<> as demonstrated by `cluster_three`. +<5> Compression is explicitly enabled for requests to `cluster_two`. +<6> Disconnected remote clusters are optional for `cluster_two`. +<7> The address for the proxy endpoint used to connect to `cluster_three`. + [discrete] [[remote-cluster-settings]] -=== Remote cluster settings for all modes +=== Global remote cluster settings These settings apply to both <> and <>. <> -and <> are described below. +and <> are described +separately. `cluster.remote..mode`:: The mode used for a remote cluster connection. The only supported modes are @@ -282,7 +293,7 @@ and <> are described below. [discrete] [[remote-cluster-sniff-settings]] -=== Remote cluster settings for sniff mode +=== Sniff mode remote cluster settings `cluster.remote..seeds`:: @@ -302,7 +313,7 @@ and <> are described below. [discrete] [[remote-cluster-proxy-settings]] -=== Remote cluster settings for proxy mode +=== Proxy mode remote cluster settings `cluster.remote..proxy_address`:: @@ -321,11 +332,3 @@ and <> are described below. <>. The TLS transport will fail to open remote connections if this field is not a valid hostname as defined by the TLS SNI specification. - -[discrete] -[[retrieve-remote-clusters-info]] -=== Retrieving remote clusters info - -You can use the <> to retrieve -information about the configured remote clusters, as well as the remote nodes -that the node is connected to.