156 lines
7.0 KiB
Plaintext
156 lines
7.0 KiB
Plaintext
[[modules-discovery-hosts-providers]]
|
|
=== Discovery
|
|
|
|
Discovery is the process by which the cluster formation module finds other
|
|
nodes with which to form a cluster. This process runs when you start an
|
|
Elasticsearch node or when a node believes the master node failed and continues
|
|
until the master node is found or a new master node is elected.
|
|
|
|
This process starts with a list of _seed_ addresses from one or more
|
|
<<built-in-hosts-providers,seed hosts providers>>, together with the addresses
|
|
of any master-eligible nodes that were in the last-known cluster. The process
|
|
operates in two phases: First, each node probes the seed addresses by
|
|
connecting to each address and attempting to identify the node to which it is
|
|
connected and to verify that it is master-eligible. Secondly, if successful, it
|
|
shares with the remote node a list of all of its known master-eligible peers
|
|
and the remote node responds with _its_ peers in turn. The node then probes all
|
|
the new nodes that it just discovered, requests their peers, and so on.
|
|
|
|
If the node is not master-eligible then it continues this discovery process
|
|
until it has discovered an elected master node. If no elected master is
|
|
discovered then the node will retry after `discovery.find_peers_interval` which
|
|
defaults to `1s`.
|
|
|
|
If the node is master-eligible then it continues this discovery process until
|
|
it has either discovered an elected master node or else it has discovered
|
|
enough masterless master-eligible nodes to complete an election. If neither of
|
|
these occur quickly enough then the node will retry after
|
|
`discovery.find_peers_interval` which defaults to `1s`.
|
|
|
|
[[built-in-hosts-providers]]
|
|
==== Seed hosts providers
|
|
|
|
By default the cluster formation module offers two seed hosts providers to
|
|
configure the list of seed nodes: a _settings_-based and a _file_-based seed
|
|
hosts provider. It can be extended to support cloud environments and other
|
|
forms of seed hosts providers via {plugins}/discovery.html[discovery plugins].
|
|
Seed hosts providers are configured using the `discovery.seed_providers`
|
|
setting, which defaults to the _settings_-based hosts provider. This setting
|
|
accepts a list of different providers, allowing you to make use of multiple
|
|
ways to find the seed hosts for your cluster.
|
|
|
|
Each seed hosts provider yields the IP addresses or hostnames of the seed
|
|
nodes. If it returns any hostnames then these are resolved to IP addresses
|
|
using a DNS lookup. If a hostname resolves to multiple IP addresses then {es}
|
|
tries to find a seed node at all of these addresses. If the hosts provider does
|
|
not explicitly give the TCP port of the node by then, it will implicitly use the
|
|
first port in the port range given by `transport.profiles.default.port`, or by
|
|
`transport.port` if `transport.profiles.default.port` is not set. The number of
|
|
concurrent lookups is controlled by
|
|
`discovery.seed_resolver.max_concurrent_resolvers` which defaults to `10`, and
|
|
the timeout for each lookup is controlled by `discovery.seed_resolver.timeout`
|
|
which defaults to `5s`. Note that DNS lookups are subject to
|
|
<<networkaddress-cache-ttl,JVM DNS caching>>.
|
|
|
|
[discrete]
|
|
[[settings-based-hosts-provider]]
|
|
===== Settings-based seed hosts provider
|
|
|
|
The settings-based seed hosts provider uses a node setting to configure a
|
|
static list of the addresses of the seed nodes. These addresses can be given as
|
|
hostnames or IP addresses; hosts specified as hostnames are resolved to IP
|
|
addresses during each round of discovery.
|
|
|
|
The list of hosts is set using the <<unicast.hosts,`discovery.seed_hosts`>>
|
|
static setting. For example:
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
discovery.seed_hosts:
|
|
- 192.168.1.10:9300
|
|
- 192.168.1.11 <1>
|
|
- seeds.mydomain.com <2>
|
|
--------------------------------------------------
|
|
<1> The port will default to `transport.profiles.default.port` and fallback to
|
|
`transport.port` if not specified.
|
|
<2> If a hostname resolves to multiple IP addresses, {es} will attempt to
|
|
connect to every resolved address.
|
|
|
|
[discrete]
|
|
[[file-based-hosts-provider]]
|
|
===== File-based seed hosts provider
|
|
|
|
The file-based seed hosts provider configures a list of hosts via an external
|
|
file. {es} reloads this file when it changes, so that the list of seed nodes
|
|
can change dynamically without needing to restart each node. For example, this
|
|
gives a convenient mechanism for an {es} instance that is run in a Docker
|
|
container to be dynamically supplied with a list of IP addresses to connect to
|
|
when those IP addresses may not be known at node startup.
|
|
|
|
To enable file-based discovery, configure the `file` hosts provider as follows
|
|
in the `elasticsearch.yml` file:
|
|
|
|
[source,yml]
|
|
----------------------------------------------------------------
|
|
discovery.seed_providers: file
|
|
----------------------------------------------------------------
|
|
|
|
Then create a file at `$ES_PATH_CONF/unicast_hosts.txt` in the format described
|
|
below. Any time a change is made to the `unicast_hosts.txt` file the new
|
|
changes will be picked up by {es} and the new hosts list will be used.
|
|
|
|
Note that the file-based discovery plugin augments the unicast hosts list in
|
|
`elasticsearch.yml`: if there are valid seed addresses in
|
|
`discovery.seed_hosts` then {es} uses those addresses in addition to those
|
|
supplied in `unicast_hosts.txt`.
|
|
|
|
The `unicast_hosts.txt` file contains one node entry per line. Each node entry
|
|
consists of the host (host name or IP address) and an optional transport port
|
|
number. If the port number is specified, is must come immediately after the
|
|
host (on the same line) separated by a `:`. If the port number is not
|
|
specified, {es} will implicitly use the first port in the port range given by
|
|
`transport.profiles.default.port`, or by `transport.port` if
|
|
`transport.profiles.default.port` is not set.
|
|
|
|
For example, this is an example of `unicast_hosts.txt` for a cluster with four
|
|
nodes that participate in discovery, some of which are not running on the
|
|
default port:
|
|
|
|
[source,txt]
|
|
----------------------------------------------------------------
|
|
10.10.10.5
|
|
10.10.10.6:9305
|
|
10.10.10.5:10005
|
|
# an IPv6 address
|
|
[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:9301
|
|
----------------------------------------------------------------
|
|
|
|
Host names are allowed instead of IP addresses and are resolved by DNS as
|
|
described above. IPv6 addresses must be given in brackets with the port, if
|
|
needed, coming after the brackets.
|
|
|
|
You can also add comments to this file. All comments must appear on their lines
|
|
starting with `#` (i.e. comments cannot start in the middle of a line).
|
|
|
|
[discrete]
|
|
[[ec2-hosts-provider]]
|
|
===== EC2 hosts provider
|
|
|
|
The {plugins}/discovery-ec2.html[EC2 discovery plugin] adds a hosts provider
|
|
that uses the https://github.com/aws/aws-sdk-java[AWS API] to find a list of
|
|
seed nodes.
|
|
|
|
[discrete]
|
|
[[azure-classic-hosts-provider]]
|
|
===== Azure Classic hosts provider
|
|
|
|
The {plugins}/discovery-azure-classic.html[Azure Classic discovery plugin] adds
|
|
a hosts provider that uses the Azure Classic API find a list of seed nodes.
|
|
|
|
[discrete]
|
|
[[gce-hosts-provider]]
|
|
===== Google Compute Engine hosts provider
|
|
|
|
The {plugins}/discovery-gce.html[GCE discovery plugin] adds a hosts provider
|
|
that uses the GCE API find a list of seed nodes.
|