Commit Graph

40 Commits

Author SHA1 Message Date
David Turner 2d114a02ff
Rename static Zen1 settings ()
Renames the following settings to remove the mention of `zen` in their names:

- `discovery.zen.hosts_provider` -> `discovery.seed_providers`
- `discovery.zen.ping.unicast.concurrent_connects` -> `discovery.seed_resolver.max_concurrent_resolvers`
- `discovery.zen.ping.unicast.hosts.resolve_timeout` -> `discovery.seed_resolver.timeout`
- `discovery.zen.ping.unicast.hosts` -> `discovery.seed_addresses`
2019-02-05 08:46:52 +00:00
Daniel Mitterdorfer 75f3443c62
Rename setting to enable mmap
With this commit we rename `node.store.allow_mmapfs` to
`node.store.allow_mmap`. Previously this setting has controlled whether
`mmapfs` could be used as a store type. With the introduction of
`hybridfs` which also relies on memory-mapping,
`node.store.allow_mmapfs` also applies to `hybridfs` and thus we rename
it in order to convey that it is actually used to allow memory-mapping
but not a specific store type.

Relates 
Relates 
2019-01-03 07:10:34 +01:00
David Turner 1a23417aeb
[Zen2] Update documentation for Zen2 ()
This commit overhauls the documentation of discovery and cluster coordination,
removing mention of the Zen Discovery module and replacing it with docs for the
new cluster coordination mechanism introduced in 7.0.

Relates 
2018-12-20 13:02:44 +00:00
David Turner c32e4fb83f
[Zen2] Best-effort cluster formation if unconfigured ()
In real deployments it is important that clusters are properly configured to
avoid accidentally forming multiple independent clusters at cluster
bootstrapping time. However we also expect to be able to unpack Elasticsearch
and start up one or more nodes without any up-front configuration, and have
them do their best to find each other and form a cluster after a few seconds.

This change adds a delayed automatic bootstrapping process to nodes that start
up with no relevant settings set to support the desired out-of-the-box
experience without compromising safety in properly-configured deployments.
2018-12-07 12:47:09 +00:00
Alan Woodward cf24231b1b
Add missing full stop in Bootstrap checks doc ()
Fixes 
2018-10-30 11:43:43 +00:00
Jason Tedor bdfcc326d7
Enable avoiding mmap bootstrap check ()
The maximum map count boostrap check can be a hindrance to users that do
not own the underlying platform on which they are executing
Elasticsearch. This is because addressing it requires tuning the kernel
and a platform provider might now allow this, especially on shared
infrastructure. However, this bootstrap check is not needed if mmapfs is
not in use. Today we do not have a way for the user to communicate that
they are not going to use mmapfs. This commit therefore adds a setting
that enables the user to disallow mmapfs. When mmapfs is disallowed, the
maximum map count bootstrap check is not enforced. Additionally, we
fallback to a different default index store and prevent the explicit use
of mmapfs for an index.
2018-08-21 11:02:25 -04:00
Jason Tedor 3ac57f0ba3
Fix ordering of bootstrap checks in docs ()
In the section of the bootstrap checks docs for the maximum map count
check, we refer to max size virtual memory check and explicitly call out
the maximum size virtual memory check as being the previous
point. However, this is not correct as the previous point is currently
the max file size check. It does make sense for these two checks to be
proximate to each other in the docs so this commit reorders the checks
so that the maximum size virtual memory check indeed comes before the
maximum map count check. This makes the sense in the maximum map count
check correct.
2018-07-27 10:40:16 -04:00
Sachin Frayne 183ba9a7e5 Fix typo in max number of threads check docs ()
Historically, the bootstrap checks used 2048 as the minimum limit for
the maximum number of threads. This limit was guided by the fact that
the number of processors was artificially capped at 32. This limit was
removed in 6.0.0 and the minimum limit was raised to 4096 to accommodate
this. However, the docs were not updated and this commit addresses that
miss.
2018-04-12 21:16:37 -04:00
Jason Tedor b5d9774c65
Remove leftover doc comment on client JVM check
We previously specified the -server flag to force the JVM to use the
server JVM. This is the default on all the systems that we support when
using a 64-bit JVM (and we no longer support 32-bit JVMs). There was
some trouble with this flag for the Windows service since procrun did
not understand what to do with it; as such, we had to filter this flag
out in the service. When we migrated to parsing JVM options in Java (via
the JVM options parser) we simplified this situation and removed
specifying the -server flag. This commit removes a leftover statement
that we are forcing the server JVM.

Relates 
2018-02-20 08:49:26 -05:00
Jason Tedor a91da9a9af
Only bind loopback addresses when binding to local
* Only bind loopback addresses when binding to local

Today when binding to local (the default) we bind to any address that is
a loopback address, or any address on an interface that declares itself
as a loopback interface. Yet, not all addresses on loopback interfaces
are loopback addresses. This arises on macOS where there is a link-local
address assigned to the loopback interface (fe80::1%lo0) and in Docker
services where virtual IPs of the service are assigned to the loopback
interface (docker/libnetwork#1877). These situations cause problems:
 - because we do not handle the scope ID of a link-local address, we end
   up bound to an address for which publishing of that address does not
   allow that address to be reached (since we drop the scope)
 - the virtual IPs in the Docker situation are not loopback addresses,
   they are not link-local addresses, so we end up bound to interfaces
   that cause the bootstrap checks to be enforced even though the
   instance is only bound to local

We address this by only binding to actual loopback addresses, and skip
binding to any address on a loopback interface that is not a loopback
address. This lets us simplify some code where in the bootstrap checks
we were skipping link-local addresses, and in writing the ports file
where we had to skip link-local addresses because again the formatting
of them does not allow them to be connected to by another node (to be
clear, they could be connected to via the scope-qualified address, but
that information is not written out).

Relates 
2018-01-02 07:04:09 -05:00
Jason Tedor 008296e2b6
Reorganize configuring Elasticsearch docs
This commit reorganizes some of the content in the configuring
Elasticsearch section of the docs. The changes are:
 - move JVM options out of system configuration into configuring
   Elasticsearch
 - move JVM options to its own page of the docs
 - move configuring the heap to important Elasticsearch settings
 - move configuring the heap to its own page of the docs
 - move all important settings to individual pages in the docs
 - remove bootstrap.memory_lock from important settings, this is covered
   in the swap section of system configuration

Relates 
2017-12-12 10:24:37 -05:00
Andrew Banchich 1be286c592 [Docs] Grammatical fix in bootstrap-checks.asciidoc () 2017-12-05 11:46:19 +01:00
Jason Tedor d8c28044da
Forbid granting the all permission in production
Running with the all permission java.security.AllPermission granted is
equivalent to disabling the security manager. This commit adds a
bootstrap check that forbids running with this permission granted.

Relates 
2017-11-27 16:05:27 -05:00
Jason Tedor 279be13a00 Clarify development vs. production mode
The definition of development vs. production mode has evolved slightly
over time (with the introduction of single-node) discovery. This commit
clarifies the documentation to better account for this adjustment.

Relates 
2017-09-02 09:47:39 -04:00
Jason Tedor 2ef0f8af38 Add max file size bootstrap check
This commit adds a bootstrap check for the maximum file size, and
ensures the limit is set correctly when Elasticsearch is installed as a
service on systemd-based systems.

Relates 
2017-07-31 21:01:47 +09:00
Viral-Inc f3569debb6 [Docs] Fix typo in bootstrap-checks.asciidoc () 2017-07-07 15:37:40 +02:00
Dimitrios Liappis 79857357bf Docs: Update production notes for Docker
Add info about the base image used and the github repo of
elasticsearch-docker.

Clarify that setting `memlock=-1:-1` is only a requirement when
`bootstrap_memory_lock=true` and the alternatives we document
elsewhere in docs for disabling swap are valid for Docker as well.

Additionally, with latest versions of docker-ce shipping with
unlimited (or high enough) defaults for `nofile` and `nproc`, clarify
that explicitly setting those per ES container is not required, unless
they are not defined in the Docker daemon.

Finally simplify production `docker-compose.yml` example by removing
unneeded options.

Relates 
2017-05-03 14:27:31 +03:00
Jason Tedor 51b5dbffb7 Disable bootstrap checks for single-node discovery
While there are use-cases where a single-node is in production, there
are also use-cases for starting a single-node that binds transport to an
external interface where the node is not in production (for example, for
testing the transport client against a node started in a Docker
container). It's tricky to balance the desire to always enforce the
bootstrap checks when a node might be in production with the need for
the community to perform testing in situations that would trip the
bootstrap checks. This commit enables some flexibility for these
users. By setting the discovery type to "single-node", we disable the
bootstrap checks independently of how transport is bound. While this
sounds like a hole in the bootstrap checks, the bootstrap checks can
already be avoided in the single-node use-case by binding only HTTP but
not transport. For users that are genuinely in production on a
single-node use-case with transport bound to an external use-case, they
can set the system property "es.enable.bootstrap.checks" to force
running the bootstrap checks. It would be a mistake for them not to do
this.

Relates 
2017-04-04 09:39:04 -04:00
Jason Tedor a6c4234575 Add early-access check
The OpenJDK project provides early-access builds of upcoming
releases. These early-access builds are not suitable for
production. These builds sometimes end up on systems due to aggressive
packaging (e.g., Ubuntu). This commit adds a bootstrap check to ensure
these early-access builds are not being used in production.

Relates 
2017-03-24 14:52:50 -04:00
Jason Tedor f7b8128f92 Enable explicitly enforcing bootstrap checks
This commit adds a system property that enables end-users to explicitly
enforce the bootstrap checks, independently of the binding of the
transport protocol. This can be useful for single-node production
systems that do not bind the transport protocol (and thus the bootstrap
checks would not be enforced).

Relates 
2017-03-15 10:36:17 -07:00
Jason Tedor 30806af6bd Rename bootstrap.seccomp to bootstrap.system_call_filter
We try to install a system call filter on various operating systems
(Linux, macOS, BSD, Solaris, and Windows) but the setting
(bootstrap.seccomp) to control this is named after the Linux
implementation (seccomp). This commit replaces this setting with
bootstrap.system_call_filter. For backwards compatibility reasons, we
fallback to bootstrap.seccomp and log a deprecation message if
bootstrap.seccomp is set. We intend to remove this fallback in
6.0.0. Note that now is the time to make this change it's likely that
most users are not making this setting anyway as prior to version 5.2.0
(currently unreleased) it was not necessary to configure anything to
enable a node to start up if the system call filter failed to install
(we marched on anyway) but starting in 5.2.0 it will be necessary in
this case.

Relates 
2016-12-16 18:22:54 -05:00
Juan Paulo Gutierrez acc39b1496 Fixed typo () 2016-12-09 10:34:29 +01:00
Jason Tedor 0afef53a17 Add system call filter bootstrap check
Today if system call filters fail to install on startup, we log a
message but otherwise march on. This might leave users without system
call filters installed not knowing that they have implicitly accepted
the additional risk. We should not be lenient like this, instead clearly
informing the user that they have to either fix their configuration or
accept the risk of not having system call filters installed. This commit
adds a bootstrap check that if system call filters are enabled, they
must successfully install.

Relates 
2016-12-02 14:27:54 -05:00
Joeyn414 aefffca763 minor grammatical fix () 2016-11-08 11:52:52 +01:00
Jason Tedor 1f0c6cd137 Remove double space in bootstrap checks docs
This commit removes a double space in the G1GC check section of the
bootstrap check docs.
2016-11-07 16:28:04 -05:00
Jason Tedor b30732c464 Migrate G1GC JVM check to bootstrap check
This commit fixes an assertion in G1GCCheck#jvmVersion that was
mistakenly asserting on itself.

Relates 
2016-11-07 16:19:05 -05:00
Jason Tedor f81d28d0a0 Fix transport host setting
This commit fixes a reference to the transport.host setting which was
mistakenly identified as transport.tcp.host.
2016-10-31 20:48:55 -04:00
Jason Tedor 5fab17c7bc Fix markup for bootstrap checks docs
This commit adds a missing backtick to the bootstrap check docs.
2016-10-31 17:33:15 -04:00
Jason Tedor 1d8d8ea55f Clarify production mode for bootstrap checks
This commit clarifies that production mode for the bootstrap checks is
only tripped if transport is bound to an external interface.

Relates 
2016-10-31 16:34:21 -04:00
Pascal Borreli fcb01deb34 Fixed typos () 2016-10-10 14:51:47 -06:00
Jason Tedor 05b4e0c0e3 Add serial collector bootstrap check
The serial collector is not suitable for running with a server
application like Elasticsearch and can decimate performance and lead to
cluster instability. This commit adds a bootstrap check to prevent usage
of the serial collector when Elasticsearch is running in production
mode.

Relates 
2016-09-19 20:25:50 -04:00
Jason Tedor 90c2aae28b Remove minimum master nodes bootstrap docs
The minimum master nodes bootstrap check was removed in
069fc22696 but the docs were left behind.
This commit removes these stale docs.

Relates 
2016-08-23 14:31:58 -04:00
Jason Tedor e94408c0d2 Bootstrap check for OnError and seccomp
This commit adds a bootstrap check for the JVM option OnError being in
use and seccomp being enabled. These two options are incompatible
because OnError allows the user to specify an arbitrary program to fork
when the JVM encounters an fatal error, and seccomp enables system call
filters that prevents forking.
2016-06-06 22:18:44 -04:00
Jason Tedor 9695caa3fb Bootstrap check for OnOutOfMemoryError and seccomp
This commit adds a bootstrap check for the JVM option OnOutOfMemoryError
being in use and seccomp being enabled. These two options are
incompatible because OnOutOfMemoryError allows the user to specify an
arbitrary program to fork when the JVM encounters an
OutOfMemoryError, and seccomp enables system call filters that prevents
forking.

This commit also adds support for bootstrap checks that are always
enforced, whether or not Elasticsearch is in production mode.
2016-06-06 17:31:42 -04:00
Jason Tedor 8e2a7d0fe1 Rename boostrap.mlockall to bootstrap.memory_lock
The setting bootstrap.mlockall is useful on both POSIX-like systems
(POSIX mlockall) and Windows (Win32 VirtualLock). But mlockall is really
a POSIX only thing so the name should not be tied POSIX. This commit
renames the setting to "bootstrap.memory_lock".

Relates 
2016-06-01 16:25:51 -04:00
Jason Tedor 37a3588c37 Fix min. master nodes links in boostrap check docs
This commit fixes two links to the minimum master nodes configuration
section of the docs in the bootstrap check docs.
2016-05-29 08:01:16 -04:00
Clinton Gormley e35bd11581 Update bootstrap-checks.asciidoc
Fixed asciidoc
2016-05-29 11:56:02 +02:00
Jason Tedor 46162a40e7 Additional bootstrap check doc fixes
This commit fixes some additional poorly-formatted internal and external
links in the bootstrap check docs.
2016-05-27 10:58:13 -04:00
Jason Tedor 123e40726e Fix bootstrap check docs
This commit fixes some incorrect links in the bootstrap check docs.
2016-05-27 09:19:49 -04:00
Jason Tedor 82713bab6d Add bootstrap check docs
This commit adds documentation for the bootstrap checks and provides
either links or inline guidance for setting the necessary settings to
pass the bootstrap checks.

Relates 
2016-05-27 06:03:35 -04:00