Commit Graph

44 Commits

Author SHA1 Message Date
David Turner cf0cab614d Clarify remote clusters' use of transport layer (#60268)
Today there are a few places in the transport layer docs where we talk
about communication between nodes _within a cluster_. We also use the
transport layer for remote cluster connections, and these statements
also apply there, but this is not clear from today's docs. This commit
generalises these statements to make it clear that they apply to remote
cluster connections too.

It also adds a link from the docs on configuring TCP retries to the
(deeply-buried) docs on preserving long-lived connections.
2020-07-29 13:04:10 +01:00
David Turner 53fa52d618 Fix whitespace bug in #59222 2020-07-27 12:26:33 +01:00
David Turner d8fdb82efb Suggest reducing tcp_retries2 (#59222)
Adds documentation suggesting reducing `tcp_retries2` on Linux to detect
network partitions more quickly.

Relates #34405
2020-07-27 11:40:12 +01:00
Eddie Turizo 27673aa59c [DOCS] Fix ulimit value in system settings docs (#56973)
Correctly documents the ulimit value as 65535.

The limit was lowered as part of #37537.
2020-05-20 08:44:07 -04:00
James Rodewig 1f36c4e50c
[DOCS] Replace "// CONSOLE" comments with [source,console] (#46159) (#46332) 2019-09-05 10:11:25 -04:00
Peter Dyson 4313c2d4a7
[DOCS] More info on disabling swap (#37248)
* [DOCS] More info on disabling swap, inform that no Elasticsearch restart is required.
2019-01-25 08:39:58 +10:00
Jason Tedor 18a3e48a4a
Change file descriptor limit to 65535 (#37537)
Some systems default to a nofile ulimit of 65535. To reduce the pain of
deploying Elasticsearch to such systems, this commit lowers the required
limit from 65536 to 65535.
2019-01-16 17:19:12 -05:00
Jason Tedor 2afa7faefd
Override the JVM DNS cache policy (#36570)
When a security manager is present, the JVM will cache positive hostname
lookups indefinitely. This can be problematic, especially in the modern
world with cloud services where DNS addresses can change, or
environments using Docker containers where IP addresses could be
considered ephemeral. This behavior impacts cluster discovery,
cross-cluster replication and cross-cluster search, reindex from remote,
snapshot repositories, webhooks in Watcher, external authentication
mechanisms, and the Elastic Stack Monitoring Service. The experience of
watching a DNS lookup change yet not be reflected within Elasticsearch
is a poor experience for users. The reason the JVM has this is guard
against DNS cache posioning attacks. Yet, there is already a defense in
the modern world against such attacks: TLS. With proper certificate
validation, even if a resolver falls prey to a DNS cache poisoning
attack, using TLS would neuter the attack. Therefore we have a policy
with dubious security value that significantly impacts usability. As
such we make the usability/security tradeoff towards usability, since
the security risks are very low. This commit introduces new system
properties that Elasticsearch observes to override the JVM DNS cache
policy.
2018-12-13 10:23:45 -05:00
Jason Tedor a6073f5130
Add docs on JNA temp directory not being noexec (#35355)
If the underlying mount point for the JNA temporary directory is mounted
noexec on Linux, then the JVM will not be able to map the native code in
as executable. This will prevent JNA from executing and will prevent
Elasticsearch from being able to execute some functions that rely on
native code (e.g., memory locking, and installing system call
filters). We do not want to get into the business of catching exceptions
and parsing messages towards this because these exception messages can
change on us. We also do not want to jump through a lot of hoops to
check the underlying mount point for noexec. Instead, we will rely on
documentation to address this problem. This commit adds to the important
system configuration section of the docs that the JNA temporary
directory is not on a mount point with the noexec mount option.
2018-11-07 22:25:37 -05:00
Jason Tedor 588db621ac
Remove reference to non-existent store type (#32418)
We removed the default_fs store type yet the docs still contain a
reference to them. This commit addresses that by removing this
reference, and changing a reference to this section of the docs to
instead refer to mmapfs.
2018-07-27 11:24:03 -04:00
sandstrom e542c03bea Clarify that number of threads is set by packages
This commit clarifies that on systemd the packages already set the number of
threads for the Elasticsearch process.

Relates #27840
2017-12-15 11:29:13 -05:00
Jason Tedor 008296e2b6
Reorganize configuring Elasticsearch docs
This commit reorganizes some of the content in the configuring
Elasticsearch section of the docs. The changes are:
 - move JVM options out of system configuration into configuring
   Elasticsearch
 - move JVM options to its own page of the docs
 - move configuring the heap to important Elasticsearch settings
 - move configuring the heap to its own page of the docs
 - move all important settings to individual pages in the docs
 - remove bootstrap.memory_lock from important settings, this is covered
   in the swap section of system configuration

Relates #27755
2017-12-12 10:24:37 -05:00
Jason Tedor 6c7374804f
Extend JVM options to support multiple versions
JDK 9 has removed JVM options that were valid in JDK 8 (e.g., GC logging
flags) and replaced them with new flags that are not available in JDK
8. This means that a single JVM options file can no longer apply to JDK
8 and JDK 9, complicating development, complicating our packaging story,
and complicating operations. This commit extends the JVM options syntax
to specify the range of versions the option applies to. If the running
JVM matches the range of versions, the flag will be used to start the
JVM otherwise the flag will be ignored.

We implement this parser in Java for simplicity, and with this we start
our first step towards a Java launcher.

Relates #27675
2017-12-06 18:03:13 -05:00
Jason Tedor 6655689b15 Move DNS cache docs to system configuration docs
When these docs were moved they should have been moved to the system
configuration docs. This commit does that, and also fixes a missing
heading that broke the docs build.
2017-11-29 19:57:26 -05:00
Kanako Nakai 23f85fe6d4 Fix max number of threads bootstrap docs
Previously the bootstrap check for max number of threads was increased
from 2048 to 4096 yet the docs were never adjusted for this change. This
commit addresses this so the docs are in-line with the limit enforced in
the bootstrap check.

Relates #27511
2017-11-28 22:19:04 -05:00
Nicolas Sierra d6fc4affae Clarify systemd overrides
This commit clarifies how to apply an override to the systemd unit file
for Elasticsearch.

Relates #26950
2017-10-10 13:06:34 -04:00
Steve Kotsopoulos dd95849b62 Document JVM option MaxFDLimit for macOS ()
This commit documents a JVM option that is needed on macOS when raising
file descriptor limits there.

Relates #26900
2017-10-05 14:56:15 -04:00
Jason Tedor 540413b24a Also skip JAVA_TOOL_OPTIONS on Windows
On non-Windows platforms, we ignore the environment variable
JAVA_TOOL_OPTIONS (this is an environment variable that the JVM respects
by default for picking up extra JVM options). The primary reason that we
ignore this because of the Jayatana agent on Ubuntu; a secondary reason
is that it produces an annoying "Picked up JAVA_TOOL_OPTIONS: ..."
output message. When the elasticsearch-env batch script was introduced
for Windows, ignoring this environment variable was deliberately not
carried over as the primary reason does not apply on Windows. However,
after additional thinking, it seems that we should simply be consistent
to the extent possible here (and also avoid that annoying "Picked up
JAVA_TOOL_OPTIONS: ..." on Windows too). This commit causes the Windows
version of elasticsearch-env to also ignore JAVA_TOOL_OPTIONS.

Relates #25968
2017-07-31 21:27:42 +09:00
Adrien Grand 57e8b9ee29 Remove another reference to 64-bit systems. 2017-07-31 09:56:06 +02:00
propulkit a2fd1b9f6d No more using 'hybrid mmapfs / niofs' (#25944)
It looks a bit ambiguous here.

ElasticSearch no more using 'hybrid mmapfs / niofs' which chooses filesystem based on the file. It is any one of the mmapfs, niofs or simplefs depending on the operating system.
As quoted here https://www.elastic.co/guide/en/elasticsearch/reference/5.5/index-modules-store.html

Thanks,
Pulkit Agrawal
2017-07-31 09:53:55 +02:00
Ryan Ernst b1762d69b5 Setup: Change default heap to 1G (#25695)
This commit changes the default heap size to 1 GB. Experimenting with
elasticsearch is often done on laptops, and 1 GB is much friendlier to
laptop memory. It does put more pressure on the gc, but the tradeoff is
a smaller default footprint. Users running in production can (and
should) adjust the heap size as necessary for their usecase.
2017-07-14 09:38:08 -07:00
Jason Tedor 4e21a33689 Clarify disabling swap in docs
Our strong recommendation is disabling swap over any other alternative
to avoid the JVM from landing on disk. This commit clarifies the docs in
this regard.
2017-05-12 16:09:52 -04:00
Nik Everett 1c1b29400b Docs: Fix language on a few snippets
They aren't `js`, they are their own thing.

Relates to #18160
2017-03-22 15:57:28 -04:00
Itamar Syn-Hershko dea8cee70f Fixing a second "second" reference in docs (#22345) 2016-12-28 18:50:10 +01:00
Jared Carey 317866894e Fix systemd override example in configuring docs
When overriding a systemd configuration via a drop-in file, the
[Service] header is required. This commit adds this to an example
drop-in override in the configuring docs.

Relates #22038
2016-12-07 19:41:59 -05:00
Nikhil Patel b5e3d351d9 Fix typos in threads docs
This commit fixes a typo in the threads docs where the past tense form
of a verb was used when current tense is needed.

Relates #22016
2016-12-07 08:22:49 -05:00
Jason Tedor b9df2e2287 Improve the out-of-the-box experience
Elasticsearch can be run in a few different ways:
 - from the command line on Linux and Windows
 - as a service on Linux and Windows

on both 32-bit client and 64-bit server VMs. We strive for a great
out-of-the-box experience any of these combinations but today it is
lacking on 32-bit client JVMs and on the Windows service. There are two
deficiencies that arise:
 - on any 32-bit client JVM we fail to start out of the box because we
   force the server JVM in jvm.options
 - when installing the Windows service, the thread stack size must be
   specified in jvm.options

This commit attempts to address these deficiencies.

We should continue to force the server JVM because there are systems
where the server JVM is not active by default (e.g., the 32-bit JDK on
Windows). This does mean that if a user tries to run with a client JVM
they will see a failure message at startup but this is the best that we
can do if we want to continue to force the server JVM. Thus, this commit
at least documents this situation.

To improve the situation with installing the Windows service, this
commit adds a default setting for the thread stack size. This default is
chosen based on the default thread stack size across all 64-bit server
JVMs. This means that if a user tries to run with a 32-bit JVM they
could otherwise see significantly higher memory usage (this situation is
complicated, it's really only on Windows where the extra memory usage is
egregious, but cutting into the 32-bit address space on any system is
bad). So this commit makes it so that the out-of-the-box experience is
improved for the Windows service on 64-bit server JVMs and we document
the need to adjust this setting on 32-bit JVMs.

Again, we are focusing on the out-of-the-box experience here and this
means optimizing for the best experience on any 64-bit server JVM as
this covers the vast majority of the user base. The users that are on
32-bit JVMs will suffer a little bit but at least now any user on any
64-bit server JVM can start Elasticsearch out of the box.

Finally, we fix some references to the jvm.options documentation.

Relates #21920
2016-12-01 17:26:29 -05:00
Jason Tedor 0fe6b12553 Remove misplaced parenthesis from configuring docs
This commit removes a misplaced parenthesis from the systemd section of
the configuring docs.
2016-11-30 12:11:47 -05:00
Pascal Borreli fcb01deb34 Fixed typos (#20843) 2016-10-10 14:51:47 -06:00
Nik Everett c414faf2fb Reword note about windows and FDs
Make it clear why this isn't a thing in windows and remove value
judgement about file descriptors.

Relates to #20737
2016-10-10 08:47:25 -04:00
Nik Everett 2b3760f78b File descriptors limit doesn't apply to Windows
On Windows the JDK uses `CreateFileW` which has a stupidly high
limit for the number of `Handle`s it can make - `16 * 1024 * 1024`.
So this isn't really a problem on Windows at all.

Closes #20732
2016-10-04 11:32:08 -04:00
Jason Tedor a7c777f58a Fix documentation for setting Java I/O temp dir
This commit fixes the documentation for configuring the Java I/O temp
dir which incorrectly suggested using the -D flag as a parameter on the
command line; these flags have been removed and should now be specified
as arguments to the JVM using either the ES_JAVA_OPTS environment
variable or using the jvm.options configuration file.

Closes #20652
2016-09-24 21:13:22 -04:00
Clinton Gormley 57d40a4691 Fix JVM heap size docs for 5.0
Closes #20587
2016-09-20 18:51:56 +02:00
Nik Everett 5cff2a046d Remove most of the need for `// NOTCONSOLE`
and be much more stingy about what we consider a console candidate.

* Add `// CONSOLE` to check-running
* Fix version in some snippets
* Mark groovy snippets as groovy
* Fix versions in plugins
* Fix language marker errors
* Fix language parsing in snippets

  This adds support for snippets who's language is written like
  `[source, txt]` and `["source","js",subs="attributes,callouts"]`.

  This also makes language required for snippets which is nice because
  then we can be sure we can grep for snippets in a particular language.
2016-09-06 10:32:54 -04:00
Nik Everett c24ad1df4e Mark lots of stuff `// NOTCONSOLE` 2016-08-29 08:28:16 -04:00
Jason Tedor fb893a993f Add note regarding Windows service heap size
This commit adds a note regarding the difference in configuration for
the Windows service heap size from any other installation of
Elasticsearch.

Relates #18606
2016-06-01 16:31:16 -04:00
Jason Tedor 8e2a7d0fe1 Rename boostrap.mlockall to bootstrap.memory_lock
The setting bootstrap.mlockall is useful on both POSIX-like systems
(POSIX mlockall) and Windows (Win32 VirtualLock). But mlockall is really
a POSIX only thing so the name should not be tied POSIX. This commit
renames the setting to "bootstrap.memory_lock".

Relates #18669
2016-06-01 16:25:51 -04:00
Clinton Gormley b2f2d38ebb Update configuring.asciidoc
Minor asciidoc changes
2016-05-29 11:50:36 +02:00
bloublou 83944c5628 Typo correction heap_size.asciidoc (#17745)
* Typo correction Xms Xmx

Typo correction on "-Xms4000mb -Xmx4000mb"

* Change mb to m for Xms/Xmx
2016-04-14 20:37:37 +02:00
Clinton Gormley bdf62b5615 More asciidoc errors 2016-04-13 10:14:09 +02:00
Clinton Gormley 1a15e55f94 More asciidoc errors 2016-04-13 10:02:09 +02:00
Clinton Gormley b201605a81 Fix bad asciidoc 2016-04-13 09:57:00 +02:00
Jason Tedor 3879aa2a98 Add JVM options configuration file
This commit adds a new configuration file jvm.options to centralize and
simplify management of JVM options. This separates the configuration of
the JVM from the packaging scripts (bin/elasticsearch*, bin/service.bat,
and init.d/elasticsearch) simplifying end-user operational management of
custom JVM options.
2016-04-12 11:19:16 -04:00
Clinton Gormley 06604708d4 Docs: Complete rewrite of setup, installation, and configuration docs 2016-04-03 16:09:48 +02:00