honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Docs: Complete rewrite of setup, installation, and configuration docs

This commit is contained in:
Clinton Gormley 2016-04-03 16:09:24 +02:00
parent e85535724e
commit 06604708d4
32 changed files with 1624 additions and 929 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/reference/index.asciidoc
View File

@ -4,7 +4,7 @@
:version: 5.0.0-alpha1
:major-version: 5.x
:branch: master
:jdk: 1.8.0_25
:jdk: 1.8.0_73
:defguide: https://www.elastic.co/guide/en/elasticsearch/guide/current
:plugins: https://www.elastic.co/guide/en/elasticsearch/plugins/master
:javaclient: https://www.elastic.co/guide/en/elasticsearch/client/java-api/master/

2
docs/reference/modules/node.asciidoc
View File

@ -297,6 +297,6 @@ same data directory. This can lead to unexpected data loss.
== Other node settings
More node settings can be found in <<modules,Modules>>. Of particular note are
the <<cluster-name,`cluster.name`>>, the <<node-name,`node.name`>> and the
the <<cluster.name,`cluster.name`>>, the <<node.name,`node.name`>> and the
<<modules-network,network settings>>.

4
docs/reference/redirects.asciidoc
View File

@ -7,8 +7,8 @@ The following pages have moved or been deleted.
=== Nodes shutdown
The `_shutdown` API has been removed. Instead, setup Elasticsearch to run as
a service (see <<setup-service>> or <<setup-service-win>>) or use the `-p`
command line option to <<setup-installation-pid,write the PID to a file>>.
a service (see <<rpm>>, <<deb>>, or <<windows>>) or use the `-p`
command line option to <<setup-installation-daemon,write the PID to a file>>.
[role="exclude",id="docs-bulk-udp"]
=== Bulk UDP API

91
docs/reference/setup.asciidoc
View File

@ -1,14 +1,15 @@
[[setup]]
= Setup
= Setup Elasticsearch
[partintro]
--
This section includes information on how to setup *elasticsearch* and
get it running. If you haven't already, http://www.elastic.co/downloads[download] it, and
then check the <<setup-installation,installation>> docs.
This section includes information on how to setup Elasticsearch and get it
running, including:
NOTE: Elasticsearch can also be installed from our repositories using `apt` or `yum`.
See <<setup-repositories>>.
* Downloading
* Installing
* Starting
* Configuring
[[supported-platforms]]
[float]
@ -18,89 +19,29 @@ The matrix of officially supported operating systems and JVMs is available here:
link:/support/matrix[Support Matrix]. Elasticsearch is tested on the listed
platforms, but it is possible that it will work on other platforms too.
[[setup-installation]]
[float]
== Installation
After link:/downloads/elasticsearch[downloading] the latest release and extracting it,
*elasticsearch* can be started using:
[source,sh]
--------------------------------------------------
$ bin/elasticsearch
--------------------------------------------------
On *nix systems, the command will start the process in the foreground.
[[setup-installation-daemon]]
[float]
=== Running as a daemon
To run it in the background, add the `-d` switch to it:
[source,sh]
--------------------------------------------------
$ bin/elasticsearch -d
--------------------------------------------------
[[setup-installation-pid]]
[float]
=== PID
The Elasticsearch process can write its PID to a specified file on startup,
making it easy to shut down the process later on:
[source,sh]
--------------------------------------------------
$ bin/elasticsearch -d -p pid <1>
$ kill `cat pid` <2>
--------------------------------------------------
<1> The PID is written to a file called `pid`.
<2> The `kill` command sends a `TERM` signal to the PID stored in the `pid` file.
NOTE: The startup scripts provided for <<setup-service,Linux>> and <<setup-service-win,Windows>>
take care of starting and stopping the Elasticsearch process for you.
.*NIX
*************************************************************************
There are added features when using the `elasticsearch` shell script.
The first, which was explained earlier, is the ability to easily run the
process either in the foreground or the background.
Another feature is the ability to pass `-E` configuration parameters
directly to the script. For example:
[source,sh]
--------------------------------------------------
$ bin/elasticsearch -Ees.index.refresh_interval=5s -Ees.node.name=my-node
--------------------------------------------------
*************************************************************************
[float]
[[jvm-version]]
== Java (JVM) version
== Java (JVM) Version
Elasticsearch is built using Java, and requires at least
http://www.oracle.com/technetwork/java/javase/downloads/index.html[Java 8] in
order to run. Only Oracle's Java and the OpenJDK are supported. The same JVM
version should be used on all Elasticsearch nodes and clients.
We recommend installing the *Java 8 update 20 or later*.
Elasticsearch will refuse to start if a known-bad version of Java is used.
We recommend installing Java version *{jdk} or later*. Elasticsearch will
refuse to start if a known-bad version of Java is used.
The version of Java to use can be configured by setting the `JAVA_HOME`
environment variable.
The version of Java that Elasticsearch will use can be configured by setting
the `JAVA_HOME` environment variable.
--
include::setup/install.asciidoc[]
include::setup/configuration.asciidoc[]
include::setup/as-a-service.asciidoc[]
include::setup/important-settings.asciidoc[]
include::setup/as-a-service-win.asciidoc[]
include::setup/dir-layout.asciidoc[]
include::setup/repositories.asciidoc[]
include::setup/sysconfig.asciidoc[]
include::setup/upgrade.asciidoc[]

67
docs/reference/setup/as-a-service-win.asciidoc
View File

@ -1,67 +0,0 @@
:backslash: \
[[setup-service-win]]
== Running as a Service on Windows
Windows users can configure Elasticsearch to run as a service to run in the background or start automatically
at startup without any user interaction.
This can be achieved through `service.bat` script under `bin/` folder which allows one to install,
remove, manage or configure the service and potentially start and stop the service, all from the command-line.
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
c:\elasticsearch-{version}{backslash}bin>service
Usage: service.bat install|remove|start|stop|manager [SERVICE_ID]
--------------------------------------------------
The script requires one parameter (the command to execute) followed by an optional one indicating the service
id (useful when installing multiple Elasticsearch services).
The commands available are:
[horizontal]
`install`:: Install Elasticsearch as a service
`remove`:: Remove the installed Elasticsearch service (and stop the service if started)
`start`:: Start the Elasticsearch service (if installed)
`stop`:: Stop the Elasticsearch service (if started)
`manager`:: Start a GUI for managing the installed service
Note that the environment configuration options available during the installation are copied and will be used during
the service lifecycle. This means any changes made to them after the installation will not be picked up unless
the service is reinstalled.
Based on the architecture of the available JDK/JRE (set through `JAVA_HOME`), the appropriate 64-bit(x64) or 32-bit(x86)
service will be installed. This information is made available during install:
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
c:\elasticsearch-{version}\bin>service install
Installing service : "elasticsearch-service-x64"
Using JAVA_HOME (64-bit): "c:\jvm\jdk1.8"
The service 'elasticsearch-service-x64' has been installed.
--------------------------------------------------
NOTE: While a JRE can be used for the Elasticsearch service, due to its use of a client VM (as oppose to a server JVM which
offers better performance for long-running applications) its usage is discouraged and a warning will be issued.
[float]
=== Customizing service settings
There are two ways to customize the service settings:
Manager GUI:: accessible through `manager` command, the GUI offers insight into the installed service including its status, startup type,
JVM, start and stop settings among other things. Simply invoking `service.bat` from the command-line with the aforementioned option
will open up the manager window:
image::images/service-manager-win.png["Windows Service Manager GUI",align="center"]
Customizing `service.bat`:: at its core, `service.bat` relies on http://commons.apache.org/proper/commons-daemon/[Apache Commons Daemon] project
to install the services. For full flexibility such as customizing the user under which the service runs, one can modify the installation
parameters to tweak all the parameters accordingly. Do note that this requires reinstalling the service for the new settings to be applied.
NOTE: There is also a community supported customizable MSI installer available: https://github.com/salyh/elasticsearch-msi-installer (by Hendrik Saly).

87
docs/reference/setup/as-a-service.asciidoc
View File

@ -1,87 +0,0 @@
[[setup-service]]
== Running as a Service on Linux
In order to run elasticsearch as a service on your operating system, the provided packages try to make it as easy as possible for you to start and stop elasticsearch during reboot and upgrades.
[float]
=== Linux
Currently our build automatically creates a debian package and an RPM package, which is available on the download page. The package itself does not have any dependencies, but you have to make sure that you installed a JDK.
Each package features a configuration file, which allows you to set the following parameters
[horizontal]
`ES_USER`:: The user to run as, defaults to `elasticsearch`
`ES_GROUP`:: The group to run as, defaults to `elasticsearch`
`ES_HEAP_SIZE`:: The heap size to start with
`ES_HEAP_NEWSIZE`:: The size of the new generation heap
`ES_DIRECT_SIZE`:: The maximum size of the direct memory
`MAX_OPEN_FILES`:: Maximum number of open files, defaults to `65536`
`MAX_LOCKED_MEMORY`:: Maximum locked memory size. Set to "unlimited" if you use the bootstrap.mlockall option in elasticsearch.yml. You must also set ES_HEAP_SIZE.
`MAX_MAP_COUNT`:: Maximum number of memory map areas a process may have. If you use `mmapfs` as index store type, make sure this is set to a high value. For more information, check the https://github.com/torvalds/linux/blob/master/Documentation/sysctl/vm.txt[linux kernel documentation] about `max_map_count`. This is set via `sysctl` before starting elasticsearch. Defaults to `65535`
`LOG_DIR`:: Log directory, defaults to `/var/log/elasticsearch`
`DATA_DIR`:: Data directory, defaults to `/var/lib/elasticsearch`
`CONF_DIR`:: Configuration file directory (which needs to include `elasticsearch.yml` and `logging.yml` files), defaults to `/etc/elasticsearch`
`ES_JAVA_OPTS`:: Any additional java options you may want to apply. This may be useful, if you need to set the `node.name` property, but do not want to change the `elasticsearch.yml` configuration file, because it is distributed via a provisioning system like puppet or chef. Example: `ES_JAVA_OPTS="-Des.node.name=search-01"`
`RESTART_ON_UPGRADE`:: Configure restart on package upgrade, defaults to `false`. This means you will have to restart your elasticsearch instance after installing a package manually. The reason for this is to ensure, that upgrades in a cluster do not result in a continuous shard reallocation resulting in high network traffic and reducing the response times of your cluster.
`ES_GC_LOG_FILE` :: The absolute log file path for creating a garbage collection logfile, which is done by the JVM. Note that this logfile can grow pretty quick and thus is disabled by default.
[float]
==== Debian/Ubuntu
The debian package ships with everything you need as it uses standard debian tools like update `update-rc.d` to define the runlevels it runs on. The init script is placed at `/etc/init.d/elasticsearch` as you would expect it. The configuration file is placed at `/etc/default/elasticsearch`.
The debian package does not start up the service by default. The reason for this is to prevent the instance to accidentally join a cluster, without being configured appropriately. After installing using `dpkg -i` you can use the following commands to ensure, that elasticsearch starts when the system is booted and then start up elasticsearch:
[source,sh]
--------------------------------------------------
sudo update-rc.d elasticsearch defaults 95 10
sudo /etc/init.d/elasticsearch start
--------------------------------------------------
Users running Debian 8 or Ubuntu 14 or later may require configuration of systemd instead of `update-rc.d`. In those cases, please refer to the <<using-systemd>> section.
[float]
===== Installing the oracle JDK
The usual recommendation is to run the Oracle JDK with elasticsearch. However Ubuntu and Debian only ship the OpenJDK due to license issues. You can easily install the oracle installer package though. In case you are missing the `add-apt-repository` command under Debian GNU/Linux, make sure have at least Debian Jessie and the package `python-software-properties` installed
[source,sh]
--------------------------------------------------
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
java -version
--------------------------------------------------
The last command should verify a successful installation of the Oracle JDK.
[float]
==== RPM based distributions
[float]
===== Using chkconfig
Some RPM based distributions are using `chkconfig` to enable and disable services. The init script is located at `/etc/init.d/elasticsearch`, where as the configuration file is placed at `/etc/sysconfig/elasticsearch`. Like the debian package the RPM package is not started by default after installation, you have to do this manually by entering the following commands
[source,sh]
--------------------------------------------------
sudo /sbin/chkconfig --add elasticsearch
sudo service elasticsearch start
--------------------------------------------------
[[using-systemd]]
[float]
==== Using systemd
Distributions like Debian Jessie, Ubuntu 14, and many of the SUSE derivatives do not use the `chkconfig` tool to register services, but rather `systemd` and its command `/bin/systemctl` to start and stop services (at least in newer versions, otherwise use the `chkconfig` commands above). The configuration file is also placed at `/etc/sysconfig/elasticsearch` if the system is rpm based and `/etc/default/elasticsearch` if it is deb. After installing the RPM, you have to change the systemd configuration and then start up elasticsearch
[source,sh]
--------------------------------------------------
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
sudo /bin/systemctl start elasticsearch.service
--------------------------------------------------
Also note that changing the `MAX_MAP_COUNT` setting in `/etc/sysconfig/elasticsearch` does not have any effect, you will have to change it in `/usr/lib/sysctl.d/elasticsearch.conf` in order to have it applied at startup.

86
docs/reference/setup/backup.asciidoc
View File

@ -1,86 +0,0 @@
[[backup]]
=== Back Up Your Data!
Always back up your data before performing an upgrade. This will allow you to
roll back in the event of a problem. The upgrades sometimes include upgrades
to the Lucene libraries used by Elasticsearch to access the index files, and
after an index file has been updated to work with a new version of Lucene, it
may not be accessible to the versions of Lucene present in earlier
Elasticsearch releases.
[WARNING]
.Always back up your data before upgrading
=========================================
You cannot roll back to an earlier version unless you have a backup of your data.
=========================================
==== Backing up 1.0 and later
To back up a running 1.0 or later system, it is simplest to use the snapshot
feature. See the complete instructions for
<<modules-snapshots,backup and restore with snapshots>>.
==== Backing up 0.90.x and earlier
To back up a running 0.90.x system:
===== Step 1: Disable index flushing
This will prevent indices from being flushed to disk while the backup is in
process:
[source,js]
-----------------------------------
PUT /_all/_settings
{
"index": {
"translog.disable_flush": "true"
}
}
-----------------------------------
// AUTOSENSE
===== Step 2: Disable reallocation
This will prevent the cluster from moving data files from one node to another
while the backup is in process:
[source,js]
-----------------------------------
PUT /_cluster/settings
{
"transient": {
"cluster.routing.allocation.enable": "none"
}
}
-----------------------------------
// AUTOSENSE
===== Step 3: Backup your data
After reallocation and index flushing are disabled, initiate a backup of
Elasticsearch's data path using your favorite backup method (tar, storage
array snapshots, backup software).
===== Step 4: Reenable allocation and flushing
When the backup is complete and data no longer needs to be read from the
Elasticsearch data path, allocation and index flushing must be re-enabled:
[source,js]
-----------------------------------
PUT /_all/_settings
{
"index": {
"translog.disable_flush": "false"
}
}
PUT /_cluster/settings
{
"transient": {
"cluster.routing.allocation.enable": "all"
}
}
-----------------------------------
// AUTOSENSE

45
docs/reference/setup/cluster_restart.asciidoc
View File

@ -2,8 +2,9 @@
=== Full cluster restart upgrade
Elasticsearch requires a full cluster restart when upgrading across major
versions: from 0.x to 1.x or from 1.x to 2.x. Rolling upgrades are not
supported across major versions.
versions. Rolling upgrades are not supported across major versions. Consult
this <<setup-upgrade,table>> to verify that a full cluster restart is
required.
The process to perform an upgrade with a full cluster restart is as follows:
@ -25,20 +26,6 @@ PUT /_cluster/settings
--------------------------------------------------
// AUTOSENSE
If upgrading from 0.90.x to 1.x, then use these settings instead:
[source,js]
--------------------------------------------------
PUT /_cluster/settings
{
"persistent": {
"cluster.routing.allocation.disable_allocation": true,
"cluster.routing.allocation.enable": "none"
}
}
--------------------------------------------------
// AUTOSENSE
==== Step 2: Perform a synced flush
Shard recovery will be much faster if you stop indexing and issue a
@ -59,7 +46,13 @@ multiple times if necessary.
Stop all Elasticsearch services on all nodes in the cluster. Each node can be
upgraded following the same procedure described in <<upgrade-node>>.
==== Step 4: Start the cluster
==== Step 4: Upgrade any plugins
Elasticsearch plugins must be upgraded when upgrading a node. Use the
`elasticsearch-plugin` script to install the correct version of any plugins
that you need.
==== Step 5: Start the cluster
If you have dedicated master nodes -- nodes with `node.master` set to
`true`(the default) and `node.data` set to `false` -- then it is a good idea
@ -82,7 +75,7 @@ GET _cat/nodes
Use these APIs to check that all nodes have successfully joined the cluster.
==== Step 5: Wait for yellow
==== Step 6: Wait for yellow
As soon as each node has joined the cluster, it will start to recover any
primary shards that are stored locally. Initially, the
@ -94,7 +87,7 @@ Once each node has recovered its local shards, the `status` will become
shards are allocated. This is to be expected because allocation is still
disabled.
==== Step 6: Reenable allocation
==== Step 7: Reenable allocation
Delaying the allocation of replicas until all nodes have joined the cluster
allows the master to allocate replicas to nodes which already have local shard
@ -112,20 +105,6 @@ PUT /_cluster/settings
------------------------------------------------------
// AUTOSENSE
If upgrading from 0.90.x to 1.x, then use these settings instead:
[source,js]
--------------------------------------------------
PUT /_cluster/settings
{
"persistent": {
"cluster.routing.allocation.disable_allocation": false,
"cluster.routing.allocation.enable": "all"
}
}
--------------------------------------------------
// AUTOSENSE
The cluster will now start allocating replica shards to all data nodes. At this
point it is safe to resume indexing and searching, but your cluster will
recover more quickly if you can delay indexing and searching until all shards

375
docs/reference/setup/configuration.asciidoc
View File

@ -1,288 +1,76 @@
[[setup-configuration]]
== Configuration
[float]
=== Environment Variables
Within the scripts, Elasticsearch comes with built in `JAVA_OPTS` passed
to the JVM started. The most important setting for that is the `-Xmx` to
control the maximum allowed memory for the process, and `-Xms` to
control the minimum allocated memory for the process (_in general, the
more memory allocated to the process, the better_).
Most times it is better to leave the default `JAVA_OPTS` as they are,
and use the `ES_JAVA_OPTS` environment variable in order to set / change
JVM settings or arguments.
The `ES_HEAP_SIZE` environment variable allows to set the heap memory
that will be allocated to elasticsearch java process. It will allocate
the same value to both min and max values, though those can be set
explicitly (not recommended) by setting `ES_MIN_MEM` (defaults to
`256m`), and `ES_MAX_MEM` (defaults to `1g`).
It is recommended to set the min and max memory to the same value, and
enable <<setup-configuration-memory,`mlockall`>>.
[float]
[[system]]
=== System Configuration
[float]
[[file-descriptors]]
==== File Descriptors
Make sure to increase the number of open files descriptors on the
machine (or for the user running elasticsearch). Setting it to 32k or
even 64k is recommended.
You can retrieve the `max_file_descriptors` for each node
using the <<cluster-nodes-info>> API, with:
[source,js]
--------------------------------------------------
curl localhost:9200/_nodes/stats/process?pretty
--------------------------------------------------
[float]
[[max-number-of-threads]]
==== Number of threads
Make sure that the number of threads that the Elasticsearch user can
create is at least 2048.
[float]
[[vm-max-map-count]]
==== Virtual memory
Elasticsearch uses a <<default_fs,`hybrid mmapfs / niofs`>> directory by default to store its indices. The default
operating system limits on mmap counts is likely to be too low, which may
result in out of memory exceptions. On Linux, you can increase the limits by
running the following command as `root`:
[source,sh]
-------------------------------------
sysctl -w vm.max_map_count=262144
-------------------------------------
To set this value permanently, update the `vm.max_map_count` setting in
`/etc/sysctl.conf`.
NOTE: If you installed Elasticsearch using a package (.deb, .rpm) this setting will be changed automatically. To verify, run `sysctl vm.max_map_count`.
[float]
[[setup-configuration-memory]]
==== Memory Settings
Most operating systems try to use as much memory as possible for file system
caches and eagerly swap out unused application memory, possibly resulting
in the elasticsearch process being swapped. Swapping is very bad for
performance and for node stability, so it should be avoided at all costs.
There are three options:
* **Disable swap**
+
--
The simplest option is to completely disable swap. Usually Elasticsearch
is the only service running on a box, and its memory usage is controlled
by the `ES_HEAP_SIZE` environment variable. There should be no need
to have swap enabled.
On Linux systems, you can disable swap temporarily
by running: `sudo swapoff -a`. To disable it permanently, you will need
to edit the `/etc/fstab` file and comment out any lines that contain the
word `swap`.
On Windows, the equivalent can be achieved by disabling the paging file entirely
via `System Properties → Advanced → Performance → Advanced → Virtual memory`.
--
* **Configure `swappiness`**
+
--
The second option is to ensure that the sysctl value `vm.swappiness` is set
to `0`. This reduces the kernel's tendency to swap and should not lead to
swapping under normal circumstances, while still allowing the whole system
to swap in emergency conditions.
NOTE: From kernel version 3.5-rc1 and above, a `swappiness` of `0` will
cause the OOM killer to kill the process instead of allowing swapping.
You will need to set `swappiness` to `1` to still allow swapping in
emergencies.
--
* **`mlockall`**
+
--
The third option is to use
http://opengroup.org/onlinepubs/007908799/xsh/mlockall.html[mlockall] on Linux/Unix systems, or https://msdn.microsoft.com/en-us/library/windows/desktop/aa366895%28v=vs.85%29.aspx[VirtualLock] on Windows, to
try to lock the process address space into RAM, preventing any Elasticsearch
memory from being swapped out. This can be done, by adding this line
to the `config/elasticsearch.yml` file:
[source,yaml]
--------------
bootstrap.mlockall: true
--------------
After starting Elasticsearch, you can see whether this setting was applied
successfully by checking the value of `mlockall` in the output from this
request:
[source,sh]
--------------
curl http://localhost:9200/_nodes/process?pretty
--------------
If you see that `mlockall` is `false`, then it means that the `mlockall`
request has failed. The most probable reason, on Linux/Unix systems, is that
the user running Elasticsearch doesn't have permission to lock memory. This can
be granted by running `ulimit -l unlimited` as `root` before starting Elasticsearch.
Another possible reason why `mlockall` can fail is that the temporary directory
(usually `/tmp`) is mounted with the `noexec` option. This can be solved by
specifying a new temp directory, by starting Elasticsearch with:
[source,sh]
--------------
./bin/elasticsearch -Djna.tmpdir=/path/to/new/dir
--------------
WARNING: `mlockall` might cause the JVM or shell session to exit if it tries
to allocate more memory than is available!
--
[float]
[[settings]]
=== Elasticsearch Settings
== Configuring Elasticsearch
*elasticsearch* configuration files can be found under `ES_HOME/config`
folder. The folder comes with two files, the `elasticsearch.yml` for
configuring Elasticsearch different
<<modules,modules>>, and `logging.yml` for
configuring the Elasticsearch logging.
Elasticsearch ships with good defaults and requires very little configuration.
Most settings can be changed on a running cluster using the
<<cluster-update-settings>> API.
The configuration files should contain settings which are node-specific (such
as `node.name` and paths), or settings which a node requires in order to be
able to join a cluster, such as `cluster.name` and `network.host`.
[float]
=== Config file location
Elasticsearch has two configuration files:
* `elasticsearch.yml` for configuring Elasticsearch, and
* `logging.yml` for configuring Elasticsearch logging.
These files are located in the config directory, whose location defaults to
`$ES_HOME/config/`. The Debian and RPM packages set the config directory
location to `/etc/elasticsearch/`.
The location of the config directory can be changed with the `path.conf`
setting, as follows:
[source,sh]
-------------------------------
./bin/elasticsearch -E es.path.conf=/path/to/my/config/
-------------------------------
[float]
=== Config file format
The configuration format is http://www.yaml.org/[YAML]. Here is an
example of changing the address all network based modules will use to
bind and publish to:
[source,yaml]
--------------------------------------------------
network :
host : 10.0.0.4
--------------------------------------------------
[float]
[[paths]]
==== Paths
In production use, you will almost certainly want to change paths for
data and log files:
example of changing the path of the data and logs directories:
[source,yaml]
--------------------------------------------------
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch
data: /var/lib/elasticsearch
logs: /var/logs/elasticsearch
--------------------------------------------------
Settings can also be flattened as follows:
[source,yaml]
--------------------------------------------------
path.data: /var/lib/elasticsearch
path.logs: /var/logs/elasticsearch
--------------------------------------------------
[float]
[[cluster-name]]
==== Cluster name
=== Environment variable subsitution
Also, don't forget to give your production cluster a name, which is used
to discover and auto-join other nodes:
Environment variables referenced with the `${...}` notation within the
configuration file will be replaced with the value of the environment
variable, for instance:
[source,yaml]
--------------------------------------------------
cluster:
name: <NAME OF YOUR CLUSTER>
node.name: ${HOSTNAME}
network.host: ${ES_NETWORK_HOST}
--------------------------------------------------
Make sure that you don't reuse the same cluster names in different
environments, otherwise you might end up with nodes joining the wrong cluster.
For instance you could use `logging-dev`, `logging-stage`, and `logging-prod`
for the development, staging, and production clusters.
[float]
[[node-name]]
==== Node name
=== Prompting for settings
You may also want to change the default node name for each node to
something like the display hostname. By default Elasticsearch will
randomly pick a Marvel character name from a list of around 3000 names
when your node starts up.
[source,yaml]
--------------------------------------------------
node:
name: <NAME OF YOUR NODE>
--------------------------------------------------
The hostname of the machine is provided in the environment
variable `HOSTNAME`. If on your machine you only run a
single elasticsearch node for that cluster, you can set
the node name to the hostname using the `${...}` notation:
[source,yaml]
--------------------------------------------------
node:
name: ${HOSTNAME}
--------------------------------------------------
Internally, all settings are collapsed into "namespaced" settings. For
example, the above gets collapsed into `node.name`. This means that
its easy to support other configuration formats, for example,
http://www.json.org[JSON]. If JSON is a preferred configuration format,
simply rename the `elasticsearch.yml` file to `elasticsearch.json` and
add:
[float]
[[styles]]
==== Configuration styles
[source,yaml]
--------------------------------------------------
{
"network" : {
"host" : "10.0.0.4"
}
}
--------------------------------------------------
It also means that its easy to provide the settings externally either
using the `ES_JAVA_OPTS` or as parameters to the `elasticsearch`
command, for example:
[source,sh]
--------------------------------------------------
$ elasticsearch -Ees.network.host=10.0.0.4
--------------------------------------------------
Another option is to set `es.default.` prefix instead of `es.` prefix,
which means the default setting will be used only if not explicitly set
in the configuration file.
Another option is to use the `${...}` notation within the configuration
file which will resolve to an environment setting, for example:
[source,js]
--------------------------------------------------
{
"network" : {
"host" : "${ES_NET_HOST}"
}
}
--------------------------------------------------
Additionally, for settings that you do not wish to store in the configuration
file, you can use the value `${prompt.text}` or `${prompt.secret}` and start
Elasticsearch in the foreground. `${prompt.secret}` has echoing disabled so
that the value entered will not be shown in your terminal; `${prompt.text}`
will allow you to see the value as you type it in. For example:
For settings that you do not wish to store in the configuration file, you can
use the value `${prompt.text}` or `${prompt.secret}` and start Elasticsearch
in the foreground. `${prompt.secret}` has echoing disabled so that the value
entered will not be shown in your terminal; `${prompt.text}` will allow you to
see the value as you type it in. For example:
[source,yaml]
--------------------------------------------------
@ -290,8 +78,8 @@ node:
name: ${prompt.text}
--------------------------------------------------
On execution of the `elasticsearch` command, you will be prompted to enter
the actual value like so:
When starting Elasticsearch, you will be prompted to enter the actual value
like so:
[source,sh]
--------------------------------------------------
@ -302,49 +90,26 @@ NOTE: Elasticsearch will not start if `${prompt.text}` or `${prompt.secret}`
is used in the settings and the process is run as a service or in the background.
[float]
[[configuration-index-settings]]
=== Index Settings
=== Setting default settings
Indices created within the cluster can provide their own settings. For
example, the following creates an index with a refresh interval of 5
seconds instead of the default refresh interval (the format can be either
YAML or JSON):
New default settings may be specified on the command line using the
`es.default.` prefix instead of the `es.` prefix. This will specify a value
that will be used by default unless another value is specified in the config
file.
For instance, if Elasticsearch is started as follows:
[source,sh]
--------------------------------------------------
$ curl -XPUT http://localhost:9200/kimchy/ -d \
'
index:
refresh_interval: 5s
'
--------------------------------------------------
---------------------------
./bin/elasticsearch -E es.default.node.name=My_Node
---------------------------
Index level settings can be set on the node level as well, for example,
within the `elasticsearch.yml` file, the following can be set:
[source,yaml]
--------------------------------------------------
index :
refresh_interval: 5s
--------------------------------------------------
This means that every index that gets created on the specific node
started with the mentioned configuration will use a refresh interval of
5 seconds *unless the index explicitly sets it*. In other words, any
index level settings override what is set in the node configuration. Of
course, the above can also be set as a "collapsed" setting, for example:
[source,sh]
--------------------------------------------------
$ elasticsearch -Ees.index.refresh_interval=5s
--------------------------------------------------
All of the index level configuration can be found within each
<<index-modules,index module>>.
the value for `node.name` will be `My_Node`, unless it is overwritten on the
command line with `es.node.name` or in the config file with `node.name`.
[float]
[[logging]]
=== Logging
== Logging configuration
Elasticsearch uses an internal logging abstraction and comes, out of the
box, with http://logging.apache.org/log4j/1.2/[log4j]. It tries to simplify
@ -367,7 +132,7 @@ out of the box.
[float]
[[deprecation-logging]]
==== Deprecation logging
=== Deprecation logging
In addition to regular logging, Elasticsearch allows you to enable logging
of deprecated actions. For example this allows you to determine early, if

97
docs/reference/setup/dir-layout.asciidoc
View File

@ -1,97 +0,0 @@
[[setup-dir-layout]]
== Directory Layout
The directory layout of an installation is as follows:
[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home | Home of elasticsearch installation. | | path.home
| bin | Binary scripts including `elasticsearch` to start a node. | {path.home}/bin | | conf | Configuration files including `elasticsearch.yml` | {path.home}/config | path.conf
| data | The location of the data files of each index / shard allocated
on the node. Can hold multiple locations. | {path.home}/data| path.data
| logs | Log files location. | {path.home}/logs | path.logs
| plugins | Plugin files location. Each plugin will be contained in a subdirectory. | {path.home}/plugins | path.plugins
| repo | Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here. d| Not configured | path.repo
| script | Location of script files. | {path.conf}/scripts | path.script
|=======================================================================
Multiple `data` paths may be specified, in order to spread data across
multiple disks or locations, but all of the files from a single shard will be
written to the same path. This can be configured as follows:
---------------------------------
path.data: /mnt/first,/mnt/second
---------------------------------
Or in an array format:
----------------------------------------
path.data: ["/mnt/first", "/mnt/second"]
----------------------------------------
TIP: To stripe shards across multiple disks, please use a RAID driver
instead.
[float]
[[default-paths]]
=== Default Paths
Below are the default paths that elasticsearch will use, if not explicitly changed.
[float]
==== deb and rpm
[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Location Debian/Ubuntu | Location RHEL/CentOS
| home | Home of elasticsearch installation. | /usr/share/elasticsearch | /usr/share/elasticsearch
| bin | Binary scripts including `elasticsearch` to start a node. | /usr/share/elasticsearch/bin | /usr/share/elasticsearch/bin
| conf | Configuration files `elasticsearch.yml` and `logging.yml`. | /etc/elasticsearch | /etc/elasticsearch
| conf | Environment variables including heap size, file descriptors. | /etc/default/elasticsearch | /etc/sysconfig/elasticsearch
| data | The location of the data files of each index / shard allocated
on the node. | /var/lib/elasticsearch/data | /var/lib/elasticsearch
| logs | Log files location | /var/log/elasticsearch | /var/log/elasticsearch
| plugins | Plugin files location. Each plugin will be contained in a subdirectory. | /usr/share/elasticsearch/plugins | /usr/share/elasticsearch/plugins
| repo | Shared file system repository locations. d| Not configured d| Not configured
| script | Location of script files. | /etc/elasticsearch/scripts | /etc/elasticsearch/scripts
|=======================================================================
[float]
==== zip and tar.gz
[cols="<h,<,<m",options="header",]
|=======================================================================
| Type | Description | Location
| home | Home of elasticsearch installation | {extract.path}
| bin | Binary scripts including `elasticsearch` to start a node | {extract.path}/bin
| conf | Configuration files `elasticsearch.yml` and `logging.yml` | {extract.path}/config
| data | The location of the data files of each index / shard allocated
on the node | {extract.path}/data
| logs | Log files location | {extract.path}/logs
| plugins | Plugin files location. Each plugin will be contained in a subdirectory | {extract.path}/plugins
| repo | Shared file system repository locations. d| Not configured
| script | Location of script files. | {extract.path}/config/scripts
|=======================================================================

216
docs/reference/setup/important-settings.asciidoc Normal file
View File

@ -0,0 +1,216 @@
[[important-settings]]
== Important Elasticsearch configuration
While Elasticsearch requires very little configuration, there are a number of
settings which need to be configured manually and should definitely be
configured before going into production.
* <<path-settings,`path.data` and `path.logs`>>
* <<cluster.name,`cluster.name`>>
* <<node.name,`node.name`>>
* <<bootstrap.mlockall,`bootstrap.mlockall`>>
* <<network.host,`network.host`>>
* <<unicast.hosts,`discovery.zen.ping.unicast.hosts`>>
* <<minimum_master_nodes,`discovery.zen.minimum_master_nodes`>>
* <<node.max_local_storage_nodes,`node.max_local_storage_nodes`>>
[float]
[[path-settings]]
=== `path.data` and `path.logs`
If you are using the `.zip` or `.tar.gz` archives, the `data` and `logs`
directories are sub-folders of `$ES_HOME`. If these important folders are
left in their default locations, there is a high risk of them being deleted
while upgrading Elasticsearch to a new version.
In production use, you will almost certainly want to change the locations of
the data and log folder:
[source,yaml]
--------------------------------------------------
path:
logs: /var/log/elasticsearch
data: /var/data/elasticsearch
--------------------------------------------------
The RPM and Debian distributions already use custom paths for `data` and
`logs`.
The `path.data` settings can be set to multiple paths, in which case all paths
will be used to store data (although the files belonging to a single shard
will all be stored on the same data path):
[source,yaml]
--------------------------------------------------
path:
data:
- /mnt/elasticsearch_1
- /mnt/elasticsearch_2
- /mnt/elasticsearch_3
--------------------------------------------------
[float]
[[cluster.name]]
=== `cluster.name`
A node can only join a cluster when it shares its `cluster.name` with all the
other nodes in the cluster. The default name is `elasticsearch`, but you
should change it to an appropriate name which describes the purpose of the
cluster.
[source,yaml]
--------------------------------------------------
cluster.name: logging-prod
--------------------------------------------------
Make sure that you don't reuse the same cluster names in different
environments, otherwise you might end up with nodes joining the wrong cluster.
[float]
[[node.name]]
=== `node.name`
By default, Elasticsearch will randomly pick a descriptive `node.name` from a
list of around 3000 Marvel characters when your node starts up, but this also
means that the `node.name` will change the next time the node restarts.
It is worth configuring a more meaningful name which will also have the
advantage of persisting after restarting the node:
[source,yaml]
--------------------------------------------------
node.name: prod-data-2
--------------------------------------------------
The `node.name` can also be set to the server's HOSTNAME as follows:
[source,yaml]
--------------------------------------------------
node.name: ${HOSTNAME}
--------------------------------------------------
[float]
[[bootstrap.mlockall]]
=== `bootstrap.mlockall`
It is vitally important to the health of your node that none of the JVM is
ever swapped out to disk. One way of achieving that is set the
`bootstrap.mlockall` setting to `true`.
For this setting to have effect, other system settings need to be configured
first. See <<mlockall>> for more details about how to set up memory locking
correctly.
[float]
[[network.host]]
=== `network.host`
By default, Elasticsearch binds to loopback addresses only -- e.g. `127.0.0.1`
and `[::1]`. This is sufficient to run a single development node on a server.
TIP: In fact, more than one node can be started from the same `$ES_HOME` location
on a single node. This can be useful for testing Elasticsearch's ability to
form clusters, but it is not a configuration recommended for production.
In order to communicate and to form a cluster with nodes on other servers,
your node will need to bind to a non-loopback address. While there are many
<<modules-network,network settings>>, usually all you need to configure is
`network.host`:
[source,yaml]
--------------------------------------------------
network.host: 192.168.1.10
--------------------------------------------------
The `network.host` setting also understands some special values such as
`_local_`, `_site_`, `_global_` and modifiers like `:ip4` and `:ip6`, details
of which can be found in <<network-interface-values>>.
IMPORTANT: As soon you provide a custom setting for `network.host`,
Elasticsearch assumes that you are moving from development mode to production
mode, and upgrades a number of system startup checks from warnings to
exceptions. See <<dev-vs-prod>> for more information.
[float]
[[unicast.hosts]]
=== `discovery.zen.ping.unicast.hosts`
Out of the box, without any network configuration, Elasticsearch will bind to
the available loopback addresses and will scan ports 9300 to 9305 to try to
connect to other nodes running on the same server. This provides an auto-
clustering experience without having to do any configuration.
When the moment comes to form a cluster with nodes on other servers, you have
to provide a seed list of other nodes in the cluster that are likely to be
live and contactable. This can be specified as follows:
[source,yaml]
--------------------------------------------------
discovery.zen.ping.unicast.hosts:
- 192.168.1.10:9300
- 192.168.1.11 <1>
- seeds.mydomain.com <2>
--------------------------------------------------
<1> The port will default to 9300 if not specified.
<2> A hostname that resolves to multiple IP addresses will try all resolved addresses.
[float]
[[minimum_master_nodes]]
=== `discovery.zen.minimum_master_nodes`
To prevent data loss, it is vital to configure the
`discovery.zen.minimum_master_nodes setting` so that each master-eligible node
knows the _minimum number of master-eligible nodes_ that must be visible in
order to form a cluster.
Without this setting, a cluster that suffers a network failure is at risk of
having the cluster split into two independent clusters -- a split brain --
which will lead to data loss. A more detailed explanation is provided
in <<split-brain>>.
To avoid a split brain, this setting should be set to a _quorum_ of master-
eligible nodes:
(master_eligible_nodes / 2) + 1
In other words, if there are three master-eligible nodes, then minimum master
nodes should be set to `(3 / 2) + 1` or `2`:
[source,yaml]
--------------------------------------------------
discovery.zen.minimum_master_nodes: 2
--------------------------------------------------
IMPORTANT: If `discovery.zen.minimum_master_nodes` is not set when
Elasticsearch is running in <<dev-vs-prod,production mode>>, an exception will
be thrown which will prevent the node from starting.
[float]
[[node.max_local_storage_nodes]]
=== `node.max_local_storage_nodes`
It is possible to start more than one node on the same server from the same
`$ES_HOME`, just by doing the following:
[source,sh]
--------------------------------------------------
./bin/elasticsearch -d
./bin/elasticsearch -d
--------------------------------------------------
This works just fine: the data directory structure is designed to let multiple
nodes coexist. However, a single instance of Elasticsearch is able to use all
of the resources of a single server and it seldom makes sense to run multiple
nodes on the same server in production.
It is, however, possible to start more than one node on the same server by
mistake and to be completely unaware that this problem exists. To prevent more
than one node from sharing the same data directory, it is advisable to add the
following setting:
[source,yaml]
--------------------------------------------------
node.max_local_storage_nodes: 1
--------------------------------------------------

50
docs/reference/setup/install.asciidoc Normal file
View File

@ -0,0 +1,50 @@
[[install-elasticsearch]]
== Installing Elasticsearch
Elasticsearch is provided in the following package formats:
[horizontal]
`zip`/`tar.gz`::
The `zip` and `tar.gz` packages are suitable for installation on any system
and are the easiest choice for getting started with Elasticsearch.
+
<<zip-targz>> or <<windows>>
`deb`::
The `deb` package is suitable for Debian, Ubuntu, and other Debian-based
systems. Debian packages may be downloaded from the Elasticsearch website or
from our Debian repository.
+
<<deb>>
`rpm`::
The `rpm` package is suitable for installation on Red Hat, Centos, SLES,
OpenSuSE and other RPM-based systems. RPMs may be downloaded from the
Elasticsearch website or from our RPM repository.
+
<<rpm>>
[float]
[[config-mgmt-tools]]
=== Configuration Management Tools
We also provide the following configuration management tools to help with
large deployments:
[horizontal]
Puppet:: https://github.com/elastic/puppet-elasticsearch[puppet-elasticsearch]
Chef:: https://github.com/elastic/cookbook-elasticsearch[cookbook-elasticsearch]
include::install/zip-targz.asciidoc[]
include::install/deb.asciidoc[]
include::install/rpm.asciidoc[]
include::install/windows.asciidoc[]

28
docs/reference/setup/install/check-running.asciidoc Normal file
View File

@ -0,0 +1,28 @@
==== Checking that Elasticsearch is running
You can test that your Elasticsearch node is running by sending an HTTP
request to port `9200` on `localhost`:
[source,sh]
--------------------------------------------
curl locahost:9200
--------------------------------------------
which should give you a response something like this:
[source,js]
--------------------------------------------
{
"name" : "Harry Leland",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "5.0.0-alpha1",
"build_hash" : "f27399d",
"build_date" : "2016-03-30T09:51:41.449Z",
"build_snapshot" : false,
"lucene_version" : "6.0.0"
},
"tagline" : "You Know, for Search"
}
--------------------------------------------

191
docs/reference/setup/install/deb.asciidoc Normal file
View File

@ -0,0 +1,191 @@
[[deb]]
=== Install Elasticsearch with Debian Package
The Debian package for Elasticsearch can be <<install-rpm,downloaded from our website>>
or from our <<deb-repo,APT repository>>. It can be used to install
Elasticsearch on any Debian-based system such as Debian and Ubuntu.
The latest stable version of Elasticsearch can be found on the
link:/downloads/elasticsearch[Download Elasticsearch] page. Other versions can
be found on the link:/downloads/past-releases[Past Releases page].
[[deb-key]]
==== Import the Elasticsearch PGP Key
include::key.asciidoc[]
[source,sh]
-------------------------
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
-------------------------
[[deb-repo]]
==== Installing from the APT repository
You may need to install the `apt-transport-https` package on Debian before proceeding:
[source,sh]
--------------------------------------------------
sudo apt-get install apt-transport-https
--------------------------------------------------
Save the repository definition to +/etc/apt/sources.list.d/elasticsearch-{major-version}.list+:
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
echo "deb http://packages.elastic.co/elasticsearch/{major-version}/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-{major-version}.list
--------------------------------------------------
[WARNING]
==================================================
Do not use `add-apt-repository` as it will add a `deb-src` entry as well, but
we do not provide a source package. If you have added the `deb-src` entry, you
will see an error like the following:
Unable to find expected entry 'main/source/Sources' in Release file
(Wrong sources.list entry or malformed file)
Delete the `deb-src` entry from the `/etc/apt/sources.list` file and the
installation should work as expected.
==================================================
You can install the Elasticsearch Debian package with:
[source,sh]
--------------------------------------------------
sudo apt-get update && sudo apt-get install elasticsearch
--------------------------------------------------
[WARNING]
==================================================
If two entries exist for the same Elasticsearch repository, you will see an error like this during `apt-get update`:
["literal",subs="attributes,callouts"]
Duplicate sources.list entry http://packages.elastic.co/elasticsearch/{major-version}/debian/ ...`
Examine +/etc/apt/sources.list.d/elasticsearch-{major-version}.list+ for the duplicate entry or locate the duplicate entry amongst the files in `/etc/apt/sources.list.d/` and the `/etc/apt/sources.list` file.
==================================================
[[install-deb]]
==== Download and install the Debian package manually
The Debian package for Elastisearch v{version} can be downloaded from the website and installed as follows:
["source","sh",subs="attributes"]
--------------------------------------------
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/{version}/elasticsearch-{version}.deb
sha1sum elasticsearch-{version}.deb <1>
sudo dpkg -i elasticsearch-{version}.deb
--------------------------------------------
<1> Compare the SHA produced by `sha1sum` or `shasum` with the
https://download.elastics.co/elasticsearch/release/org/elasticsearch/distribution/deb/elasticsearch/{version}/elasticsearch-{version}.deb.sha1[published SHA].
include::init-systemd.asciidoc[]
[[deb-running-init]]
==== Running Elasticsearch with SysV `init`
Use the `update-rc.d` command to configure Elasticsearch to start automatically
when the system boots up:
[source,sh]
--------------------------------------------------
sudo update-rc.d elasticsearch defaults 95 10
--------------------------------------------------
Elasticsearch can be started and stopped using the `service` command:
[source,sh]
--------------------------------------------
sudo -i service elasticsearch start
sudo -i service elasticsearch stop
--------------------------------------------
If Elasticsearch fails to start for any reason, it will print the reason for
failure to STDOUT. Log files can be found in `/var/log/elasticsearch/`.
[[deb-running-systemd]]
include::systemd.asciidoc[]
[[deb-check-running]]
include::check-running.asciidoc[]
[[deb-configuring]]
==== Configuring Elasticsearch
Elasticsearch loads its configuration from the `/etc/elasticsearch/elasticsearch.yml`
file by default. The format of this config file is explained in
<<settings>>.
The Debian package also has a system configuration file (`/etc/default/elasticsearch`),
which allows you to set the following parameters:
include::sysconfig-file.asciidoc[]
NOTE: Distributions that use `systemd` require that system resource limits be
configured via `systemd` rather than via the `/etc/sysconfig/elasticsearch`
file. See <<systemd>> for more information.
[[deb-layout]]
==== Directory layout of Debian package
The Debian package places config files, logs, and the data directory in the appropriate
locations for a Debian-based system:
[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
| Elasticsearch home directory or `$ES_HOME`
| /usr/share/elasticsearch
d|
| bin
| Binary scripts including `elasticsearch` to start a node
and `elasticsearch-plugin` to install plugins
| /usr/share/elasticsearch/bin
d|
| conf
| Configuration files including `elasticsearch.yml`
| /etc/elasticsearch
| path.conf
| conf
| Environment variables including heap size, file descriptors.
| /etc/default/elasticsearch
d|
| data
| The location of the data files of each index / shard allocated
on the node. Can hold multiple locations.
| /var/lib/elasticsearch
| path.data
| logs
| Log files location.
| /var/logs/elasticsearch
| path.logs
| plugins
| Plugin files location. Each plugin will be contained in a subdirectory.
| /usr/share/elasticsearch/plugins
| path.plugins
| repo
| Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.
d| Not configured
| path.repo
| script
| Location of script files.
| /etc/elasticsearch/scripts
| path.script
|=======================================================================
include::next-steps.asciidoc[]

12
docs/reference/setup/install/init-systemd.asciidoc Normal file
View File

@ -0,0 +1,12 @@
==== SysV `init` vs `systemd`
Elasticsearch is not started automatically after installation. How to start
and stop Elasticsearch depends on whether your sytem uses SysV `init` or
`systemd` (used by newer distributions). You can tell which is being used by
running this command:
[source,sh]
--------------------------------------------
ps -p 1
--------------------------------------------

8
docs/reference/setup/install/key.asciidoc Normal file
View File

@ -0,0 +1,8 @@
We sign all of our packages with the Elasticsearch Signing Key (PGP key
https://pgp.mit.edu/pks/lookup?op=vindex&search=0xD27D666CD88E42B4[D88E42B4],
available from https://pgp.mit.edu) with fingerprint:
4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4
Download and install the public signing key:

11
docs/reference/setup/install/next-steps.asciidoc Normal file
View File

@ -0,0 +1,11 @@
[role="exclude"]
==== Next steps
You now have a test Elasticsearch environment set up. Before you start
serious development or go into production with Elasticsearch, you will need to
do some additional setup:
* Learn how to <<settings,configure Elasticsearch>>.
* Configure <<important-settings,important Elasticsearch settings>>.
* Configure <<system-config,important system settings>>.

182
docs/reference/setup/install/rpm.asciidoc Normal file
View File

@ -0,0 +1,182 @@
[[rpm]]
=== Install Elasticsearch with RPM
The RPM for Elasticsearch can be <<install-rpm,downloaded from our website>>
or from our <<rpm-repo,RPM repository>>. It can be used to install
Elasticsearch on any RPM-based system such as OpenSuSE, SLES, Centos, Red Hat,
and Oracle Enterprise.
The latest stable version of Elasticsearch can be found on the
link:/downloads/elasticsearch[Download Elasticsearch] page. Other versions can
be found on the link:/downloads/past-releases[Past Releases page].
[[rpm-key]]
==== Import the Elasticsearch PGP Key
include::key.asciidoc[]
[source,sh]
-------------------------
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
-------------------------
If using an older version of RPM, you may need to do this in two steps:
[source,sh]
-------------------------
curl https://packages.elastic.co/GPG-KEY-elasticsearch > elasticsearch.asc
rpm --import elasticsearch.asc
-------------------------
[[rpm-repo]]
==== Installing from the RPM repository
Create a file called `elasticsearch.repo` in the `/etc/yum.repos.d/` directory
for RedHat based distributions, or in the `/etc/zypp/repos.d/` directory for
OpenSuSE based distributions, containing:
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
[elasticsearch-{major-version}]
name=Elasticsearch repository for {major-version} packages
baseurl=https://packages.elastic.co/elasticsearch/{major-version}/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
--------------------------------------------------
And your repository is ready for use. You can now install Elasticsearch with one of the following commands:
[source,sh]
--------------------------------------------------
sudo yum install elasticsearch <1>
sudo dnf install elasticsearch <2>
sudo zypper install elasticsearch <3>
--------------------------------------------------
<1> Use `yum` on Centos and older Red Hat distributions.
<2> Use `dnf` on newer Red Hat distributions.
<3> Use `zypper` on OpenSuSe based distributions
[[install-rpm]]
==== Download and install the RPM manually
The RPM for Elastisearch v{version} can be downloaded from the website and installed as follows:
["source","sh",subs="attributes"]
--------------------------------------------
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/{version}/elasticsearch-{version}.rpm
sha1sum elasticsearch-{version}.rpm <1>
sudo rpm --install elasticsearch-{version}.rpm
--------------------------------------------
<1> Compare the SHA produced by `sha1sum` or `shasum` with the
https://download.elastics.co/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/{version}/elasticsearch-{version}.rpm.sha1[published SHA].
include::init-systemd.asciidoc[]
[[rpm-running-init]]
==== Running Elasticsearch with SysV `init`
Use the `chkconfig` command to configure Elasticsearch to start automatically
when the system boots up:
[source,sh]
--------------------------------------------------
sudo chkconfig --add elasticsearch
--------------------------------------------------
Elasticsearch can be started and stopped using the `service` command:
[source,sh]
--------------------------------------------
sudo -i service elasticsearch start
sudo -i service elasticsearch stop
--------------------------------------------
If Elasticsearch fails to start for any reason, it will print the reason for
failure to STDOUT. Log files can be found in `/var/log/elasticsearch/`.
[[rpm-running-systemd]]
include::systemd.asciidoc[]
[[rpm-check-running]]
include::check-running.asciidoc[]
[[rpm-configuring]]
==== Configuring Elasticsearch
Elasticsearch loads its configuration from the `/etc/elasticsearch/elasticsearch.yml`
file by default. The format of this config file is explained in
<<settings>>.
The RPM also has a system configuration file (`/etc/sysconfig/elasticsearch`),
which allows you to set the following parameters:
include::sysconfig-file.asciidoc[]
NOTE: Distributions that use `systemd` require that system resource limits be
configured via `systemd` rather than via the `/etc/sysconfig/elasticsearch`
file. See <<systemd>> for more information.
[[rpm-layout]]
==== Directory layout of RPM
The RPM places config files, logs, and the data directory in the appropriate
locations for an RPM-based system:
[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
| Elasticsearch home directory or `$ES_HOME`
| /usr/share/elasticsearch
d|
| bin
| Binary scripts including `elasticsearch` to start a node
and `elasticsearch-plugin` to install plugins
| /usr/share/elasticsearch/bin
d|
| conf
| Configuration files including `elasticsearch.yml`
| /etc/elasticsearch
| path.conf
| conf
| Environment variables including heap size, file descriptors.
| /etc/sysconfig/elasticsearch
d|
| data
| The location of the data files of each index / shard allocated
on the node. Can hold multiple locations.
| /var/lib/elasticsearch
| path.data
| logs
| Log files location.
| /var/logs/elasticsearch
| path.logs
| plugins
| Plugin files location. Each plugin will be contained in a subdirectory.
| /usr/share/elasticsearch/plugins
| path.plugins
| repo
| Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.
d| Not configured
| path.repo
| script
| Location of script files.
| /etc/elasticsearch/scripts
| path.script
|=======================================================================
include::next-steps.asciidoc[]

71
docs/reference/setup/install/sysconfig-file.asciidoc Normal file
View File

@ -0,0 +1,71 @@
[horizontal]
`ES_USER`::
The user to run as, defaults to `elasticsearch`.
`ES_GROUP`::
The group to run as, defaults to `elasticsearch`.
`ES_HEAP_SIZE`::
The heap size to start with.
`ES_HEAP_NEWSIZE`::
The size of the new generation heap.
`ES_DIRECT_SIZE`::
The maximum size of the direct memory.
`MAX_OPEN_FILES`::
Maximum number of open files, defaults to `65536`.
`MAX_LOCKED_MEMORY`::
Maximum locked memory size. Set to `unlimited if you use the
`bootstrap.mlockall` option in elasticsearch.yml. You must also set
ES_HEAP_SIZE.
`MAX_MAP_COUNT`::
Maximum number of memory map areas a process may have. If you use `mmapfs`
as index store type, make sure this is set to a high value. For more
information, check the
https://github.com/torvalds/linux/blob/master/Documentation/sysctl/vm.txt[linux kernel documentation]
about `max_map_count`. This is set via `sysctl` before starting
elasticsearch. Defaults to `262144`.
`LOG_DIR`::
Log directory, defaults to `/var/log/elasticsearch`.
`DATA_DIR`::
Data directory, defaults to `/var/lib/elasticsearch`.
`CONF_DIR`::
Configuration file directory (which needs to include `elasticsearch.yml`
and `logging.yml` files), defaults to `/etc/elasticsearch`.
`ES_JAVA_OPTS`::
Any additional JVM system properties you may want to apply.
`RESTART_ON_UPGRADE`::
Configure restart on package upgrade, defaults to `false`. This means you
will have to restart your elasticsearch instance after installing a
package manually. The reason for this is to ensure, that upgrades in a
cluster do not result in a continuous shard reallocation resulting in high
network traffic and reducing the response times of your cluster.
`ES_GC_LOG_FILE` ::
The absolute log file path for creating a garbage collection logfile,
which is done by the JVM. Note that this logfile can grow pretty quick and
thus is disabled by default.

30
docs/reference/setup/install/systemd.asciidoc Normal file
View File

@ -0,0 +1,30 @@
==== Running Elasticsearch with `systemd`
To configure Elasticsearch to start automatically when the system boots up,
run the following commands:
[source,sh]
--------------------------------------------------
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
--------------------------------------------------
Elasticsearch can be started and stopped as follows:
[source,sh]
--------------------------------------------
sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service
--------------------------------------------
These commands provide no feedback as to whether Elasticsearch was started
successfully or not. Instead, this information will be written to the
`systemd` journal, which can be tailed as follows:
[source,sh]
--------------------------------------------
sudo journalctl -f
--------------------------------------------
Log files can be found in `/var/log/elasticsearch/`.

174
docs/reference/setup/install/windows.asciidoc Normal file
View File

@ -0,0 +1,174 @@
[[windows]]
=== Install Elasticsearch on Windows
Elasticsearch can be installed on Windows using the `.zip` package. This
comes with a `service.bat` command which will setup Elasticsearch to run as a
service.
The latest stable version of Elasticsearch can be found on the
link:/downloads/elasticsearch[Download Elasticsearch] page.
Other versions can be found on the
link:/downloads/past-releases[Past Releases page].
[[install-windows]]
==== Download and install the `.zip` package
Download the `.zip` archive for Elastisearch v{version} from: https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/{version}/elasticsearch-{version}.zip
Unzip it with your favourite unzip tool. This will create a folder called
+elasticsearch-{version}+, which we will refer to as `$ES_HOME`. In a terminal
window, `CD` to the `$ES_HOME` directory, for instance:
["source","sh",subs="attributes"]
----------------------------
CD c:\elasticsearch-{version}
----------------------------
[[windows-running]]
==== Running Elasticsearch from the command line
Elasticsearch can be started from the command line as follows:
[source,sh]
--------------------------------------------
.\bin\elasticsearch
--------------------------------------------
By default, Elasticsearch runs in the foreground, prints its logs to `STDOUT`,
and can be stopped by pressing `Ctrl-C`.
include::check-running.asciidoc[]
[[windows-service]]
==== Installing Elasticsearch as a Service on Windows
Elasticsearch can be installed as a service to run in the background or start
automatically at boot time without any user interaction. This can be achieved
through the `service.bat` script in the `bin\` folder which allows one to
install, remove, manage or configure the service and potentially start and
stop the service, all from the command-line.
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
c:\elasticsearch-{version}{backslash}bin>service
Usage: service.bat install|remove|start|stop|manager [SERVICE_ID]
--------------------------------------------------
The script requires one parameter (the command to execute) followed by an
optional one indicating the service id (useful when installing multiple
Elasticsearch services).
The commands available are:
[horizontal]
`install`:: Install Elasticsearch as a service
`remove`:: Remove the installed Elasticsearch service (and stop the service if started)
`start`:: Start the Elasticsearch service (if installed)
`stop`:: Stop the Elasticsearch service (if started)
`manager`:: Start a GUI for managing the installed service
Note that the environment configuration options available during the
installation are copied and will be used during the service lifecycle. This
means any changes made to them after the installation will not be picked up
unless the service is reinstalled.
Based on the architecture of the available JDK/JRE (set through `JAVA_HOME`),
the appropriate 64-bit(x64) or 32-bit(x86) service will be installed. This
information is made available during install:
["source","sh",subs="attributes"]
--------------------------------------------------
c:\elasticsearch-{version}{backslash}bin>service install
Installing service : "elasticsearch-service-x64"
Using JAVA_HOME (64-bit): "c:\jvm\jdk1.8"
The service 'elasticsearch-service-x64' has been installed.
--------------------------------------------------
NOTE: While a JRE can be used for the Elasticsearch service, due to its use of a client VM (as oppose to a server JVM which
offers better performance for long-running applications) its usage is discouraged and a warning will be issued.
[float]
=== Customizing service settings
There are two ways to customize the service settings:
Manager GUI:: accessible through `manager` command, the GUI offers insight into the installed service including its status, startup type,
JVM, start and stop settings among other things. Simply invoking `service.bat` from the command-line with the aforementioned option
will open up the manager window:
image::images/service-manager-win.png["Windows Service Manager GUI",align="center"]
Customizing `service.bat`:: at its core, `service.bat` relies on http://commons.apache.org/proper/commons-daemon/[Apache Commons Daemon] project
to install the services. For full flexibility such as customizing the user under which the service runs, one can modify the installation
parameters to tweak all the parameters accordingly. Do note that this requires reinstalling the service for the new settings to be applied.
NOTE: There is also a community supported customizable MSI installer available: https://github.com/salyh/elasticsearch-msi-installer (by Hendrik Saly).
[[windows-layout]]
==== Directory layout of `.zip` archive
The `.zip` package is entirely self-contained. All files and directories are,
by default, contained within `$ES_HOME` -- the directory created when
unpacking the archive.
This is very convenient because you don't have to create any directories to
start using Elasticsearch, and uninstalling Elasticsearch is as easy as
removing the `$ES_HOME` directory. However, it is advisable to change the
default locations of the config directory, the data directory, and the logs
directory so that you do not delete important data later on.
[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
| Elasticsearch home directory or `$ES_HOME`
d| Directory created by unpacking the archive
|
| bin
| Binary scripts including `elasticsearch` to start a node
and `elasticsearch-plugin` to install plugins
| $ES_HOME\bin
d|
| conf
| Configuration files including `elasticsearch.yml`
| $ES_HOME\config
| path.conf
| data
| The location of the data files of each index / shard allocated
on the node. Can hold multiple locations.
| $ES_HOME\data
| path.data
| logs
| Log files location.
| $ES_HOME\logs
| path.logs
| plugins
| Plugin files location. Each plugin will be contained in a subdirectory.
| $ES_HOME\plugins
| path.plugins
| repo
| Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.
d| Not configured
| path.repo
| script
| Location of script files.
| $ES_HOME\scripts
| path.script
|=======================================================================
include::next-steps.asciidoc[]

165
docs/reference/setup/install/zip-targz.asciidoc Normal file
View File

@ -0,0 +1,165 @@
[[zip-targz]]
=== Install Elasticsearch with `.zip` or `.tar.gz`
Elasticsearch is provided as a `.zip` and as a `.tar.gz` package. These
packages can be used to install Elasticsearch on any system and are the
easiest package format to use when trying out Elasticsearch.
The latest stable version of Elasticsearch can be found on the
link:/downloads/elasticsearch[Download Elasticsearch] page.
Other versions can be found on the
link:/downloads/past-releases[Past Releases page].
[[install-zip]]
==== Download and install the `.zip` package
The `.zip` archive for Elastisearch v{version} can be downloaded and installed as follows:
["source","sh",subs="attributes"]
--------------------------------------------
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/{version}/elasticsearch-{version}.zip
sha1sum elasticsearch-{version}.zip <1>
unzip elasticsearch-{version}.zip
cd elasticsearch-{version}/ <2>
--------------------------------------------
<1> Compare the SHA produced by `sha1sum` or `shasum` with the
https://download.elastics.co/elasticsearch/release/org/elasticsearch/distribution/zip/elasticsearch/{version}/elasticsearch-{version}.zip.sha1[published SHA].
<2> This directory is known as `$ES_HOME`.
[[install-targz]]
==== Download and install the `.tar.gz` package
The `.tar.gz` archive for Elastisearch v{version} can be downloaded and installed as follows:
["source","sh",subs="attributes"]
--------------------------------------------
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/{version}/elasticsearch-{version}.tar.gz
sha1sum elasticsearch-{version}.tar.gz <1>
tar -xzf elasticsearch-{version}.tar.gz
cd elasticsearch-{version}/ <2>
--------------------------------------------
<1> Compare the SHA produced by `sha1sum` or `shasum` with the
https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/{version}/elasticsearch-{version}.tar.gz.sha1[published SHA].
<2> This directory is known as `$ES_HOME`.
[[zip-targz-running]]
==== Running Elasticsearch from the command line
Elasticsearch can be started from the command line as follows:
[source,sh]
--------------------------------------------
./bin/elasticsearch
--------------------------------------------
By default, Elasticsearch runs in the foreground, prints its logs to `STDOUT`,
and can be stopped by pressing `Ctrl-C`.
include::check-running.asciidoc[]
[[setup-installation-daemon]]
==== Running as a daemon
To run Elasticsearch as a daemon, specify `-d` on the command line, and record
the process ID in a file using the `-p` option:
[source,sh]
--------------------------------------------
./bin/elasticsearch -d -p pid
--------------------------------------------
Log messages can be found in the `$ES_HOME/logs/` directory.
To shut down Elasticsearch, kill the process ID recorded in the `pid` file:
[source,sh]
--------------------------------------------
kill `cat pid`
--------------------------------------------
NOTE: The startup scripts provided in the <<rpm,RPM>> and <<deb,Debian>>
packages take care of starting and stopping the Elasticsearch process for you.
[[zip-targz-configuring]]
==== Configuring Elasticsearch on the command line
Elasticsearch loads its configuration from the `$ES_HOME/config/elasticsearch.yml`
file by default. The format of this config file is explained in
<<settings>>.
Any settings that can be specified in the config file can also be specified on
the command line, using the `-E` syntax, and prepending `es.` to the setting
name, as follows:
[source,sh]
--------------------------------------------
./bin/elasticsearch -d -E es.cluster.name=my_cluster -E es.node.name=node_1
--------------------------------------------
TIP: Typically, any cluster-wide settings (like `cluster.name`) should be
added to the `elasticsearch.yml` config file, while any node-specific settings
such as `node.name` could be specified on the command line.
[[zip-targz-layout]]
==== Directory layout of `.zip` and `.tar.gz` archives
The `.zip` and `.tar.gz` packages are entirely self-contained. All files and
directories are, by default, contained within `$ES_HOME` -- the directory
created when unpacking the archive.
This is very convenient because you don't have to create any directories to
start using Elasticsearch, and uninstalling Elasticsearch is as easy as
removing the `$ES_HOME` directory. However, it is advisable to change the
default locations of the config directory, the data directory, and the logs
directory so that you do not delete important data later on.
[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
| Elasticsearch home directory or `$ES_HOME`
d| Directory created by unpacking the archive
|
| bin
| Binary scripts including `elasticsearch` to start a node
and `elasticsearch-plugin` to install plugins
| $ES_HOME/bin
d|
| conf
| Configuration files including `elasticsearch.yml`
| $ES_HOME/config
| path.conf
| data
| The location of the data files of each index / shard allocated
on the node. Can hold multiple locations.
| $ES_HOME/data
| path.data
| logs
| Log files location.
| $ES_HOME/logs
| path.logs
| plugins
| Plugin files location. Each plugin will be contained in a subdirectory.
| $ES_HOME/plugins
| path.plugins
| repo
| Shared file system repository locations. Can hold multiple locations. A file system repository can be placed in to any subdirectory of any directory specified here.
d| Not configured
| path.repo
| script
| Location of script files.
| $ES_HOME/scripts
| path.script
|=======================================================================
include::next-steps.asciidoc[]

135
docs/reference/setup/repositories.asciidoc
View File

@ -1,135 +0,0 @@
[[setup-repositories]]
== Repositories
We also have repositories available for APT and YUM based distributions. Note that we only provide
binary packages, but no source packages, as the packages are created as part of the Elasticsearch
build.
We have split the major versions in separate urls to avoid accidental upgrades across major version.
For all 2.x releases use 2.x as version number, for 3.x.y use 3.x etc...
We use the PGP key https://pgp.mit.edu/pks/lookup?op=vindex&search=0xD27D666CD88E42B4[D88E42B4],
Elasticsearch Signing Key, with fingerprint
4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4
to sign all our packages. It is available from https://pgp.mit.edu.
[float]
=== APT
Download and install the Public Signing Key:
[source,sh]
--------------------------------------------------
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
--------------------------------------------------
Save the repository definition to +/etc/apt/sources.list.d/elasticsearch-{major-version}.list+:
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
echo "deb http://packages.elastic.co/elasticsearch/{major-version}/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch-{major-version}.list
--------------------------------------------------
[WARNING]
==================================================
Use the `echo` method described above to add the Elasticsearch repository. Do not use `add-apt-repository`
as it will add a `deb-src` entry as well, but we do not provide a source package.
If you have added the `deb-src` entry, you will see an error like
the following:
Unable to find expected entry 'main/source/Sources' in Release file (Wrong sources.list entry or malformed file)
Just delete the `deb-src` entry from the `/etc/apt/sources.list` file and the installation should work as expected.
==================================================
Run apt-get update and the repository is ready for use. You can install it with:
[source,sh]
--------------------------------------------------
sudo apt-get update && sudo apt-get install elasticsearch
--------------------------------------------------
[WARNING]
==================================================
If two entries exist for the same Elasticsearch repository, you will see an error like this during `apt-get update`:
["literal",subs="attributes,callouts"]
Duplicate sources.list entry http://packages.elastic.co/elasticsearch/{major-version}/debian/ ...`
Examine +/etc/apt/sources.list.d/elasticsearch-{major-version}.list+ for the duplicate entry or locate the duplicate entry amongst the files in `/etc/apt/sources.list.d/` and the `/etc/apt/sources.list` file.
==================================================
Configure Elasticsearch to automatically start during bootup. If your
distribution is using SysV init, then you will need to run:
[source,sh]
--------------------------------------------------
sudo update-rc.d elasticsearch defaults 95 10
--------------------------------------------------
Otherwise if your distribution is using systemd:
[source,sh]
--------------------------------------------------
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
--------------------------------------------------
[float]
=== YUM / DNF
Download and install the public signing key:
[source,sh]
--------------------------------------------------
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
--------------------------------------------------
Add the following in your `/etc/yum.repos.d/` directory
in a file with a `.repo` suffix, for example `elasticsearch.repo`
["source","sh",subs="attributes,callouts"]
--------------------------------------------------
[elasticsearch-{major-version}]
name=Elasticsearch repository for {major-version} packages
baseurl=https://packages.elastic.co/elasticsearch/{major-version}/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
--------------------------------------------------
And your repository is ready for use. You can install it with:
[source,sh]
--------------------------------------------------
yum install elasticsearch
--------------------------------------------------
Or, for newer versions of Fedora and Redhat:
[source,sh]
--------------------------------------------------
dnf install elasticsearch
--------------------------------------------------
Configure Elasticsearch to automatically start during bootup. If your
distribution is using SysV `init` (check with `ps -p 1`), then you will need to run:
WARNING: The repositories do not work with older rpm based distributions
that still use RPM v3, like CentOS5.
[source,sh]
--------------------------------------------------
chkconfig --add elasticsearch
--------------------------------------------------
Otherwise if your distribution is using `systemd`:
[source,sh]
--------------------------------------------------
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
--------------------------------------------------

32
docs/reference/setup/rolling_upgrade.asciidoc
View File

@ -14,10 +14,10 @@ To perform a rolling upgrade:
==== Step 1: Disable shard allocation
When you shut down a node, the allocation process will immediately try to
replicate the shards that were on that node to other nodes in the cluster,
causing a lot of wasted I/O. This can be avoided by disabling allocation
before shutting down a node:
When you shut down a node, the allocation process will wait for one minute
before starting to replicate the shards that were on that node to other nodes
in the cluster, causing a lot of wasted I/O. This can be avoided by disabling
allocation before shutting down a node:
[source,js]
--------------------------------------------------
@ -60,15 +60,15 @@ default.
It is a good idea to place these directories in a different location so that
there is no chance of deleting them when upgrading Elasticsearch. These
custom paths can be <<paths,configured>> with the `path.conf` and
`path.data` settings.
custom paths can be <<path-settings,configured>> with the `path.conf`,
`path.logs`, and `path.data` settings.
The Debian and RPM packages place these directories in the
<<setup-dir-layout,appropriate place>> for each operating system.
The <<deb,Debian>> and <<rpm,RPM>> packages place these directories in the
appropriate place for each operating system.
================================================
To upgrade using a <<setup-repositories,Debian or RPM>> package:
To upgrade using a <<deb,Debian>> or <<rpm,RPM>> package:
* Use `rpm` or `dpkg` to install the new package. All files should be
placed in their proper locations, and config files should not be
@ -87,7 +87,13 @@ To upgrade using a zip or compressed tarball:
to your new installation, or configure the location of the data directory
in the `config/elasticsearch.yml` file, with the `path.data` setting.
==== Step 4: Start the upgraded node
==== Step 4: Upgrade any plugins
Elasticsearch plugins must be upgraded when upgrading a node. Use the
`elasticsearch-plugin` script to install the correct version of any plugins
that you need.
==== Step 5: Start the upgraded node
Start the now upgraded node and confirm that it joins the cluster by checking
the log file or by checking the output of this request:
@ -98,7 +104,7 @@ GET _cat/nodes
--------------------------------------------------
// AUTOSENSE
==== Step 5: Reenable shard allocation
==== Step 6: Reenable shard allocation
Once the node has joined the cluster, reenable shard allocation to start using
the node:
@ -114,7 +120,7 @@ PUT /_cluster/settings
--------------------------------------------------
// AUTOSENSE
==== Step 6: Wait for the node to recover
==== Step 7: Wait for the node to recover
You should wait for the cluster to finish shard allocation before upgrading
the next node. You can check on progress with the <<cat-health,`_cat/health`>>
@ -162,7 +168,7 @@ GET _cat/recovery
If you stopped indexing, then it is safe to resume indexing as soon as
recovery has completed.
==== Step 7: Repeat
==== Step 8: Repeat
When the cluster is stable and the node has recovered, repeat the above steps
for all remaining nodes.

45
docs/reference/setup/sysconfig.asciidoc Normal file
View File

@ -0,0 +1,45 @@
[[system-config]]
== Important System Configuration
Ideally, Elasticsearch should run alone on a server and use all of the
resources available to it. In order to do so, you need to configure your
operating system to allow the user running Elasticsearch to access more
resources than allowed by default.
The following settings *must* be addressed before going to production:
* <<heap-size,Set JVM heap size>>
* <<setup-configuration-memory,Disable swapping>>
* <<file-descriptors,Increase file descriptors>>
* <<vm-max-map-count,Ensure sufficient virtual memory>>
* <<max-number-of-threads,Ensure sufficient threads>>
[[dev-vs-prod]]
[float]
=== Development mode vs production mode
By default, Elasticsearch assumes that you are working in development mode.
If any of the above settings are not configured correctly, a warning will be
written to the log file, but you will be able to start and run your
Elasticsearch node.
As soon as you configure a network setting like `network.host`, Elasticsearch
assumes that you are moving to production and will upgrade the above warnings
to exceptions. These exceptions will prevent your Elasticsearch node from
starting. This is an important safety measure to ensure that you will not
lose data because of a malconfigured server.
include::sysconfig/configuring.asciidoc[]
include::sysconfig/heap_size.asciidoc[]
include::sysconfig/swap.asciidoc[]
include::sysconfig/file-descriptors.asciidoc[]
include::sysconfig/virtual-memory.asciidoc[]
include::sysconfig/threads.asciidoc[]

117
docs/reference/setup/sysconfig/configuring.asciidoc Normal file
View File

@ -0,0 +1,117 @@
[[setting-system-settings]]
=== Configuring system settings
Where to configure systems settings depends on which package you have used to
install Elasticsearch, and which operating system you are using.
When using the `.zip` or `.tar.gz` packages, system settings can be configured:
* temporarily with <<ulimit,`ulimit`>>, or
* permanently in <<limits.conf,`/etc/security/limits.conf`>>.
When using the RPM or Debian packages, most system settings are set in the
<<sysconfig,system configuration file>>. However, systems which use systemd
require that system limits are specified in a
<<systemd,systemd configuration file>>.
[[ulimit]]
==== `ulimit`
On Linux systems, `ulimit` can be used to change resource limits on a
temporary basis. Limits usually need to be set as `root` before switching to
the user that will run Elasticsearch. For example, to set the number of
open file handles (`ulimit -n`) to 65,536, you can do the following:
[source,sh]
--------------------------------
sudo su <1>
ulimit -n 65536 <2>
su elasticsearch <3>
--------------------------------
<1> Become `root`.
<2> Change the max number of open files.
<3> Become the `elasticsearch` user in order to start Elasticsearch.
The new limit is only applied during the current session.
You can consult all currently applied limits with `ulimit -a`.
[[limits.conf]]
==== `/etc/security/limits.conf`
On Linux systems, persistent limits can be set for a particular user by
editing the `/etc/security/limits.conf` file. To set the maximum number of
open files for the `elasticsearch` user to 65,536, add the following line to
the `limits.conf` file:
[source,sh]
--------------------------------
elasticsearch - nofile 65536
--------------------------------
This change will only take effect the next time the `elasticsearch` user opens
a new session.
[NOTE]
.Ubuntu and `limits.conf`
===============================
Ubuntu ignores the `limits.conf` file for processes started by `init.d`. To
enable the `limits.conf` file, edit `/etc/pam.d/su` and uncomment the
following line:
[source,sh]
--------------------------------
# session required pam_limits.so
--------------------------------
===============================
[[sysconfig]]
==== Sysconfig file
When using the RPM or Debian packages, system settings and environment
variables can be specified in the system configuration file, which is located
in:
[horizontal]
RPM:: `/etc/sysconfig/elasticsearch`
Debian:: `/etc/default/elasticsearch`
However, for systems which uses `systemd`, system limits need to be specified
via <<systemd,systemd>>.
[[systemd]]
==== Systemd configuration
When using the RPM or Debian packages on systems that use
https://en.wikipedia.org/wiki/Systemd[systemd], system limits must be
specified via systemd.
The systemd service file (`/usr/lib/systemd/system/elasticsearch.service`)
contains the limits that are applied by default.
To override these, add a file called
`/etc/systemd/system/elasticsearch.service.d/elasticsearch.conf`) and specify
any changes in that file, such as:
[source,sh]
---------------------------------
LimitMEMLOCK=infinity
---------------------------------
[[es-java-opts]]
==== Setting JVM system properties
Any Java system properties or arguments which should be passed to the JVM
should be specified in the `ES_JAVA_OPTS` environment variable, for instance:
[source,sh]
---------------------------------
EXPORT ES_JAVA_OPTS="$ES_JAVA_OPTS -Djava.io.tmpdir=/path/to/temp/dir"
./bin/elasticsearch
---------------------------------
When using the RPM or Debian packages, ES_JAVA_OPTS can be specified in the
<<sysconfig,system configuration file>>.

22
docs/reference/setup/sysconfig/file-descriptors.asciidoc Normal file
View File

@ -0,0 +1,22 @@
[[file-descriptors]]
=== File Descriptors
Elasticsearch uses a lot of file descriptors or file handles. Running out of
file descriptors can be disastrous and will most probably lead to data loss.
Make sure to increase the limit on the number of open files descriptors for
the user running Elasticsearch to 65,536 or higher.
For the `.zip` and `.tar.gz` packages, set <<ulimit,`ulimit -n 65536`>> as
root before starting Elasticsearch, or set `nofile` to `65536` in
<<limits.conf,`/etc/security/limits.conf`>>.
RPM and Debian packages already default the maximum number of file
descriptors to 65536 and do not require further configuration.
You can check the `max_file_descriptors` configured for each node
using the <<cluster-nodes-stats>> API, with:
[source,js]
--------------------------------------------------
curl 'localhost:9200/_nodes/stats/process?pretty&filter_path=**.max_file_descriptors'
--------------------------------------------------

36
docs/reference/setup/sysconfig/heap_size.asciidoc Normal file
View File

@ -0,0 +1,36 @@
[[heap-size]]
=== Set JVM heap size with ES_HEAP_SIZE
In development mode, Elasticsearch tells the JVM to use a heap with a minimum
size of 256MB and a maximum size of 1GB. When moving to production, it is
important to configure the `ES_HEAP_SIZE environment variable to ensure that
Elasticsearch has enough heap available.
Elasticsearch will assign the entire heap specified in `ES_HEAP_SIZE on startup.
The value for this setting depends on the amount of RAM available on your server. A good rule of thumb is:
* The more heap available to Elasticsearch, the more memory it can use for
caching.
* Set the `ES_HEAP_SIZE` to no more than 50% of your RAM, to ensure that there
is enough RAM left for kernel file system caches.
* Don’t set `ES_HEAP_SIZE` to more than about 31GB to ensure that the JVM uses
compressed object pointers (compressed-oops). You can verify that you are
under the limit by looking for a line in the logs like the following:
heap size [1.9gb], compressed ordinary object pointers [true]
Here are examples of how to set the ES_HEAP_SIZE:
[source,sh]
------------------
ES_HEAP_SIZE=2g ./bin/elasticsearch <1>
ES_HEAP_SIZE=4000m ./bin/elasticsearch <2>
------------------
<1> Set the min/max heap size to 2GB.
<2> Set the min/max heap size to 4,000MB.
When using the Debian or RPM package, ES_HEAP_SIZE can be configured in the
<<sysconfig,system configuration file>>.

104
docs/reference/setup/sysconfig/swap.asciidoc Normal file
View File

@ -0,0 +1,104 @@
[[setup-configuration-memory]]
=== Disable swapping
Most operating systems try to use as much memory as possible for file system
caches and eagerly swap out unused application memory. This can result in
parts of the JVM heap being swapped out to disk.
Swapping is very bad for performance and for node stability and should be
avoided at all costs. It can cause garbage collections to last for **minutes**
instead of milliseconds and can cause nodes to respond slowly or even to
disconnect from the cluster.
There are three approaches to disabling swapping:
[[mlockall]]
==== Enable `bootstrap.mlockall`
The first option is to use
http://opengroup.org/onlinepubs/007908799/xsh/mlockall.html[mlockall] on Linux/Unix systems, or https://msdn.microsoft.com/en-us/library/windows/desktop/aa366895%28v=vs.85%29.aspx[VirtualLock] on Windows, to
try to lock the process address space into RAM, preventing any Elasticsearch
memory from being swapped out. This can be done, by adding this line
to the `config/elasticsearch.yml` file:
[source,yaml]
--------------
bootstrap.mlockall: true
--------------
WARNING: `mlockall` might cause the JVM or shell session to exit if it tries
to allocate more memory than is available!
After starting Elasticsearch, you can see whether this setting was applied
successfully by checking the value of `mlockall` in the output from this
request:
[source,sh]
--------------
curl 'http://localhost:9200/_nodes?pretty&filter_path=**.mlockall'
--------------
If you see that `mlockall` is `false`, then it means that the `mlockall`
request has failed. You will also see a line with more information in the
logs with the words `Unable to lock JVM Memory`.
The most probable reason, on Linux/Unix systems, is that the user running
Elasticsearch doesn't have permission to lock memory. This can be granted as follows:
`.zip` and `.tar.gz`::
Set <<ulimit,`ulimit -l unlimited`>> as root before starting Elasticsearch,
or set `memlock` to `unlimited` in
<<limits.conf,`/etc/security/limits.conf`>>.
RPM and Debian::
Set `MAX_LOCKED_MEMORY` to `unlimited` in the
<<sysconfig,system configuration file>> (or see below for systems using `systemd`).
Systems using `systemd`::
Set `LimitMEMLOCK` to `infinity` in the <<systemd,systemd configuration>>.
Another possible reason why `mlockall` can fail is that the temporary directory
(usually `/tmp`) is mounted with the `noexec` option. This can be solved by
specifying a new temp directory, by starting Elasticsearch with:
[source,sh]
--------------
./bin/elasticsearch -Djava.io.tmpdir=/path/to/temp/dir
--------------
or using the `ES_JAVA_OPTS` environment variable:
[source,sh]
--------------
export ES_JAVA_OPTS="$ES_JAVA_OPTS -Djava.io.tmpdir=/path/to/temp/dir"
./bin/elasticsearch
--------------
[[disable-swap-files]]
==== Disable all swap files
The second option is to completely disable swap. Usually Elasticsearch
is the only service running on a box, and its memory usage is controlled
by the `ES_HEAP_SIZE` environment variable. There should be no need
to have swap enabled.
On Linux systems, you can disable swap temporarily
by running: `sudo swapoff -a`. To disable it permanently, you will need
to edit the `/etc/fstab` file and comment out any lines that contain the
word `swap`.
On Windows, the equivalent can be achieved by disabling the paging file entirely
via `System Properties → Advanced → Performance → Advanced → Virtual memory`.
[[swappiness]]
==== Configure `swappiness`
The second option available on Linux systems is to ensure that the sysctl value
`vm.swappiness` is set to `1`. This reduces the kernel's tendency to swap and
should not lead to swapping under normal circumstances, while still allowing
the whole system to swap in emergency conditions.

12
docs/reference/setup/sysconfig/threads.asciidoc Normal file
View File

@ -0,0 +1,12 @@
[[max-number-of-threads]]
=== Number of threads
Elasticsearch uses a number of thread pools for different types of operations.
It is important that it is able to created new threads whenever needed. Make
sure that the number of threads that the Elasticsearch user can create is at
least 2048.
This can be done by setting <<ulimit,`ulimit -u 2048`>> as root before
starting Elasticsearch, or by setting `nproc` to `2048` in
<<limits.conf,`/etc/security/limits.conf`>>.

20
docs/reference/setup/sysconfig/virtual-memory.asciidoc Normal file
View File

@ -0,0 +1,20 @@
[[vm-max-map-count]]
=== Virtual memory
Elasticsearch uses a <<default_fs,`hybrid mmapfs / niofs`>> directory by
default to store its indices. The default operating system limits on mmap
counts is likely to be too low, which may result in out of memory exceptions.
On Linux, you can increase the limits by running the following command as
`root`:
[source,sh]
-------------------------------------
sysctl -w vm.max_map_count=262144
-------------------------------------
To set this value permanently, update the `vm.max_map_count` setting in
`/etc/sysctl.conf`. To verify after rebooting, run `sysctl vm.max_map_count`.
The RPM and Debian packages will configure this setting automatically. No
further configuration is required.

36
docs/reference/setup/upgrade.asciidoc
View File

@ -7,8 +7,9 @@ Before upgrading Elasticsearch:
* Consult the <<breaking-changes,breaking changes>> docs.
* Test upgrades in a dev environment before upgrading your production cluster.
* Always back up your data before upgrading. You cannot roll back to an earlier
version unless you have a <<backup,backup>> of your data.
* Always <<modules-snapshots,back up your data>> before upgrading.
You **cannot roll back** to an earlier version unless you have a backup of your data.
* If you are using custom plugins, check that a compatible version is available.
===========================================
Elasticsearch can usually be upgraded using a rolling upgrade process,
@ -21,33 +22,14 @@ consult this table:
[cols="1<m,1<m,3",options="header",]
|=======================================================================
|Upgrade From |Upgrade To |Supported Upgrade Type
|0.90.x |2.x |<<restart-upgrade,Full cluster restart>>
|1.x |2.x |<<restart-upgrade,Full cluster restart>>
|2.x |2.y |<<rolling-upgrades,Rolling upgrade>> (where `y > x `)
|2.x |3.x |<<restart-upgrade,Full cluster restart>>
|2.x |2.y |<<rolling-upgrades,Rolling upgrade>> (where `y > x`)
|5.x |5.y |<<rolling-upgrades,Rolling upgrade>> (where `y > x`)
|2.x |5.x |<<restart-upgrade,Full cluster restart>>
|5.0.0-alpha1 |5.y |<<restart-upgrade,Full cluster restart>>
|5.0.0-alpha2 |5.y |<<restart-upgrade,Full cluster restart>>
|5.0.0-beta1 |5.y |<<restart-upgrade,Full cluster restart>>
|=======================================================================
===========================================
Upgrading Elasticsearch with Plugins
Take plugins into consideration as well when upgrading. Plugins must be upgraded alongside Elasticsearch.
Check with your plugin's provider to ensure that the plugin is compatible with
your targeted version of Elasticsearch. If doing a rolling upgrade, it may be
worth checking as well that the plugin works across a mixed-version cluster.
The process for both <<rolling-upgrades,Rolling upgrade>> and
<<restart-upgrade,Full cluster restart>> is generally as follows, per node.
* Shut down Elasticsearch
* Upgrade Elasticsearch
* Upgrade Plugins
* Start up Elasticsearch
===========================================
include::backup.asciidoc[]
include::rolling_upgrade.asciidoc[]
include::cluster_restart.asciidoc[]