OpenSearch/docs/reference/setup/install/docker.asciidoc

[[docker]]
=== Install {es} with Docker

{es} is also available as Docker images.
The images use https://hub.docker.com/_/centos/[centos:8] as the base image.

A list of all published Docker images and tags is available at
https://www.docker.elastic.co[www.docker.elastic.co]. The source files
are in
https://github.com/elastic/elasticsearch/blob/{branch}/distribution/docker[Github].

These images are free to use under the Elastic license. They contain open source
and free commercial features and access to paid commercial features.
{kibana-ref}/managing-licenses.html[Start a 30-day trial] to try out all of the
paid commercial features. See the
https://www.elastic.co/subscriptions[Subscriptions] page for information about
Elastic license levels.

==== Pulling the image

Obtaining {es} for Docker is as simple as issuing a +docker pull+ command
against the Elastic Docker registry.

ifeval::["{release-state}"=="unreleased"]

WARNING: Version {version} of {es} has not yet been released, so no
Docker image is currently available for this version.

endif::[]

ifeval::["{release-state}"!="unreleased"]

[source,sh,subs="attributes"]
--------------------------------------------
docker pull {docker-repo}:{version}
--------------------------------------------

Alternatively, you can download other Docker images that contain only features
available under the Apache 2.0 license. To download the images, go to
https://www.docker.elastic.co[www.docker.elastic.co].

endif::[]

[[docker-cli-run-dev-mode]]
==== Starting a single node cluster with Docker

ifeval::["{release-state}"=="unreleased"]

WARNING: Version {version} of the {es} Docker image has not yet been released.

endif::[]

ifeval::["{release-state}"!="unreleased"]

To start a single-node {es} cluster for development or testing, specify
<<single-node-discovery,single-node discovery>> to bypass the <<bootstrap-checks,bootstrap checks>>:

[source,sh,subs="attributes"]
--------------------------------------------
docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" {docker-image}
--------------------------------------------

endif::[]

[[docker-compose-file]]
==== Starting a multi-node cluster with Docker Compose

To get a three-node {es} cluster up and running in Docker,
you can use Docker Compose:

. Create a `docker-compose.yml` file:
ifeval::["{release-state}"=="unreleased"]
+
--
WARNING: Version {version} of {es} has not yet been released, so a
`docker-compose.yml` is not available for this version.

endif::[]

ifeval::["{release-state}"!="unreleased"]
[source,yaml,subs="attributes"]
--------------------------------------------
include::docker-compose.yml[]
--------------------------------------------
endif::[]

This sample Docker Compose file brings up a three-node {es} cluster.
Node `es01` listens on `localhost:9200` and `es02` and `es03` talk to `es01` over a Docker network.

Please note that this configuration exposes port 9200 on all network interfaces, and given how 
Docker manipulates `iptables` on Linux, this means that your {es} cluster is publically accessible, 
potentially ignoring any firewall settings. If you don't want to expose port 9200 and instead use 
a reverse proxy, replace `9200:9200` with `127.0.0.1:9200:9200` in the docker-compose.yml file.
{es} will then only be accessible from the host machine itself.

The  https://docs.docker.com/storage/volumes[Docker named volumes]
`data01`, `data02`, and `data03` store the node data directories so the data persists across restarts.
If they don't already exist, `docker-compose` creates them when you bring up the cluster.
--
. Make sure Docker Engine is allotted at least 4GiB of memory.
In Docker Desktop, you configure resource usage on the Advanced tab in Preference (macOS)
or Settings (Windows).
+
NOTE: Docker Compose is not pre-installed with Docker on Linux.
See docs.docker.com for installation instructions:
https://docs.docker.com/compose/install[Install Compose on Linux]

. Run `docker-compose` to bring up the cluster:
+
[source,sh,subs="attributes"]
--------------------------------------------
docker-compose up
--------------------------------------------

. Submit a `_cat/nodes` request to see that the nodes are up and running:
+
[source,sh]
--------------------------------------------------
curl -X GET "localhost:9200/_cat/nodes?v&pretty"
--------------------------------------------------
// NOTCONSOLE

Log messages go to the console and are handled by the configured Docker logging driver.
By default you can access logs with `docker logs`.

To stop the cluster, run `docker-compose down`.
The data in the Docker volumes is preserved and loaded
when you restart the cluster with `docker-compose up`.
To **delete the data volumes** when you bring down the cluster,
specify the `-v` option: `docker-compose down -v`.


[[next-getting-started-tls-docker]]
===== Start a multi-node cluster with TLS enabled

See <<configuring-tls-docker>> and
{stack-gs}/get-started-docker.html#get-started-docker-tls[Run the {stack} in Docker with TLS enabled].

[[docker-prod-prerequisites]]
==== Using the Docker images in production

The following requirements and recommendations apply when running {es} in Docker in production.

===== Set `vm.max_map_count` to at least `262144`

The `vm.max_map_count` kernel setting must be set to at least `262144` for production use.

How you set `vm.max_map_count` depends on your platform:

* Linux
+
--
The `vm.max_map_count` setting should be set permanently in `/etc/sysctl.conf`:
[source,sh]
--------------------------------------------
grep vm.max_map_count /etc/sysctl.conf
vm.max_map_count=262144
--------------------------------------------

To apply the setting on a live system, run:

[source,sh]
--------------------------------------------
sysctl -w vm.max_map_count=262144
--------------------------------------------
--

* macOS with  https://docs.docker.com/docker-for-mac[Docker for Mac]
+
--
The `vm.max_map_count` setting must be set within the xhyve virtual machine:

. From the command line, run:
+
[source,sh]
--------------------------------------------
screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
--------------------------------------------

. Press enter and use`sysctl` to configure `vm.max_map_count`:
+
[source,sh]
--------------------------------------------
sysctl -w vm.max_map_count=262144
--------------------------------------------

. To exit the `screen` session, type `Ctrl a d`.
--

* Windows and macOS with https://www.docker.com/products/docker-desktop[Docker Desktop]
+
--
The `vm.max_map_count` setting must be set via docker-machine:

[source,sh]
--------------------------------------------
docker-machine ssh
sudo sysctl -w vm.max_map_count=262144
--------------------------------------------
--

* Windows with https://docs.docker.com/docker-for-windows/wsl[Docker Desktop WSL 2 backend]
+
--
The `vm.max_map_count` setting must be set in the docker-desktop container:

[source,sh]
--------------------------------------------
wsl -d docker-desktop
sysctl -w vm.max_map_count=262144
--------------------------------------------
--

===== Configuration files must be readable by the `elasticsearch` user

By default, {es} runs inside the container as user `elasticsearch` using
uid:gid `1000:0`.

IMPORTANT: One exception is https://docs.openshift.com/container-platform/3.6/creating_images/guidelines.html#openshift-specific-guidelines[Openshift],
which runs containers using an arbitrarily assigned user ID.
Openshift presents persistent volumes with the gid set to `0`, which works without any adjustments.

If you are bind-mounting a local directory or file, it must be readable by the `elasticsearch` user.
In addition, this user must have write access to the <<path-settings,data and log dirs>>.
A good strategy is to grant group access to gid `0` for the local directory.

For example, to prepare a local directory for storing data through a bind-mount:

[source,sh]
--------------------------------------------
mkdir esdatadir
chmod g+rwx esdatadir
chgrp 0 esdatadir
--------------------------------------------

As a last resort, you can force the container to mutate the ownership of
any bind-mounts used for the <<path-settings,data and log dirs>> through the
environment variable `TAKE_FILE_OWNERSHIP`. When you do this, they will be owned by
uid:gid `1000:0`, which provides the required read/write access to the {es} process.


===== Increase ulimits for nofile and nproc

Increased ulimits for <<setting-system-settings,nofile>> and <<max-number-threads-check,nproc>>
must be available for the {es} containers.
Verify the https://github.com/moby/moby/tree/ea4d1243953e6b652082305a9c3cda8656edab26/contrib/init[init system]
for the Docker daemon sets them to acceptable values.

To check the Docker daemon defaults for ulimits, run:

[source,sh]
--------------------------------------------
docker run --rm centos:8 /bin/bash -c 'ulimit -Hn && ulimit -Sn && ulimit -Hu && ulimit -Su'
--------------------------------------------

If needed, adjust them in the Daemon or override them per container.
For example, when using `docker run`, set:

[source,sh]
--------------------------------------------
--ulimit nofile=65535:65535
--------------------------------------------

===== Disable swapping

Swapping needs to be disabled for performance and node stability.
For information about ways to do this, see <<setup-configuration-memory>>.

If you opt for the `bootstrap.memory_lock: true` approach,
you also need to define the `memlock: true` ulimit in the
https://docs.docker.com/engine/reference/commandline/dockerd/#default-ulimits[Docker Daemon],
or explicitly set for the container as shown in the  <<docker-compose-file, sample compose file>>.
When using `docker run`, you can specify:

  -e "bootstrap.memory_lock=true" --ulimit memlock=-1:-1

===== Randomize published ports

The image https://docs.docker.com/engine/reference/builder/#/expose[exposes]
TCP ports 9200 and 9300. For production clusters, randomizing the
published ports with `--publish-all` is recommended,
unless you are pinning one container per host.

[[docker-set-heap-size]]
===== Set the heap size

To configure the heap size, you can bind mount a <<jvm-options,JVM options>>
file under `/usr/share/elasticsearch/config/jvm.options.d` that includes your
desired <<heap-size,heap size>> settings. Note that while the default root
`jvm.options` file sets a default heap of 1 GB, any value you set in a
bind-mounted JVM options file will override it.

While setting the heap size via bind-mounted JVM options is the recommended
method, you can also configure this by using the `ES_JAVA_OPTS` environment
variable to set the heap size. For example, to use 16 GB, specify
`-e ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. Note that while the
default root `jvm.options` file sets a default heap of 1 GB, any value you set
in `ES_JAVA_OPTS` will override it. The `docker-compose.yml` file above sets the heap size to 512 MB.

IMPORTANT: You must <<heap-size,configure the heap size>> even if you are
https://docs.docker.com/config/containers/resource_constraints/#limit-a-containers-access-to-memory[limiting
memory access] to the container.

===== Pin deployments to a specific image version

Pin your deployments to a specific version of the {es} Docker image. For
example +docker.elastic.co/elasticsearch/elasticsearch:{version}+.

===== Always bind data volumes

You should use a volume bound on `/usr/share/elasticsearch/data` for the following reasons:

.  The data of your {es} node won't be lost if the container is killed

. {es} is I/O sensitive and the Docker storage driver is not ideal for fast I/O

. It allows the use of advanced
https://docs.docker.com/engine/extend/plugins/#volume-plugins[Docker volume plugins]

===== Avoid using `loop-lvm` mode

If you are using the devicemapper storage driver, do not use the default `loop-lvm` mode.
Configure docker-engine to use
https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-docker-with-devicemapper[direct-lvm].

===== Centralize your logs

Consider centralizing your logs by using a different
https://docs.docker.com/engine/admin/logging/overview/[logging driver]. Also
note that the default json-file logging driver is not ideally suited for
production use.

[[docker-configuration-methods]]
==== Configuring {es} with Docker

When you run in Docker, the <<config-files-location,{es} configuration files>> are loaded from
`/usr/share/elasticsearch/config/`.

To use custom configuration files, you <<docker-config-bind-mount, bind-mount the files>>
over the configuration files in the image.

You can set individual {es} configuration parameters using Docker environment variables.
The <<docker-compose-file, sample compose file>> and the
<<docker-cli-run-dev-mode, single-node example>> use this method.

To use the contents of a file to set an environment variable, suffix the environment
variable name with `_FILE`. This is useful for passing secrets such as passwords to {es}
without specifying them directly.

For example, to set the {es} bootstrap password from a file, you can bind mount the
file and set the `ELASTIC_PASSWORD_FILE` environment variable to the mount location.
If you mount the password file to `/run/secrets/password.txt`, specify:

[source,sh]
--------------------------------------------
-e ELASTIC_PASSWORD_FILE=/run/secrets/bootstrapPassword.txt
--------------------------------------------

You can also override the default command for the image to pass {es} configuration
parameters as command line options. For example:

[source,sh]
--------------------------------------------
docker run <various parameters> bin/elasticsearch -Ecluster.name=mynewclustername
--------------------------------------------

While bind-mounting your configuration files is usually the preferred method in production,
you can also <<_c_customized_image, create a custom Docker image>>
that contains your configuration.

[[docker-config-bind-mount]]
===== Mounting {es} configuration files

Create custom config files and bind-mount them over the corresponding files in the Docker image.
For example, to bind-mount `custom_elasticsearch.yml` with `docker run`, specify:

[source,sh]
--------------------------------------------
-v full_path_to/custom_elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
--------------------------------------------

IMPORTANT: The container **runs {es} as user `elasticsearch` using
uid:gid `1000:0`**. Bind mounted host directories and files must be accessible by this user,
and the data and log directories must be writable by this user.

[[docker-keystore-bind-mount]]
===== Mounting an {es} keystore

By default, {es} will auto-generate a keystore file for secure settings. This
file is obfuscated but not encrypted. If you want to encrypt your
<<secure-settings,secure settings>> with a password, you must use the
`elasticsearch-keystore` utility to create a password-protected keystore and
bind-mount it to the container as
`/usr/share/elasticsearch/config/elasticsearch.keystore`. In order to provide
the Docker container with the password at startup, set the Docker environment
value `KEYSTORE_PASSWORD` to the value of your password. For example, a `docker
run` command might have the following options:

[source, sh]
--------------------------------------------
-v full_path_to/elasticsearch.keystore:/usr/share/elasticsearch/config/elasticsearch.keystore
-E KEYSTORE_PASSWORD=mypassword
--------------------------------------------

[[_c_customized_image]]
===== Using custom Docker images
In some environments, it might make more sense to prepare a custom image that contains
your configuration. A `Dockerfile` to achieve this might be as simple as:

[source,sh,subs="attributes"]
--------------------------------------------
FROM docker.elastic.co/elasticsearch/elasticsearch:{version}
COPY --chown=elasticsearch:elasticsearch elasticsearch.yml /usr/share/elasticsearch/config/
--------------------------------------------

You could then build and run the image with:

[source,sh]
--------------------------------------------
docker build --tag=elasticsearch-custom .
docker run -ti -v /usr/share/elasticsearch/data elasticsearch-custom
--------------------------------------------

Some plugins require additional security permissions.
You must explicitly accept them either by:

* Attaching a `tty` when you run the Docker image and allowing the permissions when prompted.
* Inspecting the security permissions and accepting them (if appropriate) by adding the `--batch` flag to the plugin install command.

See {plugins}/_other_command_line_parameters.html[Plugin management]
for more information.

include::next-steps.asciidoc[]