This commit is contained in:
James Rodewig
GitHub
parent
93987c8297
commit
88634bc022
|
|
@ -2,32 +2,32 @@
|
|||
[discrete]
|
||||
=== Path settings
|
||||
|
||||
If you are using the `.zip` or `.tar.gz` archives, the `data` and `logs`
|
||||
directories are sub-folders of `$ES_HOME`. If these important folders are left
|
||||
in their default locations, there is a high risk of them being deleted while
|
||||
upgrading {es} to a new version.
|
||||
For <<targz,macOS `.tar.gz`>>, <<targz,Linux `.tar.gz`>>, and
|
||||
<<zip-windows,Windows `.zip`>> installations, {es} writes data and logs to the
|
||||
respective `data` and `logs` subdirectories of `$ES_HOME` by default.
|
||||
However, files in `$ES_HOME` risk deletion during an upgrade.
|
||||
|
||||
In production use, you will almost certainly want to change the locations of the
|
||||
`path.data` and `path.logs` folders:
|
||||
In production, we strongly recommend you set the `path.data` and `path.logs` in
|
||||
`elasticsearch.yml` to locations outside of `$ES_HOME`.
|
||||
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
path:
|
||||
logs: /var/log/elasticsearch
|
||||
data: /var/data/elasticsearch
|
||||
--------------------------------------------------
|
||||
TIP: <<docker,Docker>>, <<deb,Debian>>, <<rpm,RPM>>, <<brew,macOS Homebrew>>,
|
||||
and <<windows,Windows `.msi`>> installations write data and log to locations
|
||||
outside of `$ES_HOME` by default.
|
||||
|
||||
The RPM and Debian distributions already use custom paths for `data` and `logs`.
|
||||
Supported `path.data` and `path.logs` values vary by platform:
|
||||
|
||||
The `path.data` settings can be set to multiple paths, in which case all paths
|
||||
will be used to store data. However, the files belonging to a single shard will
|
||||
all be stored on the same data path:
|
||||
include::{es-repo-dir}/tab-widgets/code.asciidoc[]
|
||||
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
path:
|
||||
data:
|
||||
- /mnt/elasticsearch_1
|
||||
- /mnt/elasticsearch_2
|
||||
- /mnt/elasticsearch_3
|
||||
--------------------------------------------------
|
||||
include::{es-repo-dir}/tab-widgets/customize-data-log-path-widget.asciidoc[]
|
||||
|
||||
If needed, you can specify multiple paths in `path.data`. {es} stores the node's
|
||||
data across all provided paths but keeps each shard's data on the same path.
|
||||
|
||||
WARNING: {es} does not balance shards across a node's data paths. High disk
|
||||
usage in a single path can trigger a <<disk-based-shard-allocation,high disk
|
||||
usage watermark>> for the entire node. If triggered, {es} will not add shards to
|
||||
the node, even if the node’s other paths have available disk space. If you need
|
||||
additional disk space, we recommend you add a new node rather than additional
|
||||
data paths.
|
||||
|
||||
include::{es-repo-dir}/tab-widgets/multi-data-path-widget.asciidoc[]
|
||||
|
|
@ -100,68 +100,20 @@ are left untouched and in place.
|
|||
[[snapshots-filesystem-repository]]
|
||||
=== Shared file system repository
|
||||
|
||||
The shared file system repository (`"type": "fs"`) uses the shared file system to store snapshots. In order to register
|
||||
the shared file system repository it is necessary to mount the same shared filesystem to the same location on all
|
||||
master and data nodes. This location (or one of its parent directories) must be registered in the `path.repo`
|
||||
setting on all master and data nodes.
|
||||
Use a shared file system repository (`"type": "fs"`) to store snapshots on a
|
||||
shared file system.
|
||||
|
||||
Assuming that the shared filesystem is mounted to `/mount/backups/my_fs_backup_location`, the following setting should
|
||||
be added to `elasticsearch.yml` file:
|
||||
To register a shared file system repository, first mount the file system to the
|
||||
same location on all master and data nodes. Then add the file system's
|
||||
path or parent directory to the `path.repo` setting in `elasticsearch.yml` for
|
||||
each master and data node. For running clusters, this requires a
|
||||
<<restart-cluster-rolling,rolling restart>> of each node.
|
||||
|
||||
[source,yaml]
|
||||
--------------
|
||||
path.repo: ["/mount/backups", "/mount/longterm_backups"]
|
||||
--------------
|
||||
Supported `path.repo` values vary by platform:
|
||||
|
||||
The `path.repo` setting supports Microsoft Windows UNC paths as long as at least server name and share are specified as
|
||||
a prefix and back slashes are properly escaped:
|
||||
include::{es-repo-dir}/tab-widgets/code.asciidoc[]
|
||||
|
||||
[source,yaml]
|
||||
--------------
|
||||
path.repo: ["\\\\MY_SERVER\\Snapshots"]
|
||||
--------------
|
||||
|
||||
After all nodes are restarted, the following command can be used to register the shared file system repository with
|
||||
the name `my_fs_backup`:
|
||||
|
||||
[source,console]
|
||||
-----------------------------------
|
||||
PUT /_snapshot/my_fs_backup
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "/mount/backups/my_fs_backup_location",
|
||||
"compress": true
|
||||
}
|
||||
}
|
||||
-----------------------------------
|
||||
// TEST[skip:no access to absolute path]
|
||||
|
||||
If the repository location is specified as a relative path this path will be resolved against the first path specified
|
||||
in `path.repo`:
|
||||
|
||||
[source,console]
|
||||
-----------------------------------
|
||||
PUT /_snapshot/my_fs_backup
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "my_fs_backup_location",
|
||||
"compress": true
|
||||
}
|
||||
}
|
||||
-----------------------------------
|
||||
// TEST[continued]
|
||||
|
||||
The following settings are supported:
|
||||
|
||||
`location`:: Location of the snapshots. Mandatory.
|
||||
`compress`:: Turns on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. Defaults to `true`.
|
||||
`chunk_size`:: Big files can be broken down into chunks during snapshotting if needed. Specify the chunk size as a value and
|
||||
unit, for example: `1GB`, `10MB`, `5KB`, `500B`. Defaults to `null` (unlimited chunk size).
|
||||
`max_restore_bytes_per_sec`:: Throttles per node restore rate. Defaults to unlimited. Note that restores are also throttled through <<recovery,recovery settings>>.
|
||||
`max_snapshot_bytes_per_sec`:: Throttles per node snapshot rate. Defaults to `40mb` per second.
|
||||
`readonly`:: Makes repository read-only. Defaults to `false`.
|
||||
include::{es-repo-dir}/tab-widgets/register-fs-repo-widget.asciidoc[]
|
||||
|
||||
[discrete]
|
||||
[[snapshots-read-only-repository]]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,39 @@
|
|||
++++
|
||||
<div class="tabs" data-tab-group="os">
|
||||
<div role="tablist" aria-label="data-log-path">
|
||||
<button role="tab"
|
||||
aria-selected="true"
|
||||
aria-controls="unix-tab-data-log-path"
|
||||
id="unix-data-log-path">
|
||||
Unix-like systems
|
||||
</button>
|
||||
<button role="tab"
|
||||
aria-selected="false"
|
||||
aria-controls="win-tab-data-log-path"
|
||||
id="win-data-log-path">
|
||||
Windows
|
||||
</button>
|
||||
</div>
|
||||
<div tabindex="0"
|
||||
role="tabpanel"
|
||||
id="unix-tab-data-log-path"
|
||||
aria-labelledby="unix-data-log-path">
|
||||
++++
|
||||
|
||||
include::customize-data-log-path.asciidoc[tag=unix]
|
||||
|
||||
++++
|
||||
</div>
|
||||
<div tabindex="0"
|
||||
role="tabpanel"
|
||||
id="win-tab-data-log-path"
|
||||
aria-labelledby="win-data-log-path"
|
||||
hidden="">
|
||||
++++
|
||||
|
||||
include::customize-data-log-path.asciidoc[tag=win]
|
||||
|
||||
++++
|
||||
</div>
|
||||
</div>
|
||||
++++
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
// tag::unix[]
|
||||
Linux and macOS installations support Unix-style paths:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
path:
|
||||
data: /var/data/elasticsearch
|
||||
logs: /var/log/elasticsearch
|
||||
----
|
||||
// end::unix[]
|
||||
|
||||
|
||||
// tag::win[]
|
||||
Windows installations support DOS paths with escaped backslashes:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
path:
|
||||
data: "C:\\Elastic\\Elasticsearch\\data"
|
||||
logs: "C:\\Elastic\\Elasticsearch\\logs"
|
||||
----
|
||||
// end::win[]
|
||||
|
|
@ -25,7 +25,7 @@
|
|||
aria-controls="mac-tab-logs"
|
||||
id="mac-logs"
|
||||
tabindex="-1">
|
||||
MacOS
|
||||
macOS
|
||||
</button>
|
||||
<button role="tab"
|
||||
aria-selected="false"
|
||||
|
|
|
|||
|
|
@ -12,23 +12,35 @@ For <<rpm,RPM installations>>, {es} writes logs to `/var/log/elasticsearch`.
|
|||
// end::rpm[]
|
||||
|
||||
// tag::mac[]
|
||||
For <<targz,MacOS `.tar.gz`>> installations, {es} writes logs to
|
||||
For <<targz,macOS `.tar.gz`>> installations, {es} writes logs to
|
||||
`$ES_HOME/logs`.
|
||||
|
||||
Files in `$ES_HOME` risk deletion during an upgrade. In production, we strongly
|
||||
recommend you set `path.logs` to a location outside of `$ES_HOME`.
|
||||
See <<path-settings>>.
|
||||
// end::mac[]
|
||||
|
||||
// tag::brew[]
|
||||
For <<brew,MacOS Homebrew>> installations, {es} writes logs to
|
||||
For <<brew,macOS Homebrew>> installations, {es} writes logs to
|
||||
`/usr/local/var/log/elasticsearch`.
|
||||
// end::brew[]
|
||||
|
||||
// tag::linux[]
|
||||
For <<targz,Linux `.tar.gz`>> installations, {es} writes logs to
|
||||
`$ES_HOME/logs`.
|
||||
|
||||
Files in `$ES_HOME` risk deletion during an upgrade. In production, we strongly
|
||||
recommend you set `path.logs` to a location outside of `$ES_HOME`.
|
||||
See <<path-settings>>.
|
||||
// end::linux[]
|
||||
|
||||
// tag::win-zip[]
|
||||
For <<zip-windows,Windows `.zip`>> installations, {es} writes logs to
|
||||
`%ES_HOME%\logs`.
|
||||
|
||||
Files in `%ES_HOME%` risk deletion during an upgrade. In production, we strongly
|
||||
recommend you set `path.logs` to a location outside of `%ES_HOME%``.
|
||||
See <<path-settings>>.
|
||||
// end::win-zip[]
|
||||
|
||||
// tag::win-msi[]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,39 @@
|
|||
++++
|
||||
<div class="tabs" data-tab-group="os">
|
||||
<div role="tablist" aria-label="multi-data-path">
|
||||
<button role="tab"
|
||||
aria-selected="true"
|
||||
aria-controls="unix-tab-multi-data-path"
|
||||
id="unix-multi-data-path">
|
||||
Unix-like systems
|
||||
</button>
|
||||
<button role="tab"
|
||||
aria-selected="false"
|
||||
aria-controls="win-tab-multi-data-path"
|
||||
id="win-multi-data-path">
|
||||
Windows
|
||||
</button>
|
||||
</div>
|
||||
<div tabindex="0"
|
||||
role="tabpanel"
|
||||
id="unix-tab-multi-data-path"
|
||||
aria-labelledby="unix-multi-data-path">
|
||||
++++
|
||||
|
||||
include::multi-data-path.asciidoc[tag=unix]
|
||||
|
||||
++++
|
||||
</div>
|
||||
<div tabindex="0"
|
||||
role="tabpanel"
|
||||
id="win-tab-multi-data-path"
|
||||
aria-labelledby="win-multi-data-path"
|
||||
hidden="">
|
||||
++++
|
||||
|
||||
include::multi-data-path.asciidoc[tag=win]
|
||||
|
||||
++++
|
||||
</div>
|
||||
</div>
|
||||
++++
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
// tag::unix[]
|
||||
Linux and macOS installations support multiple Unix-style paths in `path.data`:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
path:
|
||||
data:
|
||||
- /mnt/elasticsearch_1
|
||||
- /mnt/elasticsearch_2
|
||||
- /mnt/elasticsearch_3
|
||||
----
|
||||
// end::unix[]
|
||||
|
||||
|
||||
// tag::win[]
|
||||
Windows installations support multiple DOS paths in `path.data`:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
path:
|
||||
data:
|
||||
- "C:\\Elastic\\Elasticsearch_1"
|
||||
- "E:\\Elastic\\Elasticsearch_1"
|
||||
- "F:\\Elastic\\Elasticsearch_3"
|
||||
----
|
||||
// end::win[]
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
++++
|
||||
<div class="tabs" data-tab-group="os">
|
||||
<div role="tablist" aria-label="fs-repo">
|
||||
<button role="tab"
|
||||
aria-selected="true"
|
||||
aria-controls="unix-tab-fs-repo"
|
||||
id="unix-fs-repo">
|
||||
Unix-like systems
|
||||
</button>
|
||||
<button role="tab"
|
||||
aria-selected="false"
|
||||
aria-controls="win-tab-fs-repo"
|
||||
id="win-fs-repo">
|
||||
Windows
|
||||
</button>
|
||||
</div>
|
||||
<div tabindex="0"
|
||||
role="tabpanel"
|
||||
id="unix-tab-fs-repo"
|
||||
aria-labelledby="unix-fs-repo">
|
||||
++++
|
||||
|
||||
include::register-fs-repo.asciidoc[tag=unix]
|
||||
|
||||
++++
|
||||
</div>
|
||||
<div tabindex="0"
|
||||
role="tabpanel"
|
||||
id="win-tab-fs-repo"
|
||||
aria-labelledby="win-fs-repo"
|
||||
hidden="">
|
||||
++++
|
||||
|
||||
include::register-fs-repo.asciidoc[tag=win]
|
||||
|
||||
++++
|
||||
</div>
|
||||
</div>
|
||||
++++
|
||||
|
|
@ -0,0 +1,103 @@
|
|||
// tag::unix[]
|
||||
Linux and macOS installations support Unix-style paths:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
path:
|
||||
repo:
|
||||
- /mount/backups
|
||||
- /mount/long_term_backups
|
||||
----
|
||||
|
||||
After restarting each node, use the <<put-snapshot-repo-api,put snapshot
|
||||
repository>> API to register the file system repository. Specify the file
|
||||
system's path in `settings.location`:
|
||||
|
||||
[source,console]
|
||||
----
|
||||
PUT /_snapshot/my_fs_backup
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "/mount/backups/my_fs_backup_location",
|
||||
"compress": true
|
||||
}
|
||||
}
|
||||
----
|
||||
// TEST[skip:no access to path]
|
||||
|
||||
If you specify a relative path in `settings.location`, {es} resolves the path
|
||||
using the first value in the `path.repo` setting.
|
||||
|
||||
[source,console]
|
||||
----
|
||||
PUT /_snapshot/my_fs_backup
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "my_fs_backup_location", <1>
|
||||
"compress": true
|
||||
}
|
||||
}
|
||||
----
|
||||
// TEST[skip:no access to path]
|
||||
|
||||
<1> The first value in the `path.repo` setting is `/mount/backups`. This
|
||||
relative path, `my_fs_backup_location`, resolves to
|
||||
`/mount/backups/my_fs_backup_location`.
|
||||
// end::unix[]
|
||||
|
||||
|
||||
// tag::win[]
|
||||
Windows installations support both DOS and Microsoft UNC paths. Escaped any
|
||||
backslashes in the paths. For UNC paths, provide the server and share name as a
|
||||
prefix.
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
path:
|
||||
repo:
|
||||
- "E:\\Mount\\Backups" <1>
|
||||
- "\\\\MY_SERVER\\Mount\\Long_term_backups" <2>
|
||||
----
|
||||
|
||||
<1> DOS path
|
||||
<2> UNC path
|
||||
|
||||
After restarting each node, use the <<put-snapshot-repo-api,put snapshot
|
||||
repository>> API to register the file system repository. Specify the file
|
||||
system's path in `settings.location`:
|
||||
|
||||
[source,console]
|
||||
----
|
||||
PUT /_snapshot/my_fs_backup
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "E:\\Mount\\Backups\\My_fs_backup_location",
|
||||
"compress": true
|
||||
}
|
||||
}
|
||||
----
|
||||
// TEST[skip:no access to path]
|
||||
|
||||
If you specify a relative path in `settings.location`, {es} resolves the path
|
||||
using the first value in the `path.repo` setting.
|
||||
|
||||
[source,console]
|
||||
----
|
||||
PUT /_snapshot/my_fs_backup
|
||||
{
|
||||
"type": "fs",
|
||||
"settings": {
|
||||
"location": "My_fs_backup_location", <1>
|
||||
"compress": true
|
||||
}
|
||||
}
|
||||
----
|
||||
// TEST[skip:no access to path]
|
||||
|
||||
<1> The first value in the `path.repo` setting is `E:\Mount\Backups`. This
|
||||
relative path, `My_fs_backup_location`, resolves to
|
||||
`E:\Mount\Backups\My_fs_backup_location`.
|
||||
// end::win[]
|
||||
Loading…
Reference in New Issue