From 88634bc022aea0505fa453ec3821f74c9a47eb8d Mon Sep 17 00:00:00 2001 From: James Rodewig <40268737+jrodewig@users.noreply.github.com> Date: Mon, 9 Nov 2020 11:15:00 -0500 Subject: [PATCH] [DOCS] Improve docs for Windows DOS/UNC paths in `path.*` settings (#64668) (#64805) --- .../important-settings/path-settings.asciidoc | 48 ++++---- .../register-repository.asciidoc | 68 ++---------- .../customize-data-log-path-widget.asciidoc | 39 +++++++ .../customize-data-log-path.asciidoc | 22 ++++ .../tab-widgets/logging-widget.asciidoc | 2 +- docs/reference/tab-widgets/logging.asciidoc | 18 ++- .../multi-data-path-widget.asciidoc | 39 +++++++ .../tab-widgets/multi-data-path.asciidoc | 26 +++++ .../register-fs-repo-widget.asciidoc | 39 +++++++ .../tab-widgets/register-fs-repo.asciidoc | 103 ++++++++++++++++++ 10 files changed, 318 insertions(+), 86 deletions(-) create mode 100644 docs/reference/tab-widgets/customize-data-log-path-widget.asciidoc create mode 100644 docs/reference/tab-widgets/customize-data-log-path.asciidoc create mode 100644 docs/reference/tab-widgets/multi-data-path-widget.asciidoc create mode 100644 docs/reference/tab-widgets/multi-data-path.asciidoc create mode 100644 docs/reference/tab-widgets/register-fs-repo-widget.asciidoc create mode 100644 docs/reference/tab-widgets/register-fs-repo.asciidoc diff --git a/docs/reference/setup/important-settings/path-settings.asciidoc b/docs/reference/setup/important-settings/path-settings.asciidoc index 2867c464e48..49e6edb356b 100644 --- a/docs/reference/setup/important-settings/path-settings.asciidoc +++ b/docs/reference/setup/important-settings/path-settings.asciidoc @@ -2,32 +2,32 @@ [discrete] === Path settings -If you are using the `.zip` or `.tar.gz` archives, the `data` and `logs` -directories are sub-folders of `$ES_HOME`. If these important folders are left -in their default locations, there is a high risk of them being deleted while -upgrading {es} to a new version. +For <>, <>, and +<> installations, {es} writes data and logs to the +respective `data` and `logs` subdirectories of `$ES_HOME` by default. +However, files in `$ES_HOME` risk deletion during an upgrade. -In production use, you will almost certainly want to change the locations of the -`path.data` and `path.logs` folders: +In production, we strongly recommend you set the `path.data` and `path.logs` in +`elasticsearch.yml` to locations outside of `$ES_HOME`. -[source,yaml] --------------------------------------------------- -path: - logs: /var/log/elasticsearch - data: /var/data/elasticsearch --------------------------------------------------- +TIP: <>, <>, <>, <>, +and <> installations write data and log to locations +outside of `$ES_HOME` by default. -The RPM and Debian distributions already use custom paths for `data` and `logs`. +Supported `path.data` and `path.logs` values vary by platform: -The `path.data` settings can be set to multiple paths, in which case all paths -will be used to store data. However, the files belonging to a single shard will -all be stored on the same data path: +include::{es-repo-dir}/tab-widgets/code.asciidoc[] -[source,yaml] --------------------------------------------------- -path: - data: - - /mnt/elasticsearch_1 - - /mnt/elasticsearch_2 - - /mnt/elasticsearch_3 --------------------------------------------------- +include::{es-repo-dir}/tab-widgets/customize-data-log-path-widget.asciidoc[] + +If needed, you can specify multiple paths in `path.data`. {es} stores the node's +data across all provided paths but keeps each shard's data on the same path. + +WARNING: {es} does not balance shards across a node's data paths. High disk +usage in a single path can trigger a <> for the entire node. If triggered, {es} will not add shards to +the node, even if the node’s other paths have available disk space. If you need +additional disk space, we recommend you add a new node rather than additional +data paths. + +include::{es-repo-dir}/tab-widgets/multi-data-path-widget.asciidoc[] \ No newline at end of file diff --git a/docs/reference/snapshot-restore/register-repository.asciidoc b/docs/reference/snapshot-restore/register-repository.asciidoc index 58df21ba346..42a0f10c8bb 100644 --- a/docs/reference/snapshot-restore/register-repository.asciidoc +++ b/docs/reference/snapshot-restore/register-repository.asciidoc @@ -100,68 +100,20 @@ are left untouched and in place. [[snapshots-filesystem-repository]] === Shared file system repository -The shared file system repository (`"type": "fs"`) uses the shared file system to store snapshots. In order to register -the shared file system repository it is necessary to mount the same shared filesystem to the same location on all -master and data nodes. This location (or one of its parent directories) must be registered in the `path.repo` -setting on all master and data nodes. +Use a shared file system repository (`"type": "fs"`) to store snapshots on a +shared file system. -Assuming that the shared filesystem is mounted to `/mount/backups/my_fs_backup_location`, the following setting should -be added to `elasticsearch.yml` file: +To register a shared file system repository, first mount the file system to the +same location on all master and data nodes. Then add the file system's +path or parent directory to the `path.repo` setting in `elasticsearch.yml` for +each master and data node. For running clusters, this requires a +<> of each node. -[source,yaml] --------------- -path.repo: ["/mount/backups", "/mount/longterm_backups"] --------------- +Supported `path.repo` values vary by platform: -The `path.repo` setting supports Microsoft Windows UNC paths as long as at least server name and share are specified as -a prefix and back slashes are properly escaped: +include::{es-repo-dir}/tab-widgets/code.asciidoc[] -[source,yaml] --------------- -path.repo: ["\\\\MY_SERVER\\Snapshots"] --------------- - -After all nodes are restarted, the following command can be used to register the shared file system repository with -the name `my_fs_backup`: - -[source,console] ------------------------------------ -PUT /_snapshot/my_fs_backup -{ - "type": "fs", - "settings": { - "location": "/mount/backups/my_fs_backup_location", - "compress": true - } -} ------------------------------------ -// TEST[skip:no access to absolute path] - -If the repository location is specified as a relative path this path will be resolved against the first path specified -in `path.repo`: - -[source,console] ------------------------------------ -PUT /_snapshot/my_fs_backup -{ - "type": "fs", - "settings": { - "location": "my_fs_backup_location", - "compress": true - } -} ------------------------------------ -// TEST[continued] - -The following settings are supported: - -`location`:: Location of the snapshots. Mandatory. -`compress`:: Turns on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. Defaults to `true`. -`chunk_size`:: Big files can be broken down into chunks during snapshotting if needed. Specify the chunk size as a value and -unit, for example: `1GB`, `10MB`, `5KB`, `500B`. Defaults to `null` (unlimited chunk size). -`max_restore_bytes_per_sec`:: Throttles per node restore rate. Defaults to unlimited. Note that restores are also throttled through <>. -`max_snapshot_bytes_per_sec`:: Throttles per node snapshot rate. Defaults to `40mb` per second. -`readonly`:: Makes repository read-only. Defaults to `false`. +include::{es-repo-dir}/tab-widgets/register-fs-repo-widget.asciidoc[] [discrete] [[snapshots-read-only-repository]] diff --git a/docs/reference/tab-widgets/customize-data-log-path-widget.asciidoc b/docs/reference/tab-widgets/customize-data-log-path-widget.asciidoc new file mode 100644 index 00000000000..a87f4c6d557 --- /dev/null +++ b/docs/reference/tab-widgets/customize-data-log-path-widget.asciidoc @@ -0,0 +1,39 @@ +++++ +
+
+ + +
+
+++++ + +include::customize-data-log-path.asciidoc[tag=unix] + +++++ +
+ +
+++++ \ No newline at end of file diff --git a/docs/reference/tab-widgets/customize-data-log-path.asciidoc b/docs/reference/tab-widgets/customize-data-log-path.asciidoc new file mode 100644 index 00000000000..45cee2babeb --- /dev/null +++ b/docs/reference/tab-widgets/customize-data-log-path.asciidoc @@ -0,0 +1,22 @@ +// tag::unix[] +Linux and macOS installations support Unix-style paths: + +[source,yaml] +---- +path: + data: /var/data/elasticsearch + logs: /var/log/elasticsearch +---- +// end::unix[] + + +// tag::win[] +Windows installations support DOS paths with escaped backslashes: + +[source,yaml] +---- +path: + data: "C:\\Elastic\\Elasticsearch\\data" + logs: "C:\\Elastic\\Elasticsearch\\logs" +---- +// end::win[] \ No newline at end of file diff --git a/docs/reference/tab-widgets/logging-widget.asciidoc b/docs/reference/tab-widgets/logging-widget.asciidoc index 8b20c525ca1..b88633bdb48 100644 --- a/docs/reference/tab-widgets/logging-widget.asciidoc +++ b/docs/reference/tab-widgets/logging-widget.asciidoc @@ -25,7 +25,7 @@ aria-controls="mac-tab-logs" id="mac-logs" tabindex="-1"> - MacOS + macOS + + +
+++++ + +include::multi-data-path.asciidoc[tag=unix] + +++++ +
+ + +++++ \ No newline at end of file diff --git a/docs/reference/tab-widgets/multi-data-path.asciidoc b/docs/reference/tab-widgets/multi-data-path.asciidoc new file mode 100644 index 00000000000..0a63c7791f6 --- /dev/null +++ b/docs/reference/tab-widgets/multi-data-path.asciidoc @@ -0,0 +1,26 @@ +// tag::unix[] +Linux and macOS installations support multiple Unix-style paths in `path.data`: + +[source,yaml] +---- +path: + data: + - /mnt/elasticsearch_1 + - /mnt/elasticsearch_2 + - /mnt/elasticsearch_3 +---- +// end::unix[] + + +// tag::win[] +Windows installations support multiple DOS paths in `path.data`: + +[source,yaml] +---- +path: + data: + - "C:\\Elastic\\Elasticsearch_1" + - "E:\\Elastic\\Elasticsearch_1" + - "F:\\Elastic\\Elasticsearch_3" +---- +// end::win[] diff --git a/docs/reference/tab-widgets/register-fs-repo-widget.asciidoc b/docs/reference/tab-widgets/register-fs-repo-widget.asciidoc new file mode 100644 index 00000000000..d0ac7041adf --- /dev/null +++ b/docs/reference/tab-widgets/register-fs-repo-widget.asciidoc @@ -0,0 +1,39 @@ +++++ +
+
+ + +
+
+++++ + +include::register-fs-repo.asciidoc[tag=unix] + +++++ +
+ +
+++++ \ No newline at end of file diff --git a/docs/reference/tab-widgets/register-fs-repo.asciidoc b/docs/reference/tab-widgets/register-fs-repo.asciidoc new file mode 100644 index 00000000000..bbfeabcbf57 --- /dev/null +++ b/docs/reference/tab-widgets/register-fs-repo.asciidoc @@ -0,0 +1,103 @@ +// tag::unix[] +Linux and macOS installations support Unix-style paths: + +[source,yaml] +---- +path: + repo: + - /mount/backups + - /mount/long_term_backups +---- + +After restarting each node, use the <> API to register the file system repository. Specify the file +system's path in `settings.location`: + +[source,console] +---- +PUT /_snapshot/my_fs_backup +{ + "type": "fs", + "settings": { + "location": "/mount/backups/my_fs_backup_location", + "compress": true + } +} +---- +// TEST[skip:no access to path] + +If you specify a relative path in `settings.location`, {es} resolves the path +using the first value in the `path.repo` setting. + +[source,console] +---- +PUT /_snapshot/my_fs_backup +{ + "type": "fs", + "settings": { + "location": "my_fs_backup_location", <1> + "compress": true + } +} +---- +// TEST[skip:no access to path] + +<1> The first value in the `path.repo` setting is `/mount/backups`. This +relative path, `my_fs_backup_location`, resolves to +`/mount/backups/my_fs_backup_location`. +// end::unix[] + + +// tag::win[] +Windows installations support both DOS and Microsoft UNC paths. Escaped any +backslashes in the paths. For UNC paths, provide the server and share name as a +prefix. + +[source,yaml] +---- +path: + repo: + - "E:\\Mount\\Backups" <1> + - "\\\\MY_SERVER\\Mount\\Long_term_backups" <2> +---- + +<1> DOS path +<2> UNC path + +After restarting each node, use the <> API to register the file system repository. Specify the file +system's path in `settings.location`: + +[source,console] +---- +PUT /_snapshot/my_fs_backup +{ + "type": "fs", + "settings": { + "location": "E:\\Mount\\Backups\\My_fs_backup_location", + "compress": true + } +} +---- +// TEST[skip:no access to path] + +If you specify a relative path in `settings.location`, {es} resolves the path +using the first value in the `path.repo` setting. + +[source,console] +---- +PUT /_snapshot/my_fs_backup +{ + "type": "fs", + "settings": { + "location": "My_fs_backup_location", <1> + "compress": true + } +} +---- +// TEST[skip:no access to path] + +<1> The first value in the `path.repo` setting is `E:\Mount\Backups`. This +relative path, `My_fs_backup_location`, resolves to +`E:\Mount\Backups\My_fs_backup_location`. +// end::win[] \ No newline at end of file