[DOCS] Cleans up monitoring configuration steps (#35111)
This commit is contained in:
Lisa Cawley
GitHub
parent
7e144b318a
commit
c88b2d7e5a
|
|
@ -15,9 +15,15 @@ To learn about monitoring in general, see
|
|||
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. Enable the collection of monitoring data. Set
|
||||
`xpack.monitoring.collection.enabled` to `true` on the production cluster. +
|
||||
`xpack.monitoring.collection.enabled` to `true` on each node in the production
|
||||
cluster. By default, it is is disabled (`false`).
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For example, you can use the following APIs to review and change this setting:
|
||||
|
||||
[source,js]
|
||||
|
|
@ -31,16 +37,21 @@ PUT _cluster/settings
|
|||
}
|
||||
}
|
||||
----------------------------------
|
||||
// CONSOLE
|
||||
// CONSOLE
|
||||
|
||||
For more information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
. Disable the default collection of {es} monitoring metrics. Set
|
||||
`xpack.monitoring.elasticsearch.collection.enabled` to `false` on the production
|
||||
cluster. +
|
||||
`xpack.monitoring.elasticsearch.collection.enabled` to `false` on each node in
|
||||
the production cluster.
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For example, you can use the following API to change this setting:
|
||||
|
||||
[source,js]
|
||||
|
|
@ -96,30 +107,26 @@ You must specify the following settings in the `modules.d/elasticsearch.yml` fil
|
|||
- shard
|
||||
period: 10s
|
||||
hosts: ["http://localhost:9200"] <1>
|
||||
xpack.enabled: true
|
||||
xpack.enabled: true <2>
|
||||
----------------------------------
|
||||
<1> This setting identifies the host and port number that are used to access {es}.
|
||||
<1> This setting identifies the host and port number that are used to access {es}.
|
||||
<2> This setting ensures that {kib} can read this monitoring data successfully.
|
||||
That is to say, it's stored in the same location and format as monitoring data
|
||||
that is sent by <<es-monitoring-exporters,exporters>>.
|
||||
--
|
||||
|
||||
.. If {security} is enabled, you must also provide a user ID and password so that
|
||||
{metricbeat} can collect metrics successfully.
|
||||
.. If Elastic {security-features} are enabled, you must also provide a user ID
|
||||
and password so that {metricbeat} can collect metrics successfully.
|
||||
|
||||
... Create or identify a user that you want to use to collect the metrics.
|
||||
+
|
||||
--
|
||||
TIP: There is a `remote_monitoring_user` built-in user that grants the privileges
|
||||
necessary for {metricbeat} to monitor {stack} products. See
|
||||
{stack-ov}/built-in-users.html[Built-in users].
|
||||
|
||||
Alternatively, you can choose a different user and give them the
|
||||
`remote_monitoring_collector` {stack-ov}/built-in-roles.html[built-in role].
|
||||
--
|
||||
... Create a user on the production cluster that has the
|
||||
{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
|
||||
Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
||||
|
||||
... Add the `username` and `password` settings to the {es} module configuration
|
||||
file.
|
||||
+
|
||||
--
|
||||
For example, add the following settings in the `modules.d/kibana.yml` file:
|
||||
For example, add the following settings in the `modules.d/elasticsearch.yml` file:
|
||||
|
||||
[source,yaml]
|
||||
----------------------------------
|
||||
|
|
@ -158,19 +165,14 @@ For more information about these configuration options, see
|
|||
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
|
||||
--
|
||||
|
||||
.. If {security} is enabled on the monitoring cluster, you must provide a valid
|
||||
user ID and password so that {metricbeat} can send metrics successfully.
|
||||
.. If {es} {security-features} are enabled on the monitoring cluster, you
|
||||
must provide a valid user ID and password so that {metricbeat} can send metrics
|
||||
successfully.
|
||||
|
||||
... Create or identify a user that you want to use to send the metrics.
|
||||
+
|
||||
--
|
||||
TIP: There is a `remote_monitoring_user` built-in user that grants the privileges
|
||||
necessary for {metricbeat} to monitor {stack} products. See
|
||||
{stack-ov}/built-in-users.html[Built-in users].
|
||||
|
||||
Alternatively, you can choose a different user and give them the
|
||||
`remote_monitoring_agent` {stack-ov}/built-in-roles.html[built-in role].
|
||||
--
|
||||
... Create a user on the monitoring cluster that has the
|
||||
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
|
||||
Alternatively, use the
|
||||
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
||||
|
||||
... Add the `username` and `password` settings to the {es} output information in
|
||||
the {metricbeat} configuration file (`metricbeat.yml`):
|
||||
|
|
|
|||
|
|
@ -6,8 +6,8 @@
|
|||
<titleabbrev>Configuring monitoring</titleabbrev>
|
||||
++++
|
||||
|
||||
If you enable the collection of monitoring data in your cluster, you can
|
||||
optionally collect metrics about {es}. By default, {monitoring} is enabled but
|
||||
If you enable the Elastic {monitor-features} in your cluster, you can
|
||||
optionally collect metrics about {es}. By default, monitoring is enabled but
|
||||
data collection is disabled.
|
||||
|
||||
The following method involves sending the metrics to the monitoring cluster by
|
||||
|
|
@ -20,15 +20,51 @@ monitoring indices. You can also adjust how monitoring data is displayed.
|
|||
To learn about monitoring in general, see
|
||||
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. To collect monitoring data about your {es} cluster:
|
||||
. Configure your cluster to collect monitoring data:
|
||||
|
||||
.. Verify that the `xpack.monitoring.enabled`,
|
||||
`xpack.monitoring.collection.enabled`, and
|
||||
`xpack.monitoring.elasticsearch.collection.enabled` settings are `true` on each
|
||||
node in the cluster. By default `xpack.monitoring.collection.enabled` is disabled
|
||||
(`false`), and that overrides `xpack.monitoring.elasticsearch.collection.enabled`,
|
||||
which defaults to being enabled (`true`). Both settings can be set dynamically
|
||||
at runtime. For more information, see <<monitoring-settings>>.
|
||||
.. Verify that the `xpack.monitoring.enabled` setting is `true`, which is its
|
||||
default value, on each node in the cluster. For more information, see
|
||||
<<monitoring-settings>>.
|
||||
|
||||
.. Verify that the `xpack.monitoring.elasticsearch.collection.enabled` setting
|
||||
is `true`, which is its default value, on each node in the cluster.
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For more information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
.. Set the `xpack.monitoring.collection.enabled` setting to `true` on each
|
||||
node in the cluster. By default, it is is disabled (`false`).
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For example, use the following APIs to review and change this setting:
|
||||
|
||||
[source,js]
|
||||
----------------------------------
|
||||
GET _cluster/settings
|
||||
|
||||
PUT _cluster/settings
|
||||
{
|
||||
"persistent": {
|
||||
"xpack.monitoring.collection.enabled": true
|
||||
}
|
||||
}
|
||||
----------------------------------
|
||||
// CONSOLE
|
||||
|
||||
For more
|
||||
information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
.. Optional: Specify which indices you want to monitor.
|
||||
+
|
||||
|
|
@ -53,62 +89,71 @@ patterns. For example, to include all indices that start with `test` except
|
|||
the `xpack.monitoring.collection.interval` setting 10 seconds. See
|
||||
<<monitoring-settings>>.
|
||||
|
||||
. Optional: Configure your cluster to route monitoring data from sources such
|
||||
as {kib}, Beats, and Logstash to a monitoring cluster:
|
||||
|
||||
.. Verify that `xpack.monitoring.collection.enabled` settings are `true` on each
|
||||
node in the cluster.
|
||||
|
||||
.. {stack-ov}/xpack-monitoring.html[Configure {monitoring} across the Elastic Stack].
|
||||
|
||||
. Identify where to store monitoring data.
|
||||
+
|
||||
--
|
||||
By default, {monitoring} uses a `local` exporter that indexes monitoring data
|
||||
on the same cluster. See <<es-monitoring-default-exporter>> and <<local-exporter>>.
|
||||
By default, the data is stored on the same cluster by using a
|
||||
<<local-exporter,`local` exporter>>.
|
||||
|
||||
Alternatively, you can use an `http` exporter to send data to a separate
|
||||
monitoring cluster. See <<http-exporter>>.
|
||||
Alternatively, you can use an <<http-exporter,`http` exporter>> to send data to
|
||||
a separate _monitoring cluster_.
|
||||
|
||||
For more information about typical monitoring architectures,
|
||||
see {stack-ov}/how-monitoring-works.html[How Monitoring Works].
|
||||
--
|
||||
|
||||
. If {security} is enabled and you are using an `http` exporter to send data to
|
||||
a dedicated monitoring cluster:
|
||||
. If you choose to use an `http` exporter:
|
||||
|
||||
.. Create a user on the monitoring cluster that has the
|
||||
{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[`remote_monitoring_agent` built-in role].
|
||||
For example, the following request creates a `remote_monitor` user that has the
|
||||
`remote_monitoring_agent` role:
|
||||
.. On the cluster that you want to monitor (often called the _production cluster_),
|
||||
configure each node to send metrics to your monitoring cluster. Configure an
|
||||
HTTP exporter in the `xpack.monitoring.exporters` settings in the
|
||||
`elasticsearch.yml` file. For example:
|
||||
+
|
||||
--
|
||||
[source, sh]
|
||||
---------------------------------------------------------------
|
||||
POST /_xpack/security/user/remote_monitor
|
||||
{
|
||||
"password" : "changeme",
|
||||
"roles" : [ "remote_monitoring_agent"],
|
||||
"full_name" : "Internal Agent For Remote Monitoring"
|
||||
}
|
||||
---------------------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[skip:needs-gold+-license]
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
.. On each node in the cluster that is being monitored, configure the `http`
|
||||
exporter to use the appropriate credentials when data is shipped to the
|
||||
monitoring cluster.
|
||||
.. If the Elastic {security-features} are enabled on the monitoring cluster, you
|
||||
must provide appropriate credentials when data is shipped to the monitoring cluster:
|
||||
|
||||
... Create a user on the monitoring cluster that has the
|
||||
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
|
||||
Alternatively, use the
|
||||
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
||||
|
||||
... Add the user ID and password settings to the HTTP exporter settings in the
|
||||
`elasticsearch.yml` file on each node. +
|
||||
+
|
||||
--
|
||||
If SSL/TLS is enabled on the monitoring cluster, you must use the HTTPS protocol
|
||||
in the `host` setting. You must also include the CA certificate in each node's
|
||||
trusted certificates in order to verify the identities of the nodes in the
|
||||
monitoring cluster.
|
||||
For example:
|
||||
|
||||
The following example specifies the location of the PEM encoded certificate with
|
||||
the `certificate_authorities` setting:
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
|
||||
auth.username: remote_monitoring_user
|
||||
auth.password: YOUR_PASSWORD
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
.. If you configured the monitoring cluster to use
|
||||
<<configuring-tls,encrypted communications>>, you must use the HTTPS protocol in
|
||||
the `host` setting. You must also specify the trusted CA certificates that will
|
||||
be used to verify the identity of the nodes in the monitoring cluster.
|
||||
|
||||
*** To add a CA certificate to an {es} node's trusted certificates, you can
|
||||
specify the location of the PEM encoded certificate with the
|
||||
`certificate_authorities` setting. For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
|
|
@ -116,18 +161,17 @@ xpack.monitoring.exporters:
|
|||
type: http
|
||||
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
|
||||
auth:
|
||||
username: remote_monitor <1>
|
||||
password: changeme
|
||||
username: remote_monitoring_user
|
||||
password: YOUR_PASSWORD
|
||||
ssl:
|
||||
certificate_authorities: [ "/path/to/ca.crt" ]
|
||||
id2:
|
||||
type: local
|
||||
--------------------------------------------------
|
||||
<1> The `username` and `password` parameters provide the user credentials.
|
||||
|
||||
Alternatively, you can configure trusted certificates using a truststore
|
||||
(a Java Keystore file that contains the certificates):
|
||||
--
|
||||
|
||||
*** Alternatively, you can configure trusted certificates using a truststore
|
||||
(a Java Keystore file that contains the certificates). For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
|
|
@ -135,25 +179,28 @@ xpack.monitoring.exporters:
|
|||
type: http
|
||||
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
|
||||
auth:
|
||||
username: remote_monitor
|
||||
password: changeme
|
||||
username: remote_monitoring_user
|
||||
password: YOUR_PASSWORD
|
||||
ssl:
|
||||
truststore.path: /path/to/file
|
||||
truststore.password: password
|
||||
id2:
|
||||
type: local
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
. If {security} is enabled and you want to visualize monitoring data in {kib},
|
||||
you must create users that have access to the {kib} indices and permission to
|
||||
read from the monitoring indices.
|
||||
. Configure your cluster to route monitoring data from sources such as {kib},
|
||||
Beats, and {ls} to the monitoring cluster. The
|
||||
`xpack.monitoring.collection.enabled` setting must be `true` on each node in the
|
||||
cluster. For information about configuring each product to collect and send
|
||||
monitoring data, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. If you updated settings in the `elasticsearch.yml` files on your production
|
||||
cluster, restart {es}. See <<stopping-elasticsearch>> and <<starting-elasticsearch>>.
|
||||
+
|
||||
--
|
||||
You set up {monitoring} UI users on the cluster where the monitoring data is
|
||||
stored, that is to say the monitoring cluster. To grant all of the necessary permissions, assign users the
|
||||
`monitoring_user` and `kibana_user` roles. For more information, see
|
||||
{stack-ov}/mapping-roles.html[Mapping users and groups to roles].
|
||||
TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard
|
||||
allocation] before you restart your nodes to avoid unnecessary shard
|
||||
reallocation during the install process.
|
||||
|
||||
--
|
||||
|
||||
. Optional:
|
||||
|
|
|
|||
|
|
@ -5,38 +5,46 @@
|
|||
|
||||
[partintro]
|
||||
--
|
||||
{monitoring} enables you to easily monitor the health of your {es} cluster. The
|
||||
monitoring metrics are collected from each node and stored in {es} indices.
|
||||
The Elastic {monitor-features} enable you to easily monitor the health of
|
||||
your {es} cluster. The monitoring metrics are collected from each node and
|
||||
stored in {es} indices.
|
||||
|
||||
TIP: In production environments, it is recommended to store the monitoring data
|
||||
in a separate _monitoring cluster_. See
|
||||
{stack-ov}/monitoring-production.html[Monitoring in a production environment].
|
||||
|
||||
Each {es} node is considered unique based on its persistent UUID, which is
|
||||
written on first start to its <<path-settings,`path.data`>> directory, which
|
||||
defaults to `./data`.
|
||||
|
||||
All settings associated with {monitoring} in {es} must be set in either the
|
||||
All settings associated with monitoring in {es} must be set in either the
|
||||
`elasticsearch.yml` file for each node or, where possible, in the dynamic
|
||||
cluster settings. For more information, see <<configuring-monitoring>>.
|
||||
|
||||
[[es-monitoring-overview]]
|
||||
{es} is also at the core of {monitoring} across the Elastic Stack. In all cases,
|
||||
{monitoring} documents are just ordinary JSON documents built by monitoring each
|
||||
Elastic Stack component at some collection interval, then indexing those
|
||||
documents into the monitoring cluster. Each component in the stack is
|
||||
responsible for monitoring itself and then forwarding those documents to {es}
|
||||
for both routing and indexing (storage).
|
||||
{es} is also at the core of monitoring across the {stack}. In all cases,
|
||||
monitoring documents are just ordinary JSON documents built by monitoring each
|
||||
{stack} component at some collection interval, then indexing those
|
||||
documents into the monitoring cluster.
|
||||
|
||||
The routing and indexing processes in {es} are handled by what are called
|
||||
<<es-monitoring-collectors,collectors>> and
|
||||
<<es-monitoring-exporters,exporters>>. In the past, collectors and exporters
|
||||
were considered to be part of a monitoring "agent", but that term is generally
|
||||
not used anymore.
|
||||
Each component in the stack is responsible for monitoring itself and then
|
||||
forwarding those documents to the {es} production cluster for both routing and
|
||||
indexing (storage). The routing and indexing processes in {es} are handled by
|
||||
what are called <<es-monitoring-collectors,collectors>> and
|
||||
<<es-monitoring-exporters,exporters>>.
|
||||
|
||||
beta[] Alternatively, in 6.4 and later, you can use {metricbeat} to collect
|
||||
monitoring data about {kib} and ship it directly to the monitoring cluster,
|
||||
rather than routing it through the production cluster. In 6.5 and later, you
|
||||
can also use {metricbeat} to collect and ship data about {es}.
|
||||
|
||||
You can view monitoring data from {kib} where it’s easy to spot issues at a
|
||||
glance or delve into the system behavior over time to diagnose operational
|
||||
issues. In addition to the built-in status warnings, you can also set up custom
|
||||
alerts based on the data in the monitoring indices.
|
||||
|
||||
For an introduction to monitoring your Elastic stack, including Beats, Logstash,
|
||||
and {kib}, see {xpack-ref}/xpack-monitoring.html[Monitoring the Elastic Stack].
|
||||
For an introduction to monitoring your {stack}, including Beats, {ls}, and {kib},
|
||||
see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
--
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue