[DOCS] Adds higher-level monitoring configuration page (#35926)
This commit is contained in:
Lisa Cawley
GitHub
parent
c26af3b0a2
commit
906421cdad
|
|
@ -0,0 +1,216 @@
|
|||
[role="xpack"]
|
||||
[testenv="gold"]
|
||||
[[collecting-monitoring-data]]
|
||||
=== Collecting monitoring data
|
||||
++++
|
||||
<titleabbrev>Collecting monitoring data</titleabbrev>
|
||||
++++
|
||||
|
||||
If you enable the Elastic {monitor-features} in your cluster, you can
|
||||
optionally collect metrics about {es}. By default, monitoring is enabled but
|
||||
data collection is disabled.
|
||||
|
||||
This method involves sending the metrics to the monitoring cluster by using
|
||||
exporters. For an alternative method, see <<configuring-metricbeat>>.
|
||||
|
||||
NOTE: If you want to collect monitoring data from sources such as Beats and {ls}
|
||||
and route it to a monitoring cluster, you must follow this method. You cannot
|
||||
use {metricbeat} to ship the monitoring data for those products yet.
|
||||
|
||||
Advanced monitoring settings enable you to control how frequently data is
|
||||
collected, configure timeouts, and set the retention period for locally-stored
|
||||
monitoring indices. You can also adjust how monitoring data is displayed.
|
||||
|
||||
To learn about monitoring in general, see
|
||||
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. Configure your cluster to collect monitoring data:
|
||||
|
||||
.. Verify that the `xpack.monitoring.enabled` setting is `true`, which is its
|
||||
default value, on each node in the cluster. For more information, see
|
||||
<<monitoring-settings>>.
|
||||
|
||||
.. Verify that the `xpack.monitoring.elasticsearch.collection.enabled` setting
|
||||
is `true`, which is its default value, on each node in the cluster.
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For more information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
.. Set the `xpack.monitoring.collection.enabled` setting to `true` on each
|
||||
node in the cluster. By default, it is is disabled (`false`).
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For example, use the following APIs to review and change this setting:
|
||||
|
||||
[source,js]
|
||||
----------------------------------
|
||||
GET _cluster/settings
|
||||
|
||||
PUT _cluster/settings
|
||||
{
|
||||
"persistent": {
|
||||
"xpack.monitoring.collection.enabled": true
|
||||
}
|
||||
}
|
||||
----------------------------------
|
||||
// CONSOLE
|
||||
|
||||
Alternatively, you can enable this setting in {kib}. In the side navigation,
|
||||
click *Monitoring*. If data collection is disabled, you are prompted to turn it
|
||||
on.
|
||||
|
||||
For more
|
||||
information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
.. Optional: Specify which indices you want to monitor.
|
||||
+
|
||||
--
|
||||
By default, the monitoring agent collects data from all {es} indices.
|
||||
To collect data from particular indices, configure the
|
||||
`xpack.monitoring.collection.indices` setting. You can specify multiple indices
|
||||
as a comma-separated list or use an index pattern to match multiple indices. For
|
||||
example:
|
||||
|
||||
[source,yaml]
|
||||
----------------------------------
|
||||
xpack.monitoring.collection.indices: logstash-*, index1, test2
|
||||
----------------------------------
|
||||
|
||||
You can prepend `+` or `-` to explicitly include or exclude index names or
|
||||
patterns. For example, to include all indices that start with `test` except
|
||||
`test3`, you could specify `+test*,-test3`.
|
||||
--
|
||||
|
||||
.. Optional: Specify how often to collect monitoring data. The default value for
|
||||
the `xpack.monitoring.collection.interval` setting 10 seconds. See
|
||||
<<monitoring-settings>>.
|
||||
|
||||
. Identify where to store monitoring data.
|
||||
+
|
||||
--
|
||||
By default, the data is stored on the same cluster by using a
|
||||
<<local-exporter,`local` exporter>>.
|
||||
|
||||
Alternatively, you can use an <<http-exporter,`http` exporter>> to send data to
|
||||
a separate _monitoring cluster_.
|
||||
|
||||
For more information about typical monitoring architectures,
|
||||
see {stack-ov}/how-monitoring-works.html[How Monitoring Works].
|
||||
--
|
||||
|
||||
. If you choose to use an `http` exporter:
|
||||
|
||||
.. On the cluster that you want to monitor (often called the _production cluster_),
|
||||
configure each node to send metrics to your monitoring cluster. Configure an
|
||||
HTTP exporter in the `xpack.monitoring.exporters` settings in the
|
||||
`elasticsearch.yml` file. For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
.. If the Elastic {security-features} are enabled on the monitoring cluster, you
|
||||
must provide appropriate credentials when data is shipped to the monitoring cluster:
|
||||
|
||||
... Create a user on the monitoring cluster that has the
|
||||
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
|
||||
Alternatively, use the
|
||||
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
||||
|
||||
... Add the user ID and password settings to the HTTP exporter settings in the
|
||||
`elasticsearch.yml` file on each node. +
|
||||
+
|
||||
--
|
||||
For example:
|
||||
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
|
||||
auth.username: remote_monitoring_user
|
||||
auth.password: YOUR_PASSWORD
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
.. If you configured the monitoring cluster to use
|
||||
<<configuring-tls,encrypted communications>>, you must use the HTTPS protocol in
|
||||
the `host` setting. You must also specify the trusted CA certificates that will
|
||||
be used to verify the identity of the nodes in the monitoring cluster.
|
||||
|
||||
*** To add a CA certificate to an {es} node's trusted certificates, you can
|
||||
specify the location of the PEM encoded certificate with the
|
||||
`certificate_authorities` setting. For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
|
||||
auth:
|
||||
username: remote_monitoring_user
|
||||
password: YOUR_PASSWORD
|
||||
ssl:
|
||||
certificate_authorities: [ "/path/to/ca.crt" ]
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
*** Alternatively, you can configure trusted certificates using a truststore
|
||||
(a Java Keystore file that contains the certificates). For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
|
||||
auth:
|
||||
username: remote_monitoring_user
|
||||
password: YOUR_PASSWORD
|
||||
ssl:
|
||||
truststore.path: /path/to/file
|
||||
truststore.password: password
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
. Configure your cluster to route monitoring data from sources such as {kib},
|
||||
Beats, and {ls} to the monitoring cluster. For information about configuring
|
||||
each product to collect and send monitoring data, see
|
||||
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. If you updated settings in the `elasticsearch.yml` files on your production
|
||||
cluster, restart {es}. See <<stopping-elasticsearch>> and <<starting-elasticsearch>>.
|
||||
+
|
||||
--
|
||||
TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard
|
||||
allocation] before you restart your nodes to avoid unnecessary shard
|
||||
reallocation during the install process.
|
||||
|
||||
--
|
||||
|
||||
. Optional:
|
||||
<<config-monitoring-indices,Configure the indices that store the monitoring data>>.
|
||||
|
||||
. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}].
|
||||
|
|
@ -1,13 +1,16 @@
|
|||
[role="xpack"]
|
||||
[testenv="gold"]
|
||||
[[configuring-metricbeat]]
|
||||
=== Monitoring {es} with {metricbeat}
|
||||
=== Collecting {es} monitoring data with {metricbeat}
|
||||
++++
|
||||
<titleabbrev>Collecting monitoring data with {metricbeat}</titleabbrev>
|
||||
++++
|
||||
|
||||
beta[]
|
||||
|
||||
In 6.5 and later, you can use {metricbeat} to collect data about {es}
|
||||
and ship it to the monitoring cluster, rather than routing it through exporters
|
||||
as described in <<configuring-monitoring>>.
|
||||
as described in <<collecting-monitoring-data>>.
|
||||
|
||||
image::monitoring/images/metricbeat.png[Example monitoring architecture]
|
||||
|
||||
|
|
|
|||
|
|
@ -6,208 +6,16 @@
|
|||
<titleabbrev>Configuring monitoring</titleabbrev>
|
||||
++++
|
||||
|
||||
If you enable the Elastic {monitor-features} in your cluster, you can
|
||||
optionally collect metrics about {es}. By default, monitoring is enabled but
|
||||
data collection is disabled.
|
||||
If you enable the Elastic {monitor-features} in your cluster, there are two
|
||||
methods to collect metrics about {es}:
|
||||
|
||||
The following method involves sending the metrics to the monitoring cluster by
|
||||
using exporters. For an alternative method, see <<configuring-metricbeat>>.
|
||||
|
||||
Advanced monitoring settings enable you to control how frequently data is
|
||||
collected, configure timeouts, and set the retention period for locally-stored
|
||||
monitoring indices. You can also adjust how monitoring data is displayed.
|
||||
* <<collecting-monitoring-data>>
|
||||
* <<configuring-metricbeat>>
|
||||
|
||||
To learn about monitoring in general, see
|
||||
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. Configure your cluster to collect monitoring data:
|
||||
|
||||
.. Verify that the `xpack.monitoring.enabled` setting is `true`, which is its
|
||||
default value, on each node in the cluster. For more information, see
|
||||
<<monitoring-settings>>.
|
||||
|
||||
.. Verify that the `xpack.monitoring.elasticsearch.collection.enabled` setting
|
||||
is `true`, which is its default value, on each node in the cluster.
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For more information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
.. Set the `xpack.monitoring.collection.enabled` setting to `true` on each
|
||||
node in the cluster. By default, it is is disabled (`false`).
|
||||
+
|
||||
--
|
||||
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
|
||||
node or across the cluster as a dynamic cluster setting. If {es}
|
||||
{security-features} are enabled, you must have `monitor` cluster privileges to
|
||||
view the cluster settings and `manage` cluster privileges to change them.
|
||||
|
||||
For example, use the following APIs to review and change this setting:
|
||||
|
||||
[source,js]
|
||||
----------------------------------
|
||||
GET _cluster/settings
|
||||
|
||||
PUT _cluster/settings
|
||||
{
|
||||
"persistent": {
|
||||
"xpack.monitoring.collection.enabled": true
|
||||
}
|
||||
}
|
||||
----------------------------------
|
||||
// CONSOLE
|
||||
|
||||
For more
|
||||
information, see <<monitoring-settings>> and <<cluster-update-settings>>.
|
||||
--
|
||||
|
||||
.. Optional: Specify which indices you want to monitor.
|
||||
+
|
||||
--
|
||||
By default, the monitoring agent collects data from all {es} indices.
|
||||
To collect data from particular indices, configure the
|
||||
`xpack.monitoring.collection.indices` setting. You can specify multiple indices
|
||||
as a comma-separated list or use an index pattern to match multiple indices. For
|
||||
example:
|
||||
|
||||
[source,yaml]
|
||||
----------------------------------
|
||||
xpack.monitoring.collection.indices: logstash-*, index1, test2
|
||||
----------------------------------
|
||||
|
||||
You can prepend `+` or `-` to explicitly include or exclude index names or
|
||||
patterns. For example, to include all indices that start with `test` except
|
||||
`test3`, you could specify `+test*,-test3`.
|
||||
--
|
||||
|
||||
.. Optional: Specify how often to collect monitoring data. The default value for
|
||||
the `xpack.monitoring.collection.interval` setting 10 seconds. See
|
||||
<<monitoring-settings>>.
|
||||
|
||||
. Identify where to store monitoring data.
|
||||
+
|
||||
--
|
||||
By default, the data is stored on the same cluster by using a
|
||||
<<local-exporter,`local` exporter>>.
|
||||
|
||||
Alternatively, you can use an <<http-exporter,`http` exporter>> to send data to
|
||||
a separate _monitoring cluster_.
|
||||
|
||||
For more information about typical monitoring architectures,
|
||||
see {stack-ov}/how-monitoring-works.html[How Monitoring Works].
|
||||
--
|
||||
|
||||
. If you choose to use an `http` exporter:
|
||||
|
||||
.. On the cluster that you want to monitor (often called the _production cluster_),
|
||||
configure each node to send metrics to your monitoring cluster. Configure an
|
||||
HTTP exporter in the `xpack.monitoring.exporters` settings in the
|
||||
`elasticsearch.yml` file. For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
.. If the Elastic {security-features} are enabled on the monitoring cluster, you
|
||||
must provide appropriate credentials when data is shipped to the monitoring cluster:
|
||||
|
||||
... Create a user on the monitoring cluster that has the
|
||||
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
|
||||
Alternatively, use the
|
||||
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
|
||||
|
||||
... Add the user ID and password settings to the HTTP exporter settings in the
|
||||
`elasticsearch.yml` file on each node. +
|
||||
+
|
||||
--
|
||||
For example:
|
||||
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
|
||||
auth.username: remote_monitoring_user
|
||||
auth.password: YOUR_PASSWORD
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
.. If you configured the monitoring cluster to use
|
||||
<<configuring-tls,encrypted communications>>, you must use the HTTPS protocol in
|
||||
the `host` setting. You must also specify the trusted CA certificates that will
|
||||
be used to verify the identity of the nodes in the monitoring cluster.
|
||||
|
||||
*** To add a CA certificate to an {es} node's trusted certificates, you can
|
||||
specify the location of the PEM encoded certificate with the
|
||||
`certificate_authorities` setting. For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
|
||||
auth:
|
||||
username: remote_monitoring_user
|
||||
password: YOUR_PASSWORD
|
||||
ssl:
|
||||
certificate_authorities: [ "/path/to/ca.crt" ]
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
*** Alternatively, you can configure trusted certificates using a truststore
|
||||
(a Java Keystore file that contains the certificates). For example:
|
||||
+
|
||||
--
|
||||
[source,yaml]
|
||||
--------------------------------------------------
|
||||
xpack.monitoring.exporters:
|
||||
id1:
|
||||
type: http
|
||||
host: ["https://es-mon1:9200", "https://es-mon2:9200"]
|
||||
auth:
|
||||
username: remote_monitoring_user
|
||||
password: YOUR_PASSWORD
|
||||
ssl:
|
||||
truststore.path: /path/to/file
|
||||
truststore.password: password
|
||||
--------------------------------------------------
|
||||
--
|
||||
|
||||
. Configure your cluster to route monitoring data from sources such as {kib},
|
||||
Beats, and {ls} to the monitoring cluster. The
|
||||
`xpack.monitoring.collection.enabled` setting must be `true` on each node in the
|
||||
cluster. For information about configuring each product to collect and send
|
||||
monitoring data, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
|
||||
|
||||
. If you updated settings in the `elasticsearch.yml` files on your production
|
||||
cluster, restart {es}. See <<stopping-elasticsearch>> and <<starting-elasticsearch>>.
|
||||
+
|
||||
--
|
||||
TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard
|
||||
allocation] before you restart your nodes to avoid unnecessary shard
|
||||
reallocation during the install process.
|
||||
|
||||
--
|
||||
|
||||
. Optional:
|
||||
<<config-monitoring-indices,Configure the indices that store the monitoring data>>.
|
||||
|
||||
. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}].
|
||||
|
||||
include::collecting-monitoring-data.asciidoc[]
|
||||
include::configuring-metricbeat.asciidoc[]
|
||||
include::indices.asciidoc[]
|
||||
include::{es-repo-dir}/settings/monitoring-settings.asciidoc[]
|
||||
Loading…
Reference in New Issue