honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[DOCS] Cleans up monitoring configuration steps (#35111)

This commit is contained in:
Lisa Cawley 2018-11-08 13:53:09 -08:00 committed by GitHub
parent 7e144b318a
commit c88b2d7e5a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 171 additions and 114 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

64
docs/reference/monitoring/configuring-metricbeat.asciidoc
View File

@ -15,9 +15,15 @@ To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
. Enable the collection of monitoring data. Set . Enable the collection of monitoring data. Set
`xpack.monitoring.collection.enabled` to `true` on the production cluster. + `xpack.monitoring.collection.enabled` to `true` on each node in the production
cluster. By default, it is is disabled (`false`).
+ +
-- --
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.
For example, you can use the following APIs to review and change this setting: For example, you can use the following APIs to review and change this setting:
[source,js] [source,js]
@ -31,16 +37,21 @@ PUT _cluster/settings
} }
} }
---------------------------------- ----------------------------------
// CONSOLE // CONSOLE
For more information, see <<monitoring-settings>> and <<cluster-update-settings>>. For more information, see <<monitoring-settings>> and <<cluster-update-settings>>.
-- --
. Disable the default collection of {es} monitoring metrics. Set . Disable the default collection of {es} monitoring metrics. Set
`xpack.monitoring.elasticsearch.collection.enabled` to `false` on the production `xpack.monitoring.elasticsearch.collection.enabled` to `false` on each node in
cluster. + the production cluster.
+ +
-- --
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.
For example, you can use the following API to change this setting: For example, you can use the following API to change this setting:
[source,js] [source,js]
@ -96,30 +107,26 @@ You must specify the following settings in the `modules.d/elasticsearch.yml` fil
- shard - shard
period: 10s period: 10s
hosts: ["http://localhost:9200"] <1> hosts: ["http://localhost:9200"] <1>
xpack.enabled: true xpack.enabled: true <2>
---------------------------------- ----------------------------------
<1> This setting identifies the host and port number that are used to access {es}. <1> This setting identifies the host and port number that are used to access {es}.
<2> This setting ensures that {kib} can read this monitoring data successfully.
That is to say, it's stored in the same location and format as monitoring data
that is sent by <<es-monitoring-exporters,exporters>>.
-- --
.. If {security} is enabled, you must also provide a user ID and password so that .. If Elastic {security-features} are enabled, you must also provide a user ID
{metricbeat} can collect metrics successfully. and password so that {metricbeat} can collect metrics successfully.
... Create or identify a user that you want to use to collect the metrics. ... Create a user on the production cluster that has the
+ {stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
-- Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
TIP: There is a `remote_monitoring_user` built-in user that grants the privileges
necessary for {metricbeat} to monitor {stack} products. See
{stack-ov}/built-in-users.html[Built-in users].
Alternatively, you can choose a different user and give them the
`remote_monitoring_collector` {stack-ov}/built-in-roles.html[built-in role].
--
... Add the `username` and `password` settings to the {es} module configuration ... Add the `username` and `password` settings to the {es} module configuration
file. file.
+ +
-- --
For example, add the following settings in the `modules.d/kibana.yml` file: For example, add the following settings in the `modules.d/elasticsearch.yml` file:
[source,yaml] [source,yaml]
---------------------------------- ----------------------------------
@ -158,19 +165,14 @@ For more information about these configuration options, see
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output]. {metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
-- --
.. If {security} is enabled on the monitoring cluster, you must provide a valid .. If {es} {security-features} are enabled on the monitoring cluster, you
user ID and password so that {metricbeat} can send metrics successfully. must provide a valid user ID and password so that {metricbeat} can send metrics
successfully.
... Create or identify a user that you want to use to send the metrics. ... Create a user on the monitoring cluster that has the
+ {stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
-- Alternatively, use the
TIP: There is a `remote_monitoring_user` built-in user that grants the privileges {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
necessary for {metricbeat} to monitor {stack} products. See
{stack-ov}/built-in-users.html[Built-in users].
Alternatively, you can choose a different user and give them the
`remote_monitoring_agent` {stack-ov}/built-in-roles.html[built-in role].
--
... Add the `username` and `password` settings to the {es} output information in ... Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file (`metricbeat.yml`): the {metricbeat} configuration file (`metricbeat.yml`):

181
docs/reference/monitoring/configuring-monitoring.asciidoc
View File

@ -6,8 +6,8 @@
<titleabbrev>Configuring monitoring</titleabbrev> <titleabbrev>Configuring monitoring</titleabbrev>
++++ ++++
If you enable the collection of monitoring data in your cluster, you can If you enable the Elastic {monitor-features} in your cluster, you can
optionally collect metrics about {es}. By default, {monitoring} is enabled but optionally collect metrics about {es}. By default, monitoring is enabled but
data collection is disabled. data collection is disabled.
The following method involves sending the metrics to the monitoring cluster by The following method involves sending the metrics to the monitoring cluster by
@ -20,15 +20,51 @@ monitoring indices. You can also adjust how monitoring data is displayed.
To learn about monitoring in general, see To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
. To collect monitoring data about your {es} cluster: . Configure your cluster to collect monitoring data:
.. Verify that the `xpack.monitoring.enabled`, .. Verify that the `xpack.monitoring.enabled` setting is `true`, which is its
`xpack.monitoring.collection.enabled`, and default value, on each node in the cluster. For more information, see
`xpack.monitoring.elasticsearch.collection.enabled` settings are `true` on each <<monitoring-settings>>.
node in the cluster. By default `xpack.monitoring.collection.enabled` is disabled
(`false`), and that overrides `xpack.monitoring.elasticsearch.collection.enabled`, .. Verify that the `xpack.monitoring.elasticsearch.collection.enabled` setting
which defaults to being enabled (`true`). Both settings can be set dynamically is `true`, which is its default value, on each node in the cluster.
at runtime. For more information, see <<monitoring-settings>>. +
--
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.
For more information, see <<monitoring-settings>> and <<cluster-update-settings>>.
--
.. Set the `xpack.monitoring.collection.enabled` setting to `true` on each
node in the cluster. By default, it is is disabled (`false`).
+
--
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.
For example, use the following APIs to review and change this setting:
[source,js]
----------------------------------
GET _cluster/settings
PUT _cluster/settings
{
"persistent": {
"xpack.monitoring.collection.enabled": true
}
}
----------------------------------
// CONSOLE
For more
information, see <<monitoring-settings>> and <<cluster-update-settings>>.
--
.. Optional: Specify which indices you want to monitor. .. Optional: Specify which indices you want to monitor.
+ +
@ -53,62 +89,71 @@ patterns. For example, to include all indices that start with `test` except
the `xpack.monitoring.collection.interval` setting 10 seconds. See the `xpack.monitoring.collection.interval` setting 10 seconds. See
<<monitoring-settings>>. <<monitoring-settings>>.
. Optional: Configure your cluster to route monitoring data from sources such
as {kib}, Beats, and Logstash to a monitoring cluster:
.. Verify that `xpack.monitoring.collection.enabled` settings are `true` on each
node in the cluster.
.. {stack-ov}/xpack-monitoring.html[Configure {monitoring} across the Elastic Stack].
. Identify where to store monitoring data. . Identify where to store monitoring data.
+ +
-- --
By default, {monitoring} uses a `local` exporter that indexes monitoring data By default, the data is stored on the same cluster by using a
on the same cluster. See <<es-monitoring-default-exporter>> and <<local-exporter>>. <<local-exporter,`local` exporter>>.
Alternatively, you can use an `http` exporter to send data to a separate Alternatively, you can use an <<http-exporter,`http` exporter>> to send data to
monitoring cluster. See <<http-exporter>>. a separate _monitoring cluster_.
For more information about typical monitoring architectures, For more information about typical monitoring architectures,
see {stack-ov}/how-monitoring-works.html[How Monitoring Works]. see {stack-ov}/how-monitoring-works.html[How Monitoring Works].
-- --
. If {security} is enabled and you are using an `http` exporter to send data to . If you choose to use an `http` exporter:
a dedicated monitoring cluster:
.. Create a user on the monitoring cluster that has the .. On the cluster that you want to monitor (often called the _production cluster_),
{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[`remote_monitoring_agent` built-in role]. configure each node to send metrics to your monitoring cluster. Configure an
For example, the following request creates a `remote_monitor` user that has the HTTP exporter in the `xpack.monitoring.exporters` settings in the
`remote_monitoring_agent` role: `elasticsearch.yml` file. For example:
+ +
-- --
[source, sh] [source,yaml]
--------------------------------------------------------------- --------------------------------------------------
POST /_xpack/security/user/remote_monitor xpack.monitoring.exporters:
{ id1:
"password" : "changeme", type: http
"roles" : [ "remote_monitoring_agent"], host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
"full_name" : "Internal Agent For Remote Monitoring" --------------------------------------------------
}
---------------------------------------------------------------
// CONSOLE
// TEST[skip:needs-gold+-license]
-- --
.. On each node in the cluster that is being monitored, configure the `http` .. If the Elastic {security-features} are enabled on the monitoring cluster, you
exporter to use the appropriate credentials when data is shipped to the must provide appropriate credentials when data is shipped to the monitoring cluster:
monitoring cluster.
... Create a user on the monitoring cluster that has the
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
Alternatively, use the
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].
... Add the user ID and password settings to the HTTP exporter settings in the
`elasticsearch.yml` file on each node. +
+ +
-- --
If SSL/TLS is enabled on the monitoring cluster, you must use the HTTPS protocol For example:
in the `host` setting. You must also include the CA certificate in each node's
trusted certificates in order to verify the identities of the nodes in the
monitoring cluster.
The following example specifies the location of the PEM encoded certificate with [source,yaml]
the `certificate_authorities` setting: --------------------------------------------------
xpack.monitoring.exporters:
id1:
type: http
host: ["http://es-mon-1:9200", "http://es-mon2:9200"]
auth.username: remote_monitoring_user
auth.password: YOUR_PASSWORD
--------------------------------------------------
--
.. If you configured the monitoring cluster to use
<<configuring-tls,encrypted communications>>, you must use the HTTPS protocol in
the `host` setting. You must also specify the trusted CA certificates that will
be used to verify the identity of the nodes in the monitoring cluster.
*** To add a CA certificate to an {es} node's trusted certificates, you can
specify the location of the PEM encoded certificate with the
`certificate_authorities` setting. For example:
+
--
[source,yaml] [source,yaml]
-------------------------------------------------- --------------------------------------------------
xpack.monitoring.exporters: xpack.monitoring.exporters:
@ -116,18 +161,17 @@ xpack.monitoring.exporters:
type: http type: http
host: ["https://es-mon1:9200", "https://es-mon2:9200"] host: ["https://es-mon1:9200", "https://es-mon2:9200"]
auth: auth:
username: remote_monitor <1> username: remote_monitoring_user
password: changeme password: YOUR_PASSWORD
ssl: ssl:
certificate_authorities: [ "/path/to/ca.crt" ] certificate_authorities: [ "/path/to/ca.crt" ]
id2:
type: local
-------------------------------------------------- --------------------------------------------------
<1> The `username` and `password` parameters provide the user credentials. --
Alternatively, you can configure trusted certificates using a truststore
(a Java Keystore file that contains the certificates):
*** Alternatively, you can configure trusted certificates using a truststore
(a Java Keystore file that contains the certificates). For example:
+
--
[source,yaml] [source,yaml]
-------------------------------------------------- --------------------------------------------------
xpack.monitoring.exporters: xpack.monitoring.exporters:
@ -135,25 +179,28 @@ xpack.monitoring.exporters:
type: http type: http
host: ["https://es-mon1:9200", "https://es-mon2:9200"] host: ["https://es-mon1:9200", "https://es-mon2:9200"]
auth: auth:
username: remote_monitor username: remote_monitoring_user
password: changeme password: YOUR_PASSWORD
ssl: ssl:
truststore.path: /path/to/file truststore.path: /path/to/file
truststore.password: password truststore.password: password
id2:
type: local
-------------------------------------------------- --------------------------------------------------
-- --
. If {security} is enabled and you want to visualize monitoring data in {kib}, . Configure your cluster to route monitoring data from sources such as {kib},
you must create users that have access to the {kib} indices and permission to Beats, and {ls} to the monitoring cluster. The
read from the monitoring indices. `xpack.monitoring.collection.enabled` setting must be `true` on each node in the
cluster. For information about configuring each product to collect and send
monitoring data, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
. If you updated settings in the `elasticsearch.yml` files on your production
cluster, restart {es}. See <<stopping-elasticsearch>> and <<starting-elasticsearch>>.
+ +
-- --
You set up {monitoring} UI users on the cluster where the monitoring data is TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard
stored, that is to say the monitoring cluster. To grant all of the necessary permissions, assign users the allocation] before you restart your nodes to avoid unnecessary shard
`monitoring_user` and `kibana_user` roles. For more information, see reallocation during the install process.
{stack-ov}/mapping-roles.html[Mapping users and groups to roles].
-- --
. Optional: . Optional:

40
docs/reference/monitoring/index.asciidoc
View File

@ -5,38 +5,46 @@
[partintro] [partintro]
-- --
{monitoring} enables you to easily monitor the health of your {es} cluster. The The Elastic {monitor-features} enable you to easily monitor the health of
monitoring metrics are collected from each node and stored in {es} indices. your {es} cluster. The monitoring metrics are collected from each node and
stored in {es} indices.
TIP: In production environments, it is recommended to store the monitoring data
in a separate _monitoring cluster_. See
{stack-ov}/monitoring-production.html[Monitoring in a production environment].
Each {es} node is considered unique based on its persistent UUID, which is Each {es} node is considered unique based on its persistent UUID, which is
written on first start to its <<path-settings,`path.data`>> directory, which written on first start to its <<path-settings,`path.data`>> directory, which
defaults to `./data`. defaults to `./data`.
All settings associated with {monitoring} in {es} must be set in either the All settings associated with monitoring in {es} must be set in either the
`elasticsearch.yml` file for each node or, where possible, in the dynamic `elasticsearch.yml` file for each node or, where possible, in the dynamic
cluster settings. For more information, see <<configuring-monitoring>>. cluster settings. For more information, see <<configuring-monitoring>>.
[[es-monitoring-overview]] [[es-monitoring-overview]]
{es} is also at the core of {monitoring} across the Elastic Stack. In all cases, {es} is also at the core of monitoring across the {stack}. In all cases,
{monitoring} documents are just ordinary JSON documents built by monitoring each monitoring documents are just ordinary JSON documents built by monitoring each
Elastic Stack component at some collection interval, then indexing those {stack} component at some collection interval, then indexing those
documents into the monitoring cluster. Each component in the stack is documents into the monitoring cluster.
responsible for monitoring itself and then forwarding those documents to {es}
for both routing and indexing (storage).
The routing and indexing processes in {es} are handled by what are called Each component in the stack is responsible for monitoring itself and then
<<es-monitoring-collectors,collectors>> and forwarding those documents to the {es} production cluster for both routing and
<<es-monitoring-exporters,exporters>>. In the past, collectors and exporters indexing (storage). The routing and indexing processes in {es} are handled by
were considered to be part of a monitoring "agent", but that term is generally what are called <<es-monitoring-collectors,collectors>> and
not used anymore. <<es-monitoring-exporters,exporters>>.
beta[] Alternatively, in 6.4 and later, you can use {metricbeat} to collect
monitoring data about {kib} and ship it directly to the monitoring cluster,
rather than routing it through the production cluster. In 6.5 and later, you
can also use {metricbeat} to collect and ship data about {es}.
You can view monitoring data from {kib} where it’s easy to spot issues at a You can view monitoring data from {kib} where it’s easy to spot issues at a
glance or delve into the system behavior over time to diagnose operational glance or delve into the system behavior over time to diagnose operational
issues. In addition to the built-in status warnings, you can also set up custom issues. In addition to the built-in status warnings, you can also set up custom
alerts based on the data in the monitoring indices. alerts based on the data in the monitoring indices.
For an introduction to monitoring your Elastic stack, including Beats, Logstash, For an introduction to monitoring your {stack}, including Beats, {ls}, and {kib},
and {kib}, see {xpack-ref}/xpack-monitoring.html[Monitoring the Elastic Stack]. see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
-- --