From 62baa85632f4eee76ef6efbad2a878b648b6b888 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Fri, 27 Sep 2019 14:58:10 -0700 Subject: [PATCH] [DOCS] Adds production monitoring page (#47184) --- .../collecting-monitoring-data.asciidoc | 2 +- docs/reference/monitoring/collectors.asciidoc | 2 +- .../monitoring/configuring-filebeat.asciidoc | 2 +- docs/reference/monitoring/index.asciidoc | 6 + docs/reference/monitoring/production.asciidoc | 124 ++++++++++++++++++ .../monitoring/troubleshooting.asciidoc | 29 ++++ .../settings/monitoring-settings.asciidoc | 6 +- 7 files changed, 165 insertions(+), 6 deletions(-) create mode 100644 docs/reference/monitoring/production.asciidoc create mode 100644 docs/reference/monitoring/troubleshooting.asciidoc diff --git a/docs/reference/monitoring/collecting-monitoring-data.asciidoc b/docs/reference/monitoring/collecting-monitoring-data.asciidoc index a2c95014b33..bd47d94ba19 100644 --- a/docs/reference/monitoring/collecting-monitoring-data.asciidoc +++ b/docs/reference/monitoring/collecting-monitoring-data.asciidoc @@ -107,7 +107,7 @@ cluster that stores the monitoring data must have at least one <>. For more information about typical monitoring architectures, -see {stack-ov}/how-monitoring-works.html[How Monitoring Works]. +see <>. -- . If you choose to use an `http` exporter: diff --git a/docs/reference/monitoring/collectors.asciidoc b/docs/reference/monitoring/collectors.asciidoc index 568d21e8354..2a091e1c23c 100644 --- a/docs/reference/monitoring/collectors.asciidoc +++ b/docs/reference/monitoring/collectors.asciidoc @@ -140,4 +140,4 @@ related to monitoring data, which can be very useful when there are a large number of Logstash nodes or Beats. For more information about typical monitoring architectures, see -{xpack-ref}/how-monitoring-works.html[How Monitoring Works]. +<>. diff --git a/docs/reference/monitoring/configuring-filebeat.asciidoc b/docs/reference/monitoring/configuring-filebeat.asciidoc index b1e22d38f54..88b9859c87f 100644 --- a/docs/reference/monitoring/configuring-filebeat.asciidoc +++ b/docs/reference/monitoring/configuring-filebeat.asciidoc @@ -23,7 +23,7 @@ TIP: In production environments, we strongly recommend using a separate cluster monitoring cluster prevents production cluster outages from impacting your ability to access your monitoring data. It also prevents monitoring activities from impacting the performance of your production cluster. See -{stack-ov}/monitoring-production.html[Monitoring in a production environment]. +<>. -- diff --git a/docs/reference/monitoring/index.asciidoc b/docs/reference/monitoring/index.asciidoc index 937a01340b3..362dd425982 100644 --- a/docs/reference/monitoring/index.asciidoc +++ b/docs/reference/monitoring/index.asciidoc @@ -10,12 +10,14 @@ performance of your {es} cluster. * <> * <> +* <> * <> * <> * <> * <> * <> * <> +* <> -- @@ -23,6 +25,8 @@ include::overview.asciidoc[] include::how-monitoring-works.asciidoc[] +include::production.asciidoc[] + include::collecting-monitoring-data.asciidoc[] include::pause-export.asciidoc[] @@ -37,3 +41,5 @@ include::collectors.asciidoc[] include::exporters.asciidoc[] include::local-export.asciidoc[] include::http-export.asciidoc[] + +include::troubleshooting.asciidoc[] diff --git a/docs/reference/monitoring/production.asciidoc b/docs/reference/monitoring/production.asciidoc new file mode 100644 index 00000000000..9b2dd1e1b80 --- /dev/null +++ b/docs/reference/monitoring/production.asciidoc @@ -0,0 +1,124 @@ +[role="xpack"] +[[monitoring-production]] +== Monitoring in a production environment + +In production, you should send monitoring data to a separate _monitoring cluster_ +so that historical data is available even when the nodes you are monitoring are +not. For example, you can use {metricbeat} to ship monitoring data about {kib}, +{es}, {ls}, and Beats to the monitoring cluster. +//If you are sending your data to the {esms-init}, see <>. + +If you have at least a gold license, using a dedicated monitoring cluster also +enables you to monitor multiple clusters from a central location. + +To store monitoring data in a separate cluster: + +. Set up the {es} cluster you want to use as the monitoring cluster. +For example, you might set up a two host cluster with the nodes `es-mon-1` and +`es-mon-2`. ++ +-- +[IMPORTANT] +=============================== +* To monitor an {es} {major-version} cluster, you must run {es} {major-version} +on the monitoring cluster. +* There must be at least one <> in the monitoring +cluster; it does not need to be a dedicated ingest node. +=============================== +-- + +.. (Optional) Verify that the collection of monitoring data is disabled on the +monitoring cluster. By default, the `xpack.monitoring.collection.enabled` setting +is `false`. ++ +-- +For example, you can use the following APIs to review and change this setting: + +[source,console] +---------------------------------- +GET _cluster/settings + +PUT _cluster/settings +{ + "persistent": { + "xpack.monitoring.collection.enabled": false + } +} +---------------------------------- + +-- + +.. If the {es} {security-features} are enabled on the monitoring cluster, create +users that can send and retrieve monitoring data. ++ +-- +NOTE: If you plan to use {kib} to view monitoring data, username and password +credentials must be valid on both the {kib} server and the monitoring cluster. + +-- + +*** If you plan to use {metricbeat} to collect data about {es} or {kib}, +create a user that has the `remote_monitoring_collector` built-in role and a +user that has the `remote_monitoring_agent` +{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[built-in role]. Alternatively, use the +`remote_monitoring_user` {stack-ov}/built-in-users.html[built-in user]. + +*** If you plan to use HTTP exporters to route data through your production +cluster, create a user that has the `remote_monitoring_agent` +{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[built-in role]. ++ +-- +For example, the +following request creates a `remote_monitor` user that has the +`remote_monitoring_agent` role: + +[source, sh] +--------------------------------------------------------------- +POST /_security/user/remote_monitor +{ + "password" : "changeme", + "roles" : [ "remote_monitoring_agent"], + "full_name" : "Internal Agent For Remote Monitoring" +} +--------------------------------------------------------------- +// CONSOLE +// TEST[skip:needs-gold+-license] + +Alternatively, use the `remote_monitoring_user` {stack-ov}/built-in-users.html[built-in user]. +-- + +. Configure your production cluster to collect data and send it to the +monitoring cluster. + +** <>. + +** <>. + +. (Optional) +{logstash-ref}/configuring-logstash.html[Configure {ls} to collect data and send it to the monitoring cluster]. + +. (Optional) Configure the Beats to collect data and send it to the monitoring +cluster. +** {auditbeat-ref}/monitoring.html[Auditbeat] +** {filebeat-ref}/monitoring.html[Filebeat] +** {heartbeat-ref}/monitoring.html[Heartbeat] +** {metricbeat-ref}/monitoring.html[Metricbeat] +** {packetbeat-ref}/monitoring.html[Packetbeat] +** {winlogbeat-ref}/monitoring.html[Winlogbeat] + +. (Optional) Configure {kib} to collect data and send it to the monitoring cluster: + +** {kibana-ref}/monitoring-metricbeat.html[Use {metricbeat}]. + +** {kibana-ref}/monitoring-kibana.html[Use HTTP exporters]. + +. (Optional) Create a dedicated {kib} instance for monitoring, rather than using +a single {kib} instance to access both your production cluster and monitoring +cluster. + +.. (Optional) Disable the collection of monitoring data in this {kib} instance. +Set the `xpack.monitoring.kibana.collection.enabled` setting to `false` in the +`kibana.yml` file. For more information about this setting, see +{kibana-ref}/monitoring-settings-kb.html[Monitoring settings in {kib}]. + +. {kibana-ref}/monitoring-data.html[Configure {kib} to retrieve and display the monitoring data]. diff --git a/docs/reference/monitoring/troubleshooting.asciidoc b/docs/reference/monitoring/troubleshooting.asciidoc new file mode 100644 index 00000000000..74e18558302 --- /dev/null +++ b/docs/reference/monitoring/troubleshooting.asciidoc @@ -0,0 +1,29 @@ +[[monitoring-troubleshooting]] +== Troubleshooting monitoring +++++ +Troubleshooting +++++ + +Use the information in this section to troubleshoot common problems and find +answers for frequently asked questions. See also +{logstash-ref}/monitoring-troubleshooting.html[Troubleshooting monitoring in {ls}]. + +For issues that you cannot fix yourself … we’re here to help. +If you are an existing Elastic customer with a support contract, please create +a ticket in the +https://support.elastic.co/customers/s/login/[Elastic Support portal]. +Or post in the https://discuss.elastic.co/[Elastic forum]. + +*Symptoms*: +There is no information about your cluster on the *Stack Monitoring* page in +{kib}. + +*Resolution*: +Check whether the appropriate indices exist on the monitoring cluster. For +example, use the <> command to verify that +there is a `.monitoring-kibana*` index for your {kib} monitoring data and a +`.monitoring-es*` index for your {es} monitoring data. If you are collecting +monitoring data by using {metricbeat} the indices have `-mb` in their names. If +the indices do not exist, review your configuration. For example, see +<>. + diff --git a/docs/reference/settings/monitoring-settings.asciidoc b/docs/reference/settings/monitoring-settings.asciidoc index 18379577e6a..5866c0bae39 100644 --- a/docs/reference/settings/monitoring-settings.asciidoc +++ b/docs/reference/settings/monitoring-settings.asciidoc @@ -117,9 +117,9 @@ the `http` exporter will not be deleted automatically. Configures where the agent stores monitoring data. By default, the agent uses a local exporter that indexes monitoring data on the cluster where it is installed. Use an HTTP exporter to send data to a separate monitoring cluster. For more -information, see <>, -<>, and -{xpack-ref}/how-monitoring-works.html[How Monitoring Works]. +information, see <>, +<>, and +<>. [float] [[local-exporter-settings]]