4.9 KiB
layout | title | parent | nav_order | redirect_from | |
---|---|---|---|---|---|
default | Metrics framework | Trace Analytics | 65 |
|
This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated GitHub issue.
{: .warning}
While the OpenSearch Stats APIs offer insight into the inner workings of each node and an OpenSearch cluster as a whole, the statistics lack certain details, such as percentiles, and do not provide the semantics of richer metric types, like histograms. Consequently, identifying outliers within cluster statistics becomes challenging when using only the Stats API.
The metrics framework feature adds comprehensive metrics support to effectively monitor an OpenSearch cluster. Using the Metrics Framework APIs, plugin, and extension, developers can add new monitoring metrics. In addition, the OpenSearch distribution bundles the telemetry-otel
plugin, which provides the implementation for metrics instrumentation based on the OpenTelemetry Java SDK.
Getting started
The metrics framework feature is experimental as of OpenSearch 2.11. To begin using the metrics framework feature, you need to first enable the telemetry feature
by using the opensearch.experimental.feature.telemetry.enabled
feature flag and subsequently by using the metrics framework feature flag.
Enabling this feature can consume system resources. Before enabling the metrics framework feature, determine whether you have sufficient cluster resources to allocate. {: .warning}
Enabling the feature flag on a node using tarball
The enable
flag is toggled using a Java Virtual Machine (JVM) parameter that is set either in OPENSEARCH_JAVA_OPTS
or in config/jvm.options
.
Option 1: Enable the experimental feature flag in the opensearch.yml
file
- Navigate to your OpenSearch directory using the following command:
cd \path\to\opensearch
- Open your
opensearch.yaml
file. - Add the following setting to
opensearch.yaml
:
opensearch.experimental.feature.telemetry.enabled=true
{% include copy.html %}
- Save your changes and close the file.
Option 2: Modify jvm.options
To enable the metrics framework feature using jvm
, add the following line to config/jvm.options
before starting OpenSearch:
-Dopensearch.experimental.feature.telemetry.enabled=true
{% include copy.html %}
Option 3: Enable from an environment variable
You can enable the metrics framework feature with a single command by adding the metrics framework environment variable to the OPENSEARCH_JAVA_OPTS
command, as shown in the following example:
OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.telemetry.enabled=true" ./opensearch-2.9.0/bin/opensearch
{% include copy.html %}
You can also define the environment variable separately before running OpenSearch by running the following command:
export OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.telemetry.enabled=true"
./bin/opensearch
{% include copy.html %}
Enable with Docker
If you’re running OpenSearch using Docker, add the following line to docker-compose.yml
under environment
:
OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.telemetry.enabled=true"
{% include copy.html %}
Enable the metrics framework feature
Once you've enabled the feature flag, you can enable the metrics framework feature by using the following setting, which enables metrics in the opensearch.yaml
file:
telemetry.feature.metrics.enabled=true
The metrics framework feature supports various telemetry solutions through plugins. Use the following instructions to enable the telemetry-otel
plugin:
- Publish interval: The metrics framework feature can locally aggregate metrics with unique information about the configured publish interval and then export those metrics. By default, the interval is 1 minute. However, you can change the interval using the
telemetry.otel.metrics.publish.interval
cluster setting. - Exporters: Exporters are responsible for persisting the data. OpenTelemetry provides several out-of-the-box exporters. OpenSearch supports the following exporters:
LoggingMetricExporter
: Exports metrics to a log file, generating a separate file in the logs directory_otel_metrics.log
. Default istelemetry.otel.metrics.exporter.class=io.opentelemetry.exporter.logging.LoggingMetricExporter
.OtlpGrpcMetricExporter
: Exports spans through gRPC. To use this exporter, you need to install theotel-collector
on the node. By default, it writes to the http://localhost:4317/ endpoint. To use this exporter, set the following static setting:telemetry.otel.metrics.exporter.class=io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter
.