You can use Data Prepper to derive metrics from OpenTelemetry traces. The following example pipeline receives incoming traces and extracts a metric called `durationInNanos`, aggregated over a tumbling window of 30 seconds. It then derives a histogram from the incoming traces.
The pipeline contains the following pipelines:
-`entry-pipeline`– Receives trace data from the OpenTelemetry collector and forwards it to the `trace_to_metrics_pipeline` pipeline.
-`trace-to-metrics-pipeline` - Receives the trace data from the `entry-pipeline` pipeline, aggregates it, and derives a histogram of `durationInNanos` from the traces based on the value of the `serviceName` field. It then sends the derived metrics to the OpenSearch index called `metrics_for_traces`.
```json
entry-pipeline:
source:
otel_trace_source:
# Provide the path for ingestion. ${pipelineName} will be replaced with pipeline name.
# In this case it would be "/entry-pipeline/v1/traces". This will be endpoint URI path in OpenTelemetry Exporter configuration.
path: "/${pipelineName}/v1/traces"
sink:
- pipeline:
name: "trace-to-metrics-pipeline"
trace-to-metrics-pipeline:
source:
pipeline:
name: "entry-pipeline"
processor:
- aggregate:
# Pick the required identification keys
identification_keys: ["serviceName"]
action:
histogram:
# Pick the appropriate values for each of the following fields
