mirror of https://github.com/apache/druid.git
166 lines
4.4 KiB
Markdown
166 lines
4.4 KiB
Markdown
<!--
|
|
~ Licensed to the Apache Software Foundation (ASF) under one
|
|
~ or more contributor license agreements. See the NOTICE file
|
|
~ distributed with this work for additional information
|
|
~ regarding copyright ownership. The ASF licenses this file
|
|
~ to you under the Apache License, Version 2.0 (the
|
|
~ "License"); you may not use this file except in compliance
|
|
~ with the License. You may obtain a copy of the License at
|
|
~
|
|
~ http://www.apache.org/licenses/LICENSE-2.0
|
|
~
|
|
~ Unless required by applicable law or agreed to in writing,
|
|
~ software distributed under the License is distributed on an
|
|
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
~ KIND, either express or implied. See the License for the
|
|
~ specific language governing permissions and limitations
|
|
~ under the License.
|
|
-->
|
|
|
|
# OpenTelemetry Emitter
|
|
|
|
The [OpenTelemetry](https://opentelemetry.io/) emitter generates OpenTelemetry Spans for queries.
|
|
|
|
## How OpenTelemetry emitter works
|
|
|
|
The [OpenTelemetry](https://opentelemetry.io/) emitter processes `ServiceMetricEvent` events for the `query/time`
|
|
metric. It extracts OpenTelemetry context from
|
|
the [query context](https://druid.apache.org/docs/latest/querying/query-context.html). To link druid spans to parent
|
|
traces, the query context should contain at least `traceparent` key.
|
|
See [context propagation](https://www.w3.org/TR/trace-context/) for more information. If no `traceparent` key is
|
|
provided, then spans are created without `parentTraceId` and are not linked to the parent span. In addition, the emitter
|
|
also adds other druid context entries to the span attributes.
|
|
|
|
## Configuration
|
|
|
|
### Enabling
|
|
|
|
To enable the OpenTelemetry emitter, add the extension and enable the emitter in `common.runtime.properties`.
|
|
|
|
Load the plugin:
|
|
|
|
```properties
|
|
druid.extensions.loadList=[..., "opentelemetry-emitter"]
|
|
```
|
|
|
|
Then there are 2 options:
|
|
|
|
* You want to use only `opentelemetry-emitter`
|
|
|
|
```properties
|
|
druid.emitter=opentelemetry
|
|
```
|
|
|
|
* You want to use `opentelemetry-emitter` with other emitters
|
|
|
|
```properties
|
|
druid.emitter=composing
|
|
druid.emitter.composing.emitters=[..., "opentelemetry"]
|
|
```
|
|
|
|
_*More about Druid configuration [here](https://druid.apache.org/docs/latest/configuration/index.html)._
|
|
|
|
## Testing
|
|
|
|
### Part 1: Run zipkin and otel-collector
|
|
|
|
Create `docker-compose.yaml` in your working dir:
|
|
|
|
```yaml
|
|
version: "2"
|
|
services:
|
|
|
|
zipkin-all-in-one:
|
|
image: openzipkin/zipkin:latest
|
|
ports:
|
|
- "9411:9411"
|
|
|
|
otel-collector:
|
|
image: otel/opentelemetry-collector:latest
|
|
command: ["--config=otel-local-config.yaml", "${OTELCOL_ARGS}"]
|
|
volumes:
|
|
- ${PWD}/config.yaml:/otel-local-config.yaml
|
|
ports:
|
|
- "4317:4317"
|
|
```
|
|
|
|
Create `config.yaml` file with configuration for otel-collector:
|
|
|
|
```yaml
|
|
version: "2"
|
|
receivers:
|
|
receivers:
|
|
otlp:
|
|
protocols:
|
|
grpc:
|
|
|
|
exporters:
|
|
zipkin:
|
|
endpoint: "http://zipkin-all-in-one:9411/api/v2/spans"
|
|
format: proto
|
|
|
|
logging:
|
|
|
|
processors:
|
|
batch:
|
|
|
|
service:
|
|
pipelines:
|
|
traces:
|
|
receivers: [otlp]
|
|
processors: [batch]
|
|
exporters: [logging, zipkin]
|
|
```
|
|
|
|
*_How to configure otel-collector you can read [here](https://opentelemetry.io/docs/collector/configuration/)._
|
|
|
|
Run otel-collector and zipkin.
|
|
|
|
```bash
|
|
docker-compose up
|
|
```
|
|
|
|
### Part 2: Run Druid
|
|
|
|
Build Druid:
|
|
|
|
```bash
|
|
mvn clean install -Pdist
|
|
tar -C /tmp -xf distribution/target/apache-druid-0.21.0-bin.tar.gz
|
|
cd /tmp/apache-druid-0.21.0
|
|
```
|
|
|
|
Edit `conf/druid/single-server/micro-quickstart/_common/common.runtime.properties` to enable the emitter (
|
|
see `Configuration` section above).
|
|
|
|
Start the quickstart with the apppropriate environment variables for opentelemetry autoconfiguration:
|
|
|
|
```
|
|
OTEL_SERVICE_NAME="org.apache.druid" OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" bin/start-micro-quickstart
|
|
```
|
|
|
|
*_More about opentelemetry
|
|
autoconfiguration [here](https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions/autoconfigure)_
|
|
|
|
Load sample data - [example](https://druid.apache.org/docs/latest/tutorials/index.html#step-4-load-data).
|
|
|
|
### Part 3: Send queries
|
|
|
|
Create `query.json`:
|
|
|
|
```json
|
|
{
|
|
"query":"SELECT COUNT(*) as total FROM wiki WHERE countryName IS NOT NULL",
|
|
"context":{
|
|
"traceparent":"00-54ef39243e3feb12072e0f8a74c1d55a-ad6d5b581d7c29c1-01"
|
|
}
|
|
}
|
|
```
|
|
|
|
Send query:
|
|
|
|
```bash
|
|
curl -XPOST -H'Content-Type: application/json' http://localhost:8888/druid/v2/sql/ -d @query.json
|
|
```
|
|
|
|
Then open `http://localhost:9411/zipkin/` and you can see there your spans. |