Instrumenting Metric Data to Kafka in JSON Format

TLDR kapila wanted to instrument metric data in JSON format to Kafka but struggled with OTLP format. Ankit advised changing the encoding. kapila then asked about loading otel metric data to Druid via Kafka, but received no response.

Photo of kapila
kapila
Tue, 04 Jul 2023 08:18:21 UTC

I am trying to instrument a metric data to kafka via otel in json format. But the data is written in otlp format in kafka topic. Any inputs please? receivers: otlp: protocols: grpc: http: exporters: logging: kafka/metrics: protocol_version: 2.0.0 topic: otlp.metrics brokers: - localhost:9092 encoding: otlp_proto connectors: spanmetrics: service: pipelines: metrics: receivers: [otlp, http] processors: [filter, batch] exporters: [kafka/metrics]

Photo of Ankit
Ankit
Tue, 04 Jul 2023 08:25:56 UTC

change the encoding? `encoding: otlp_json`

Photo of kapila
kapila
Tue, 04 Jul 2023 08:49:35 UTC

thanks

Photo of kapila
kapila
Tue, 04 Jul 2023 11:28:25 UTC

I am trying to load otel metric data to druid via kafka. do you have any sample spec, how the json telemetery data can be flattened to druid via kafka?

Photo of kapila
kapila
Tue, 04 Jul 2023 11:28:38 UTC

Photo of kapila
kapila
Tue, 04 Jul 2023 11:28:42 UTC

referring here