#support

Issues with Opentelemetry Maven Plugin and Kafka Exporter

TLDR sarthak is facing issues implementing Kafka with Maven for Opentelemetry while trying to decouple the source application performance from the Signoz cluster. Pranay and nitya-signoz ask clarifying questions about the desired flow.

today

Join the chat

May 25, 2023 (4 days ago)

Photo of md5-bc67a182dac7a9be9e74164eebcd86ee

sarthak

07:02 AM

i m trying to use kafka as a log receiver to collect telemetry events from application and then consume from that topic to export all data to the clickhouse db so i after reading about otel configuration and opentelemetry maven plugin , so i created one dedicated topic for this purpose and configured this in my sample spring boot application and provided kafka broker address and topic name in the application.properties so that opntelemetry can push telemetry events to kafka and also added kafka broker address and topic name in signoz-collector.yaml file so that it can read the data from topic and export to clickhouse but what problem i m constantly facing is that , i m not able to receive any event on topic and also opntelemetry is always trying to connect with localhost:4317 which is deafault grpc endpoint reciever in otlp so not able to ovveride that default configuration so can anyone give any input on the same if anyone tried to use opentelemetry maven plugin with kafka exporter ?

Photo of md5-8df7ce0274b2473ec07403336e48b574

Pranay

01:47 PM

sarthak Are you trying to send directly from Kafka topic to ClickHouse DB or you are sending from Kafka topic to otel collector to ClickHouse?

Also, curious why are you trying ingest to Kafka? What's your use case?

May 26, 2023 (3 days ago)

sarthak

05:34 AM

Hi Pranay , actually initially i deployed signoz with it's default grpc endpoint otel receiver but what i observed when i instrumented one of high throuput microservices is when signoz cluster was down due to high cpu utilisation , the otel agent was continuously throwing that grpc error as was not able to make a connection and after 1,2 days i found service unhealthy so it was affecting the source application performance during signoz downtime , so after doing some research on opntelemetry, if found that we can also use kafka as a telemetry event receiver by configuring broker address and topic name in source application so that i can decouple the service directly with signoz and produce data in async mode into kafka which can be read by signoz to export to clickhouse .

05:36

sarthak

05:36 AM

mainly the use case is to fully prevent source application performance and downtime in case signoz is not up

May 29, 2023 (today)

Photo of md5-a52b9d6c34f193d9a1ff940024f36f77

nitya-signoz

04:28 AM

If I understand correctly this is your flow you want to achieve

otel_events(application) -> Kafka -> otel_collector -> clickhouse is what you are trying to acheive right ?

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Join Our Community

Similar Threads

Sending Metrics of Kafka App to Signoz Otel Collector

Shubham wanted advice to send metrics to a signoz otel collector. nitya-signoz provided guidance and Srikanth confirmed using the collector for Kafka server telemetry.

Using Kafka Broker for Telem Data to Ensure Fault Tolerance

sarthak inquires about using kafka broker with ClickHouse for fault tolerance. Srikanth and Ankit discuss scale and data flow feasibility. sarthak also requests a circuit-breaking mechanism, which Ankit explains is already in place.

Injecting Otelp and Resolving "Something Went Wrong" Issue

sarthak had questions about injecting Otelp agent into multiple microservices and faced an issue on SigNoz portal. Srikanth clarified Otelp agent usage and recommended checking docker and query-service.

1mo

Configuring SigNoz with Istio Traces and Service Monitors

Zakaria asked about integrating SigNoz with istio traces and Prometheus service monitors. Srikanth suggested using OpenTelemetry operator and provided guidance for troubleshooting and solving configuration issues in the process.

1mo

Troubleshooting SigNoz Auto-Instrumentation Configuration

igor is having trouble configuring auto-instrumentation for Java applications using SigNoz, with traces not appearing in the SigNoz UI. Prashant advises to check logs of the otel sidecar, use service name for endpoint, verify supported libraries, and test with telemetrygen. However, the issue still persists.

1mo