Using Kafka Broker for Telem Data to Ensure Fault Tolerance
TLDR sarthak inquires about using kafka broker with ClickHouse for fault tolerance. Srikanth and Ankit discuss scale and data flow feasibility. sarthak also requests a circuit-breaking mechanism, which Ankit explains is already in place.
May 17, 2023 (4 months ago)
May 18, 2023 (4 months ago)
correct. I think the data flow would look like otel-collector => kafka => clickhouse so we expect Kafka to handle bursts in traffic and downtime of clickhouse
Indexed 825 threads (61% resolved)
Issues with Opentelemetry Maven Plugin and Kafka Exporter
sarthak is facing issues implementing Kafka with Maven for Opentelemetry while trying to decouple the source application performance from the Signoz cluster. Pranay and nitya-signoz ask clarifying questions about the desired flow.
Configuring OTel Collector to Export Data to SigNoz
James asked how to configure the otel collector to send data to SigNoz. Ankit suggested using the otlp exporter in the collector configuration. James found that SigNoz uses its own otel collector and adjusted the port bindings accordingly.
Setting Up Dual Exporters for Otel-collector and Resolving Payload Error
Guillaume was facing an issue setting up dual exporters for otel-collector and had a pod not being "Ready". Srikanth suggested checking error logs and dealing with large payloads, which led to a successful setup.