#general

Monitoring Signoz-k8s-metrics-*-infra-otel-agent Telemetry Data

TLDR Al inquires about the best practices to monitor the telemetry data shipping. Srikanth suggests checking collector's internal metrics and resolving the underlying issue.

Powered by Struct AI
3
4mo
Solved
Join the chat
May 03, 2023 (4 months ago)
Al
Photo of md5-001a6b8b05601dc8ac56c5f364768cc1
Al
09:50 PM
Hi everyone,

Is there a standard or best practice to monitor or detect when a signoz-k8s-metrics-*-infra-otel-agent stops collecting and shipping telemetry data?
For example, we have dozens across many clusters and occasionally, I'll notice one has stopped shipping data and we'll restart it.
May 04, 2023 (4 months ago)
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
12:46 AM
Each of these collectors sends its own internal metrics, which is ideal for monitoring them but I would suggest finding the reason why did they stop shipping data because that sounds like a bigger issue.
Al
Photo of md5-001a6b8b05601dc8ac56c5f364768cc1
Al
01:47 AM
It has not occurred often, I don't think there is a problem to be concerned about. It did prompt me to consider monitoring them.