#general

Missing Metrics in Springboot Micro-Service Instrumentation

TLDR Jose reports missing metrics in a springboot micro-service instrumentation. Srikanth and Ankit suggest checking the metrics collection interval and updating the PromQL query.

Powered by Struct AI
Mar 06, 2023 (6 months ago)
Jose
Photo of md5-827f15e2b6bae6a945bc10a5d44205a6
Jose
11:57 PM
Hi Team, I'm having a particular issue with a micro-service (springboot) that was instrumented (automatic instrumentation) and some metrics are not being seen. If I check /actuator/prometheus I see:

http_server_requests_seconds_count{exception="None",method="PUT",outcome="SUCCESS",status="200",uri="/api/v1/payment-orders/{id}/cancel",} 12.0
http_server_requests_seconds_sum{exception="None",method="PUT",outcome="SUCCESS",status="200",uri="/api/v1/payment-orders/{id}/cancel",} 0.919633034

But I I try to run PromQL sum(rate(http_server_requests_seconds_count{uri=~"{{.uri}}"}[1m])) I don't get any data.

I've instrumented other services the same way and I'm getting that metric in particular.

Also I'd like to mention that I'm getting traces from that service
Mar 07, 2023 (6 months ago)
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
01:49 AM
What is the metrics collection interval? The reason could be your [1m] in the query. If they are collected at same or higher interval then you will not see any data.
Jose
Photo of md5-827f15e2b6bae6a945bc10a5d44205a6
Jose
01:59 AM
I've tried with even 1 day
01:59
Jose
01:59 AM
and no results
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
02:18 AM
Make sure your uri is non null value that returns the results.
02:18
Srikanth
02:18 AM
> I’ve tried with even 1 day
I day in the [1m] here?
Ankit
Photo of md5-dbe7088320fe1d922707613e02f3420d
Ankit
10:34 AM
Srikanth will the actuator metrics be automatically captured? Or do we need to enable via receiver in opentelemetry-collector-metrics?
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
11:29 AM
No, they are not automatically captured, but OP mentioned some metrics are not being seen, not that nothing is working. And it is more likely the [1m] selection in the promQL. If the metrics are collected every 60s or higher, then promql won’t return data. It has to be something higher. You can verify this by running sum(rate(signoz_calls_total{}[1m])) vs sum(rate(signoz_calls_total{}[2m])) in testbed.
Ankit
Photo of md5-dbe7088320fe1d922707613e02f3420d
Ankit
11:40 AM
yeah...I remember Jose can you check by changing [1m] to [2m] in promql?