Validating Prometheus Configuration and Grafana JSON Support

TLDR zane.chen asked how to validate Prometheus config and sought help with a Grafana JSON template. Srikanth advised on validation, discussing Grafana version compatibility.

Photo of zane.chen
zane.chen
Thu, 23 Feb 2023 10:31:13 UTC

I have setup prometheus receiver in metrics of config file, how to validate this config is effective?

Photo of Srikanth
Srikanth
Thu, 23 Feb 2023 12:14:33 UTC

You can go to dashboards panel and search for the metric name you would expect after successful scape.

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:14:14 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:15:26 UTC

please help confirm the config that weather it‘s ok

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:16:17 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:16:42 UTC

is it effective?

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:29:32 UTC

do you have common json file? can display metrics data like grafana

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:29:37 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 01:30:01 UTC

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 01:58:31 UTC

What metrics do you expect to be seen?

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:09:45 UTC

prometheus metrics,now I have added prometheus, but i need json template to display metrics data

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:10:29 UTC

I am not sure I know what _json template to display metrics data mean_

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:12:14 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:12:34 UTC

just like the grafana json file import panel

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:12:57 UTC

There is a import grafana json option in SigNoz dashboards

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:15:06 UTC

yes, do you have this grafana json file in signoz?

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:15:34 UTC

Yes, there is a feature to import grafana json

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:23:44 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:24:53 UTC

but i load my prepared grafana json file, appears above error

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:25:43 UTC

You need to make sure the imported dashboard doesn’t have an invalid expression.

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:26:52 UTC

my imported json file is standard grafana dashboard json file, no special expression

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:27:14 UTC

Can you open the panel in edit mode and share the query

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:27:54 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:29:03 UTC

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:29:25 UTC

The `$interval` is not supported

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:33:07 UTC

ok, do you give me a usable grafana json file, let me try?

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:41:26 UTC

I will share

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:42:56 UTC

thank you very much!

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:48:10 UTC

I create one simple grafana dashboard from prometheus data, then import the json file in signoz, but unfortunately, it cannot expect to display, so i suppose maybe not strong support to grafana dashboard function in signoz

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:48:23 UTC

{ "annotations": { "list": [ { "builtIn": 1, "datasource": "-- Grafana --", "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "target": { "limit": 100, "matchAny": false, "tags": [], "type": "dashboard" }, "type": "dashboard" } ] }, "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 0, "id": 16, "links": [], "liveNow": false, "panels": [ { "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 9, "w": 12, "x": 0, "y": 0 }, "id": 2, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom" }, "tooltip": { "mode": "single", "sort": "none" } }, "targets": [ { "datasource": { "type": "prometheus", "uid": "X6_V2pO4z" }, "exemplar": true, "expr": "prometheus_engine_queries", "interval": "", "legendFormat": "", "refId": "A" }, { "datasource": { "type": "prometheus", "uid": "X6_V2pO4z" }, "exemplar": true, "expr": "prometheus_http_requests_total", "hide": false, "interval": "", "legendFormat": "", "refId": "B" } ], "title": "prometheus监控测试", "type": "timeseries" } ], "schemaVersion": 35, "style": "dark", "tags": [], "templating": { "list": [] }, "time": { "from": "now-6h", "to": "now" }, "timepicker": {}, "timezone": "", "title": "prometheus监控测试", "uid": "QWjgtZbVz", "version": 1, "weekStart": "" }

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:48:47 UTC

the above is simple grafana dashbard json file

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:49:23 UTC

Right, no strong support but something that most helps. What does the above show in dashboards UI?

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:50:13 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:50:22 UTC

empty panel

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:51:09 UTC

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:51:17 UTC

The grafana JSON is very different b/w version to version so there is a chance the current logic doesn’t detect this version.

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:51:28 UTC

this is show in grafana ui

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:53:10 UTC

do you mean signoz cannot support my grafana version?

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:54:55 UTC

The Grafana JSON has a version associated with it which we may not support yet

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:56:21 UTC

could you tell me the gafana version you have supported?

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:57:00 UTC

There is no definitive list. There is some heuristic code which we tested on some major versions.

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 02:58:57 UTC

got it, do you give me a usable grafana json file, let me try?

Photo of Srikanth
Srikanth
Fri, 24 Feb 2023 02:59:25 UTC

Sure, I will get back to you on this.

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 03:00:10 UTC

expect for your replay :blush:

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 03:01:03 UTC

I have another question about logs

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 03:01:27 UTC

maybe signoz cannot support application inside logs?

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 03:03:46 UTC

I can only find some http request record in signoz, just like below

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 03:03:55 UTC

Photo of zane.chen
zane.chen
Fri, 24 Feb 2023 03:04:48 UTC

2、and can go-zero framework be supported by signoz? my go application used go-zero framework not gin