Customizing Signoz Error Alert Descriptions
TLDR Sumanth needed help with making Signoz error alert descriptions more descriptive. Amol suggested using {{.Labels.serviceName}} and discussed limitations of adding labels in queries.
Sumanth
Mon, 26 Jun 2023 06:09:39 UTCattaching also the clickhouse query used and error alert message i have received. Would love to hear any tips or improvements i could make
```SELECT
count() as value,
toStartOfInterval(timestamp, toIntervalMinute(1)) AS interval,
serviceName
FROM signoz_traces.distributed_signoz_error_index_v2
WHERE timestamp BETWEEN {{.start_datetime}} AND {{.end_datetime}}
GROUP BY serviceName, interval;
-- available variables:
-- {{.start_datetime}}
-- {{.end_datetime}}
-- required column alias:
-- value
-- interval```
The example error alert is as follows:
```Signoz-Alerts
APP 12:06 PM
[FIRING:1] General Error Catch for (serviceName="Dev-L1Enricher", severity="error")
Alert: General Error Catch - error
Summary: The rule threshold is set to 2, and the observed metric value is 4
Description: This alert is fired when the error count (current value: 4) crosses the threshold (2)
Details:
• alertname: General Error Catch
• ruleId: 1
• ruleSource:
Pranay
Mon, 26 Jun 2023 07:38:08 UTCAmol Do you have more ideas on this?
Amol
Mon, 26 Jun 2023 08:06:59 UTCeach column in select clause other than value and timestamp is treated as label. so in your query serviceName would be available under {{.Labels.serviceName}} if you add this go template in the description, the service name would appear in place of {{.Labels.serviceName}}
Amol
Mon, 26 Jun 2023 08:09:17 UTCthe caveat is labels are also used to uniquely identify the row, so there is a limitation on what you can pick in the select query. so let's say you want to group by servicename and apply alert condition on the resulting row but you add error desc in select, the result of query would change as well so make sure your query result has same no of rows after inclusion of new labels
SigNoz Community
Indexed 1060 threads
Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI
View in App
Sumanth
Mon, 26 Jun 2023 06:05:50 UTCA bit new to signoz, unable to get answers to some questions i have over the internet. Please help if possible I want to set up alerts on general exceptions received by Signoz. Currently this is the existing alert description: ```This alert is fired when the error count (current value: {{$value}}) crosses the threshold ({{$threshold}}) ``` I want to make the description more descriptive with information on what the error is. I would like to know what all possible variables i can access in the description with ${{}}. Where can i find the list of variables?