#support

Clickhouse Query Timeout in SigNoz Dashboard

TLDR Daniel experienced query timeouts in SigNoz dashboard. vishal-signoz confirmed it's a non-configurable 60s limit, but created a GitHub issue for adding the feature.

5mo

Join the chat

Apr 17, 2023 (5 months ago)

Photo of md5-3076442c4997c13ee86556ecd18af2a8

Daniel

11:50 AM

Hi guys, I was wondering if this is a signoz issue or a clickhouse related issue. When I run clickhouse queries in the signoz dashboard editor, it always times out after 1 minute (and thus fails, queries faster than 1 minute do not fail). So is this something that signoz does not support? Or is this a clickhouse setting that I need to change? Any help would be greatly appreciated.

Photo of md5-dbe7088320fe1d922707613e02f3420d

Ankit

11:59 AM

I think this is a clickhouse setting we enabled. Do you really need your queries to be running for more than 1 min? cc vishal-signoz

Daniel

12:01 PM

Yes, we have a lot of data we are querying over. Is this a setting I could configure?

Ankit

12:10 PM

> Is this a setting I could configure?
I will let vishal-signoz reply on this.

> we have a lot of data we are querying over.
Is this a standard APM query or some other use case? Can you share a sample query (in simple English or SQL)? We might have ways to make it performant.

Daniel

12:19 PM

SELECT quantile(0.99)(durationNano) as p99,
name,
toDateTime(toStartOfMinute(timestamp)) AS ts
FROM
()
WHERE( timestamp >= start_t
AND timestamp <= end_t
GROUP BY name,ts
ORDER BY p99 DESC

this is the basic query, it contains some more filtering to make it specific for a cluster and remove some unwanted data

Photo of md5-f936d3e5743d23344d6c60813189716f

vishal-signoz

12:28 PM

Daniel We have set default query timeout to 60s: https://github.com/SigNoz/signoz/blob/0bc44c6fd986ecde0513a153dd37a5baf4a7b339/pkg/query-service/constants/constants.go#L100
Currently it’s not configurable.
Created an issue to allow it to be configurable via env variable, Track here: https://github.com/SigNoz/signoz/issues/2582

Daniel

12:30 PM

Thanks for the help! I'll keep an eye on the progress.

Ankit

12:34 PM

> this is the basic query, it contains some more filtering to make it specific for a cluster and remove some unwanted data
do you have serviceName in the filters? The query will be faster with ServiceName.

Daniel

12:34 PM

yeah i do

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 826 threads (62% resolved)

Join Our Community Manage subscription

Similar Threads

Dashboard Load Issues and Possible Solutions

Al experiences dashboard loading issues since updating to `0.18.1`. Srikanth believes the issue is not version related and suggests examining queries, memory resources, and server distribution for improvements.

5mo

Issues with SigNoz Setup and Data Persistence in AKS

Vaibhavi experienced issues setting up SigNoz in AKS, and faced data persistence issues after installation. Srikanth provided guidance on ClickHouse version compatibility and resource requirements, helping Vaibhavi troubleshoot and resolve the issue.

5mo

Incorrect Alert Firing Issue in Threshold Condition

Harshith.R.S is experiencing an issue with alerts firing when the value is above the threshold. The issue remains unresolved after trying different query modifications suggested by Amol.

7mo

Debugging and Optimizing Slow Signoz Dashboard

Div reported slow Signoz dashboard and errors. Srikanth suggested creating a materialized column and adding filters to speed up queries. vishal-signoz confirmed the suggested approach.

5mo

Adjusting Dashboard Details with Clickhouse Query and Time Dropdown

Harshith.R.S asked how to change dashboard details with clickhouse query based on time dropdown. Srikanth provided a solution using time template variables in the query.

7mo