#support

Scale and Optimization Problem in the Production Move of Signoz

TLDR Simran had problems with scaling Signoz and tried a distributed shards setup with multiple replicas, but faced issues. Prashant confirmed that multiple replicas are not supported and suggested multiple shards of ClickHouse and multiple replicas of SigNoz Otel-Collector as a solution.

Powered by Struct AI
thankyou

1

1

3
2mo
Solved
Join the chat
Sep 09, 2023 (2 months ago)
Simran
Photo of md5-75fcf9dbcf7e4bda919dd9b8c66c8256
Simran
10:40 AM
#support Recently we moved Signoz to production but faced issue with scale. Hence we tried to moved to distributed shards setup with 2 replicas and 3 shards. But I can see multiple replicas with shards is not supported as mention here - https://signoz.io/docs/operate/clickhouse/distributed-clickhouse/

Wanted to check if this is same with the current version of Signoz as well ?



Plus any leads towards optimizing Signoz's clickhouse for arounds 600k - 1millions span per minute setup will be helpfull. Based on what we are seeing, signoz collector is facing throttle to write to clickhouse.

1

Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:36 PM
As mentioned in the shared docs, multiple replicas of ClickHouse cluster are not supported. But only multiple shards.
06:37
Prashant
06:37 PM
Adding multiple shard of ClickHouse (spread across multiple nodes) and multiple replicas of SigNoz Otel-Collector should help to handle high load.
thankyou

1