Comparing SigNoz Cloud Performance at Scale with Other Log and Metric Tools
TLDR Christian asked about SigNoz Cloud's scalability, comparing it with Humio, victoriametrics, and Loki. Nocnica and Ankit assured it scales easily and offers solutions like bloom filters and inverted indexes.
Nocnica
Tue, 24 Oct 2023 17:07:10 UTCI'm going to look with the team at what's the max scale we're being used at Christian, will look at what data we can share.
In general, SigNoz is using Clickhouse as its datastore, they've written a bit about
Ankit
Wed, 25 Oct 2023 12:18:47 UTCHi Christian , it is difficult to do exact comparison but we have tested with 5TB/day and 1M active timeseries per minute.
Ankit
Wed, 25 Oct 2023 12:19:07 UTCIt should definitely easily scale upto 5 times more
Christian
Wed, 25 Oct 2023 13:52:58 UTChave you done something like search/query for a random string (like `foo`) on the entire data (and not just a pre-filtered dataset or specific column) of the last 2 days - which would, in the worst case - scan about 10TB of data? I know that things like that are very problematic in loki and some other systems (in loki this would almost certainly time out), but in humio they optimize queries like these via bloom filters.
Christian
Wed, 25 Oct 2023 13:56:09 UTCBtw, you don't have by chance a way to ingest log data from
Ankit
Mon, 30 Oct 2023 08:25:04 UTCbloom filters are already in place.
Ankit
Mon, 30 Oct 2023 08:25:32 UTCYou can also use inverted index (still experimental in clickhouse)
SigNoz Community
Indexed 1061 threads
Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI
View in App
Christian
Mon, 23 Oct 2023 01:10:38 UTCHey, does anyone have any experience to share how signoz cloud performs at scale? For context, we're currently using Humio for logs (3TB ingest/day) and victoriametrics for metrics (don't remember the scale out of my head, but will look it up later). We tried out Loki and it really didn't work for us - full text queries that take single digit seconds in humio would be very slow or time out in Loki.