#support

Hosting ClickHouse Cluster with Three Shards

TLDR Kshitij asked how to host a ClickHouse cluster with three shards. nitya-signoz explained Otel collector doesn't need to know about shards. Migration process requires manual env variable change for Docker.

7mo

Join the chat

Mar 01, 2023 (7 months ago)

Photo of md5-6478aa892c53d805bb90275a91b561c9

Kshitij

05:26 AM

If I am hosting my own clickhouse cluster with three shards how should I mention the url of my three shards https://signoz.io/docs/operate/clickhouse/distributed-clickhouse/, in this doc we are not really changing the clickhouse url in otel-collector-config file...so does signoz automatically distribute data between different nodes/shards....should I connect a load balancer between the clickhouse instances and other services to distribute the load?(also is it possible to run migrations without toggling the setting in otel collector service env DOCKER_MULTI_NODE_CLUSTER=false)

Photo of md5-a52b9d6c34f193d9a1ff940024f36f77

nitya-signoz

05:33 AM

Otel collector doesn’t need to know about the shards. It’s taken care of by clickhouse using zookeeper.

Kshitij

05:39 AM

Thank you...so about the migration process in case of shards so is there a way to do it without manually changing the env variable?

nitya-signoz

05:41 AM

Don’t think there is another way of doing it. vishal-signoz any thoughts on the above?

Btw why do you want to avoid changing the env variable ?

Photo of md5-f936d3e5743d23344d6c60813189716f

vishal-signoz

05:42 AM

In case you are using docker, you have to follow env change process.

Kshitij

05:44 AM

I am hosting all the services on aws ecs fargate separately...so it would be really weird to redeploy a service after changing env variable ....but i guess it will only need to be done once at the start of deployment?

Mar 02, 2023 (7 months ago)

nitya-signoz

03:59 PM

Yes, it runs when the otel collector starts. If the migrations are applied already then it is skipped.

The case in which you will need migrations is when upgrading signoz.

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 825 threads (61% resolved)

Join Our Community Manage subscription

Similar Threads

Problems Encountered with ClickHouse After Fresh Install

Romil experienced issues after a fresh install of ClickHouse. Prashant advised restarting certain pods, but that wasn't fully successful. Srikanth acknowledged the issue and pointed to a related open problem on GitHub.

1mo

Issues with Multicluster Functionality in a Clickhouse Setup

Hoang has trouble implementing multi-cluster functionality in a Clickhouse setup as the collector does not read `CLICKHOUSE_CLUSTER`. Shivanshu suggests using `cluster` for the cluster name, while Prashant refers to the relevant documentation.

1mo

Scale and Optimization Problem in the Production Move of Signoz

Simran had problems with scaling Signoz and tried a distributed shards setup with multiple replicas, but faced issues. Prashant confirmed that multiple replicas are not supported and suggested multiple shards of ClickHouse and multiple replicas of SigNoz Otel-Collector as a solution.