#support

Running SigNoz with External ClickHouse Without Zookeeper

TLDR Dave asked how to run SigNoz without ClickHouse in cluster mode with Zookeeper. Srikanth explained the need for Zookeeper or ClickHouse Keeper. Dave found a solution by enabling the ClickHouse Keeper configuration within ClickHouse.

9mo

Join the chat

Mar 03, 2023 (9 months ago)

Photo of md5-9da54779b3d266fa72edda9b8a8ae481

Dave

06:38 PM

Is there any way to run Signoz without ClickHouse in cluster mode with Zookeeper?

Photo of md5-8df7ce0274b2473ec07403336e48b574

Pranay

06:46 PM

do you want to add. your own clickhouse Dave?

Dave

06:47 PM

We have set up a standalone ClickHouse.

Pranay

06:47 PM

Got it. This might be helpful - https://signoz.io/docs/operate/clickhouse/connect-to-clickhouse/

Dave

06:47 PM

We aren't used K8s. We are setting this up in ECS/Fargate, so we are bringing up all of the individual elements.

06:49

Dave

06:49 PM

It seems the SigNoz otel-collector must have a cluster called "cluster" in ClickHouse. Is this right? But then does this make Zookeeper absolutely required?

06:49

Dave

06:49 PM

It doesn't seem like ClickHouse explicitly requires ZK.. so I don't want to add any unnecessary dependencies.

Photo of md5-ce04a9988e2fd758a659dc55be6f2543

Srikanth

06:51 PM

> It seems the SigNoz otel-collector must have a cluster called “cluster” in ClickHouse
Yes

06:52

Srikanth

06:52 PM

> It doesn’t seem like ClickHouse explicitly requires ZK.. so I don’t want to add any unnecessary dependencies.
If you want to use external clickhouse you should be able to do it. Let me share the docs for it.

Pranay

06:52 PM

I have already shared Srikanth - https://signoz.io/docs/operate/clickhouse/connect-to-clickhouse/

Dave

06:53 PM

Pranay This isn't the correct doc. This is just how to connect to ClickHouse.

Pranay

06:54 PM

Ok, will let Srikanth get back

Dave

06:55 PM

In your docker-compose example, you have something like this:

<?xml version="1.0"?>
<clickhouse>
    <remote_servers>
        <cluster>
            <shard>
                <replica>
                    <host>localhost</host>
                    <port>9000</port>
                </replica>
            </shard>
        </cluster>
    </remote_servers>
</clickhouse>

06:56

Dave

06:56 PM

This makes "cluster" in ClickHouse... but then, I believe ZK is required?

Srikanth

07:07 PM

ZK that comes with SigNoz installation is not required if you want to use external ClickHouse. It’s main purpose is to provide coordination system for distributed ClickHouse. But if you want to use SigNoz there should be cluster called cluster so you may already have zk or clickhouse keeper on your ClickHouse server.

Dave

07:08 PM

So I must have ZK or Keeper to back my ClickHouse.

Srikanth

07:09 PM

Yes, the distributed ClickHouse setup requires zookeeper or clickhouse-keeper.

Dave

07:10 PM

Ok. And since you require "cluster", I have to have a distributed ClickHouse, even though I only have 1 ClickHouse.

Srikanth

07:17 PM

> Let me share the docs for it.
Unfortunately, there is no written documentation around this. You can disable the ClickHouse and setup the appropriate values for externalClickHouse. And use the override-values to apply the custom config.

Dave

08:46 PM

The easy solution for this was to enable the ClickHouse Keeper configuration within ClickHouse.

08:46

Dave

08:46 PM

It starts it as a part of the service, and then I don't need an external ZK

Mar 04, 2023 (9 months ago)

Pranay

03:25 AM

Nice, were you able to make this work for your setup? Dave

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 1023 threads (61% resolved)

Join Our Community

Similar Threads

External ClickHouse Server Installation Issue

Syed faced a connection error with an external ClickHouse server. Srikanth and Ankit suggested changing the cluster name to 'cluster'. The issue was resolved after updating the name.

8mo

Issues Running SigNoz on Azure AKS with External ClickHouse DB

Nicolai faced an error while trying to run SigNoz with an external ClickHouse database on Azure AKS. Prashant suggested additional steps and the need for detailed documentation for using SigNoz with an external ClickHouse database.

9mo

Issues with SigNoz Setup and Data Persistence in AKS

Vaibhavi experienced issues setting up SigNoz in AKS, and faced data persistence issues after installation. Srikanth provided guidance on ClickHouse version compatibility and resource requirements, helping Vaibhavi troubleshoot and resolve the issue.

8mo

SigNoz crashing in k8s due to ClickHouse OOM

Travis reported SigNoz crashing in k8s due to ClickHouse OOM. The team suggested increasing resources for ClickHouse, and other troubleshooting steps, but the issue remains unresolved.

8mo

Troubleshooting SigNoz Local Setup on M1 Mac

Sindhu had issues with their local SigNoz installation using Docker on an M1 Mac. Srikanth helped diagnose the problem. It was resolved by manually starting the ClickHouse container and then starting SigNoz via Docker Compose in the terminal.

8mo