#general

Deploying otel+signoz on EC2: Recommended Architecture Pattern

TLDR Agustin sought advice on deploying otel+signoz on EC2. Both Srikanth and Alexei suggested option 1, with Alexei also referencing otel docs for further information on deployment before Agustin closed the thread.

3mo

Join the chat

Aug 10, 2023 (3 months ago)

Photo of md5-1e9aee66f7b4d965681954d12addd998

Agustin

03:47 PM

H! i’m trying to deploy otel+signoz on a large fleet of EC2, managed by multiple autoscaling groups. What’s the recommend architecture pattern for it?
1. Each EC2 instance has an otel collector that fowards the data to signoz otel-collector
2. Each EC2 instance has an otel collector that fowards the data directly to the clickhouse DB
3. The signoz otel-collector scraps the data from each EC2 instance’s otel collector
4. The signoz otel-collector scraps the data directly form eachh EC2 instance receiver (prometheus, jaeger, etc)
Im guessing 3 and 4 would require some sort of service discovery

Photo of md5-ce04a9988e2fd758a659dc55be6f2543

Srikanth

03:59 PM

I would suggest going with option one.

Photo of md5-d44b8ad79c1dd6595985e83066c4506b

Alexei

06:41 PM

Option 1 is how I do it in production with billions of rows in clickhouse and scales well. Can run the otel-collector as a daemon service if using ECS

06:44

Alexei

06:44 PM

More docs on the OTEL site here around deployment where you can gauge the tradeoffs on how many layers of OTEL collectors you want before reaching your “backend” storage for the metrics/traces/logs: https://opentelemetry.io/docs/collector/deployment/

Agustin

06:51 PM

thanks folks!

06:52

Agustin

06:52 PM

i will take a look at those docs

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 1023 threads (61% resolved)

Join Our Community

Similar Threads

Integrating Otel Collector Agent with Signoz on ECS

Div asks how to integrate an otel collector agent for their microservices on ECS with Signoz. Srikanth suggests running it as a separate container or a background process, and shares a relevant document.

9mo

SigNoz Deployment and Collector Concerns in Multi-Cluster Application

Romil asks about deploying k8s-infra chart and collector roles in a multi-cluster setup. Ankit recommends deploying k8s-infra chart for each cluster, and mentions the importance of using SigNoz's otel-collector for their data schema.

9mo

Configuring OTel Collector to Export Data to SigNoz

James asked how to configure the otel collector to send data to SigNoz. Ankit suggested using the otlp exporter in the collector configuration. James found that SigNoz uses its own otel collector and adjusted the port bindings accordingly.

5mo