#support

Exporting Logs from Fargate to SigNoz

TLDR Pranay needed a way to export logs from Fargate services to SigNoz. Kshitij suggested using AWS Firelens and a custom Fluent Bit image to forward logs to the SigNoz OTel collector service.

7mo

Join the chat

Mar 02, 2023 (7 months ago)

Photo of md5-aa606316d920ca1d24aa3bb0b85792d3

Pranay

06:26 AM

Hey everyone,

I have 5 services built with ExpressJS running out of Amazon ECS in a Fargate cluster
Is there anyway to export logs from those services directly to SigNoz, since it is fargate and we technically don’t have access to the “host” I figured there is no way for the same since log collectors generally scrape logs from the docker logs of a container

Any ideas on how this could be done?

Photo of md5-6478aa892c53d805bb90275a91b561c9

Kshitij

07:03 AM

We collected logs from fargate using aws firelens, where we used a custom fluent bit image to forward logs to signoz otel collector service which is also hosted on our fargate cluster....if you are using copilot to deploy you can just configure the image in the manifest file of the node.js service

Image 1 for We collected logs from fargate using aws firelens, where we used a custom fluent bit image to forward logs to signoz otel collector service which is also hosted on our fargate cluster....if you are using copilot to deploy you can just configure the image in the manifest file of the node.js service

Image 2 for We collected logs from fargate using aws firelens, where we used a custom fluent bit image to forward logs to signoz otel collector service which is also hosted on our fargate cluster....if you are using copilot to deploy you can just configure the image in the manifest file of the node.js service

Image 3 for We collected logs from fargate using aws firelens, where we used a custom fluent bit image to forward logs to signoz otel collector service which is also hosted on our fargate cluster....if you are using copilot to deploy you can just configure the image in the manifest file of the node.js service

07:12

Kshitij

07:12 AM

Also you have to configure the otel metrics collector to accept logs from there

Pranay

07:15 AM

That sounds like a doable approach, I’m gonna try that out
Thanks a ton!!

10:00

Pranay

10:00 AM

Kshitij what do you think about writing all express logs to a file, mounting that file as a volume and using a local otel-collector as a sidecar to send that file further to signoz

Kshitij

10:18 AM

I am using aws copilot cli to deploy a go application so adding a fluentbit log forwarder seemed much simpler..just had to add two lines to the config ....so I did not try the file mounting method

Pranay

10:25 AM

Oh makes sense alright

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 825 threads (61% resolved)

Join Our Community Manage subscription

Similar Threads

Troubleshooting SigNoz Dashboard Visibility Issue

Naman had an issue with SigNoz not displaying services even though they've enabled it and sent data successfully. Srikanth engaged in investigation and troubleshooting, but the issue remained unresolved.

1mo

Troubleshooting and Adding Log Files to SigNoz POC

Noor has requested help with incorporating log files into their SigNoz POC. In collaboration with vishal-signoz and nitya-signoz, they managed to successfully setup and resolve their issues.

115

1mo

Syncing Node JS Application Logs with Signoz

Romil asks for a way to sync Node JS application logs without storing them in a file. vishal-signoz suggests the experimental OTLP logs exporter for JS.

3mo