SigNoz not Collecting Pod Logs in K8s Cluster on Bare Metal

TLDR Ayan installed SigNoz in a bare metal k8s cluster but isn't seeing logs. nitya-signoz and Prashant suggested checking paths, container runtime tool, and scaling, but issue still unresolved.

Photo of Ayan
Ayan
Sun, 04 Jun 2023 18:34:16 UTC

Hi team! I've just installed signoz in k8s cluster via helm. I can see my java app activities in services section. But there is no any logs in Logs section (No data). In documentation I've read that by default it will automatically start collecting all the pod logs. But in my case not. Any sugestions ?

Photo of nitya-signoz
nitya-signoz
Mon, 05 Jun 2023 04:11:43 UTC

Are you using EKS ?

Photo of Ayan
Ayan
Tue, 06 Jun 2023 03:38:48 UTC

Not EKS. k8s was deployed in bare metal

Photo of nitya-signoz
nitya-signoz
Tue, 06 Jun 2023 03:51:17 UTC

By default this is the path that is checked for pod logs . `/var/log/pods/*/*/*.log` Prashant any idea if there can be some path issue while running k8s on bare metal ?

Photo of Prashant
Prashant
Tue, 06 Jun 2023 14:04:50 UTC

Ayan which container runtime tool are you using for your bare metal cluster?

Photo of Prashant
Prashant
Tue, 06 Jun 2023 14:05:26 UTC

anyways, updating the above mentioned path to the logs path should do it.

Photo of Ayan
Ayan
Wed, 07 Jun 2023 04:25:10 UTC

Container runtime tool is Docker v20.10.11 Signoz installed with default parameters also with this path for logs. I checked path `/var/log/pods/*/*/*.log` in worker nodes for logs, there it's ok. Logs of pods are exists.

Photo of Ayan
Ayan
Wed, 07 Jun 2023 04:27:00 UTC

But still No Data. Should I scale to more replicas for each worker node deployment of signoz-k8s-infra-otel-deployment. To locate in each node to scrape logs maybe ?