TLDR Alex experienced filtering issues in logs view. After troubleshooting with Ankit, they discovered a bug in the otelcollector helm chart and intends to file a PR to fix it.
I'm a bit confused about the logs view. When I choose "Last 1 hour", "Last 5 minutes" in the logs view, what does that actually give me? On my installation, choosing either one gives me 3 day old logs. What does it actually filter on?
This is not much useful if you have continuous logs coming in. If you specify a filter with a few logs that came the last day only. Such dropdown will be useful.
Moreover, it is more useful when you want to view a custom time range
but what field does the filter actually apply to? Intuitively it would filter based on the timestamp.
but clearly it does not since I get logs with timestamps 3 days in the past even when I select 1 hour
I have a setup where I run vanilla otel-collector-contrib and export logs via OTLP to signoz
I didn't get you. You mean you can see logs that happened 3 days back when the selected filter is 1 hour?
It's weird. Can you share the complete API request curl from the network tab?
I pressed refresh, and now I get nothing again
> I pressed refresh, and now I get nothing again
I think this is a bug. Can you show request payload when you see last 3 days data?
I will try but it's hard to reproduce
I am struggling to get my setup working. I cannot seem to get the logs delivered to signoz reliably, and I can't even debug the problem since there is nothing output to the collector logs (not a signoz problem, but still)
you can use logging exporter at signoz to verify logs are being received by signoz
tried that. I even added an operator `-type: stdout` to the `filelog` receiver, but I still get nothing. I tried this locally and it worked fine.
I am totally at my wit's end trying to get logging to work.
I know you don't have any obligation to help me, as this is not a signoz problem per se
(and even if it was)
ok, it looks like `filelog` receiver isn't seeing any logs
this was a bug in the otelcollector helm chart. I'm going to file PRs to fix that.
for posterity, the problem was that the include patterns started with `/var/log/` when they should've been prefixed with `/hostfs/var/log` as the host file system is mounted at `/hostfs`
Indexed 1061 threads
Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI