Troubleshooting K8s Deployment and Suggested Fields
TLDR saajid had issues with k8s deployment suggested fields. nitya-signoz suggested redeploying and shared troubleshooting steps.
Mar 08, 2023 (6 months ago)
The docs state that the k8s deployment automatically parses container_name and other relevant k8 fields, and I can see them within the log record. Do I have to do another step to make it a "suggested field" or should I be able to query it as is.
Side Question: Are there any k8 helm examples for modifying the collector for json parsing specifically?
Thanks in advance.
Mar 09, 2023 (6 months ago)
Are your logs getting ingested? also where is your k8s cluster running can you provide some more info.
resources_string.k8s_container_namein the log record. I'm running on eks on aws.
can you share the output of the following commands.
kubectl exec -n platform -it chi-my-release-clickhouse-cluster-0-0-0 -- sh clickhouse client use signoz_logs; select distinct name from distributed_logs_resource_keys;
SELECT DISTINCT name FROM distributed_logs_resource_keys Query id: 02a3aa7e-bcc4-4b13-a527-0223530f3dd4 ┌─name────────────────────────┐ │ host_name │ │ os_type │ │ signoz_component │ └─────────────────────────────┘
signoz_logsdatabase and restart the collectors.
Else we can schedule a call to fix this.
Commands will be same as this https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1678376272927989?thread_ts=1678299713.440809&cid=C01HWQ1R0BC
drop database signoz_logs
Mar 10, 2023 (6 months ago)
And by broke can you share what happened exactly ?
And by broke I mean no logs after starting them back up and no errors in either the agents or the collectors.
Indexed 825 threads (61% resolved)
Issues with SigNoz Setup and Data Persistence in AKS
Vaibhavi experienced issues setting up SigNoz in AKS, and faced data persistence issues after installation. Srikanth provided guidance on ClickHouse version compatibility and resource requirements, helping Vaibhavi troubleshoot and resolve the issue.
SigNoz crashing in k8s due to ClickHouse OOM
Travis reported SigNoz crashing in k8s due to ClickHouse OOM. The team suggested increasing resources for ClickHouse, and other troubleshooting steps, but the issue remains unresolved.
Issue with Filters on Logs and Duplicate Fields
Patrick reported issues with filter functionality and duplicate fields in logs. Srikanth looked into the problem. nitya-signoz offered a solution involving an update and guide.
Dashboard Load Issues and Possible Solutions
Al experiences dashboard loading issues since updating to `0.18.1`. Srikanth believes the issue is not version related and suggests examining queries, memory resources, and server distribution for improvements.
Troubleshooting and Adding Log Files to SigNoz POC
Noor has requested help with incorporating log files into their SigNoz POC. In collaboration with vishal-signoz and nitya-signoz, they managed to successfully setup and resolve their issues.