Log Monitoring Connection Error in Signoz
TLDR Utkarsh encountered a connection error while adding log monitoring using Signoz. nitya-signoz identified the incorrect otlp exporter port and suggested using 4317.
1
Mar 16, 2023 (9 months ago)
Utkarsh
08:50 AMWe are looking to add log monitoring to our product and following this doc----->https://signoz.io/docs/userguide/collect_logs_from_file/
In our case signoz is running on a different host.
But after mounting our logs file, we are getting this error in out docker logs---->
Err: connection error: desc = "transport: Error while dialing dial tcp: address tcp/8080/: unknown port" {"grpc_log": true}
2023-03-16T08:48:25.681Z info fileconsumer/file.go:178 Started watching file {"kind": "receiver", "name": "filelog", "pipeline": "logs", "component": "fileconsumer", "path": "/home/ubuntu/.pm2/logs/DISHA-PROD-out.log"}
2023-03-16T08:48:25.682Z error helper/transformer.go:110 Failed to process entry {"kind": "receiver", "name": "filelog", "pipeline": "logs", "operator_id": "json_parser", "operator_type": "json_parser", "error": "ReadMapCB: expect { or n, but found 2, error found in #1 byte of ...|2023-03-15 |..., bigger context ...|2023-03-15 12:37:31.260 +00:00: socket sever starte|...", "action": "send", "entry": {"observed_timestamp":"2023-03-16T08:48:25.682141943Z","timestamp":"0001-01-01T00:00:00Z","body":"2023-03-15 12:37:31.260 +00:00: socket sever started on PORT 4000","attributes":{"log.file.name":"DISHA-PROD-out.log"},"severity":0,"scope_name":""}}
Would really appreciate some help with this.
vishal-signoz
09:42 AMnitya-signoz
09:45 AM1. There is some connection error.
2. There is a parsing error in
json_parser
can you share your config file and some sample logs which you are parsing using that config file.
Pranay
11:54 AM1
Utkarsh
12:09 PMGod rid of the parsing error but still facing connection error.
led to connect to {
"Addr": "localhost:8080/",
"ServerName": "localhost:8080/",
"Attributes": null,
"BalancerAttributes": null,
"Type": 0,
"Metadata": null
}. Err: connection error: desc = "transport: Error while dialing dial tcp: address tcp/8080/: unknown port" {"grpc_log": true}
2023-03-16T12:07:39.656Z info fileconsumer/file.go:178 Started watching file {"kind": "receiver", "name": "filelog", "pipeline": "logs", "component": "fileconsumer", "path": "/home/ubuntu/.pm2/logs/DISHA-PROD-out.log"}
Did anyone come across this before?
nitya-signoz
12:18 PMUtkarsh
12:58 PMreceivers:
filelog:
include: [ /home/ubuntu/.pm2/logs/DISHA-PROD-out.log ]
start_at: beginning
processors:
batch:
send_batch_size: 10000
send_batch_max_size: 11000
timeout: 10s
exporters:
otlp/log:
endpoint:
tls:
insecure: true
service:
pipelines:
logs:
receivers: [filelog]
processors: [batch]
exporters: [ otlp/log ]
Path----->/signoz/deploy/docker/clickhouse-setup.
Our log file is at this path----->/home/ubuntu/.pm2/logs/DISHA-PROD-out.log
We are using this command to mount it ----->
docker run -d --name signoz-host-otel-collector --user root -v /home/ubuntu/.pm2/logs/DISHA-PROD-out.log:/home/ubuntu/.pm2/logs/DISHA-PROD-out.log:ro -v $(pwd)/otel-collector-config.yaml:/etc/otel/config.yaml signoz/signoz-otel-collector:0.55.0-rc.3
nitya-signoz
04:10 PMSigNoz Community
Indexed 1023 threads (61% resolved)
Similar Threads
Troubleshooting and Adding Log Files to SigNoz POC
Noor has requested help with incorporating log files into their SigNoz POC. In collaboration with vishal-signoz and nitya-signoz, they managed to successfully setup and resolve their issues.
Removing Default Logs and Collecting Application Logs
Srikanth wanted to remove default logs and collect their application logs. nitya-signoz provided two solutions, and Srikanth chose the second one, which involves editing the ote-collector-config.yaml file.
Data Not Visible on Dashboard, Collector Container Restarting
Div reported data not visible on dashboard after updating app. Srikanth identified ClickHouse having issues with Zookeeper instance, causing collector container to restart. Reinstallation solved the issue but led to data loss.
Troubleshooting SigNoz Auto-Instrumentation Configuration
igor is having trouble configuring auto-instrumentation for Java applications using SigNoz, with traces not appearing in the SigNoz UI. Prashant advises to check logs of the otel sidecar, use service name for endpoint, verify supported libraries, and test with telemetrygen. However, the issue still persists.
Connection Issues with signoz-otel-collector on k8s Cluster
Ankit encounters connection issues with signoz-otel-collector on a k8s cluster. Prashant suggests checking logs and status, but the issue remains unresolved.