Issues with Fluent-bit JSON parsing in SigNoz Log Viewer
TLDR Ulisses is having problem with fluent-bit sending logs to SigNoz. nitya-signoz advises using the json parser operator, and provides an example solution.
Prashant
Thu, 21 Sep 2023 19:06:52 UTCcc nitya-signoz
nitya-signoz
Fri, 22 Sep 2023 04:42:57 UTCPlease use the json parser operator
Ulisses
Fri, 22 Sep 2023 12:25:51 UTCThe config bellow results in error: ```processors: logstransform: operators: - type: trace_parser trace_id: parse_from: attributes.trace_id span_id: parse_from: attributes.span_id - type: json_parser parse_from: other_info - type: remove field: attributes.trace_id - type: remove field: attributes.span_id``` Error: ```signoz-otel-collector | Error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding: signoz-otel-collector | signoz-otel-collector | * error decoding 'processors': error reading configuration for "logstransform": 1 error(s) decoding: signoz-otel-collector | signoz-otel-collector | * error decoding 'operators[1]': unmarshal to json_parser: 1 error(s) decoding: signoz-otel-collector | signoz-otel-collector | * error decoding 'parse_from': unrecognized prefix```
Ulisses
Fri, 22 Sep 2023 12:36:55 UTCI changed the app logger to log the field "other_info" as a json instead of a formatted json string and removed the json_parser. The field is show in the logs explorer as a json formated string: other_info [Type:Tag] [Data type: String] "{\"a\":\"123\",\"b\":\"456\",\"c\":\"789\",\"d\":\"987\",\"e\":\"654\",\"f\":\"321\"}" Added the json_parser againg to the logstransformer: ```- type: json_parser parse_from: attributes.other_info``` Results in: ```signoz-otel-collector | 2023-09-22T12:32:36.175Z error helper/transformer.go:99 Failed to process entry { "kind": "processor", "name": "logstransform", "pipeline": "logs", "operator_id": "json_parser", "operator_type": "json_parser", "error": "type map[string]interface {} cannot be parsed as JSON", "action": "send", "entry": { ...```
Ulisses
Fri, 22 Sep 2023 12:49:38 UTCfull error from signoz-otel-collector container:
```signoz-otel-collector | 2023-09-22T12:43:01.273Z error helper/transformer.go:99 Failed to process entry {"kind": "processor", "name": "logstransform", "pipeline": "logs", "operator_id": "json_parser", "operator_type": "json_parser", "error": "type map[string]interface {} cannot be parsed as JSON", "action": "send", "entry": {"observed_timestamp": "2023-09-22T12:43:01.173937618Z", "timestamp": "2023-09-22T12:43:01.087Z", "body": "ERROR POST /some/url/test", "attributes": {"app_module": "module_a", "duration": "10222951", "ecs.version": "1.2.0", "error_message": "Some error message", "error_type": "some.java.ExceptionClass", "event.dataset": "the_app", "fluent.tag": "...", "other_info": {"a": "123", "b": "456", "c": "789", "d": "987", "e": "654", "f": "321"}, "http_request_method": "POST", "http_url_path": "/some/url/test", "log.level": "INFO", "log.logger": "json.logger", "process.thread.name": "http-nio-8080-exec-3", "service.name": "the_app", "status_code": "ERROR", "trace_flags": "01", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36", "user_id": "kj4l5j23lç4j62jlkj5lk", "user_ip_address": "0:0:0:0:0:0:0:1", "uuid": "a66e242b-95a3-484b-b9c8-8d730289baf6"}, "severity": 0, "scope_name": ""}}
signoz-otel-collector |
nitya-signoz
Sat, 23 Sep 2023 04:39:56 UTCYou will have to pass the entire body to json parser and the move it, here is an example
SigNoz Community
Indexed 1060 threads
Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI
View in App
Ulisses
Thu, 21 Sep 2023 18:50:18 UTCHello, i'm using fluent-bit to send logs to signoz. The log is in the json format and this json contains an attribute named "other_info" that contains an escaped json string. I need to convert this "other_info" attribute to appear structured in the SigNoz log viewer. Where else should I configure besides fluentbit? log example: ```{"@timestamp":"2023-09-21T15:38:31.950Z","log.level": "INFO","message":"OK GET /mnt/index.do", "ecs.version": "1.2.0","service.name":"the_app","event.dataset":"the_app","process.thread.name":"http-nio-8080-exec-4","log.logger":"the_app.logger","app_module":"app_module","duration":"1442235497","http_request_method":"GET","http_url_path":"/mnt/index.do","user_id":"123456","span_id":"b6f1a671a3b298fe","trace_flags":"01","trace_id":"38a8eb9837cce5bb0e7f9276018b1985", "other_info": "{\"a\":\"123\",\"b\":\"456\",\"c\":\"789\",\"d\":\"987\",\"e\":\"654\",\"f\":\"321\"}"}``` fluent-bit config: ```[INPUT] Name tail Tag log4j2.ecs.* Path /path/to/logs/app-json.log Parser log4j2.ecs.json.parser [OUTPUT] Name forward Match * Host localhost Port 24224``` parser config ```[PARSER] Name log4j2.ecs.json.parser Format json Time_Key @timestamp Time_Format %Y-%m-%dT%H:%M:%S.%LZ Decode_Field_As escaped other_info do_next Decode_Field_As json other_info```