Non-json data from serial bridge not escaped in SerialReceived string value

[abmaonline]

PROBLEM DESCRIPTION

When receiving text from a serial input using the serial bridge (not serial_bridge_raw), it is copied 1:1 into the string value of the SerialReceived property when generating the output message (the data from serial is not json itself in this case, so it doesn't begin with a {).

When the text contains a \ (or one of the other forbidden chars in json, like " or some control characters) this is not properly escaped in the resulting json message, causing invalid json that can be an issue for the consuming systems.

Sample input from serial:
/ISK5\2M550E-1011

Resulting message:
{"SerialReceived":"/ISK5\2M550E-1011"}

Should be:
{"SerialReceived":"/ISK5\\2M550E-1011"}

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

• Read the Contributing Guide and Policy and the Code of Conduct
• Searched the problem in issues
• Searched the problem in the docs
• Searched the problem in the forum
• Searched the problem in the chat
• Device used (e.g., Sonoff Basic): Wemos D1 mini v3.1.0
• Tasmota binary firmware version number used: 8.2.0.6 Lite
  • Pre-compiled
  • Self-compiled
    • IDE / Compiler used: _____
• Flashing tools used: OTA from webinterface
• Provide the output of command: Backlog Template; Module; GPIO 255:

  Configuration output here:
20:44:42 MQT: stat/p1-smart-meter/RESULT = {"GPIO0":{"0":"None"},"GPIO1":{"0":"None"},"GPIO2":{"0":"None"},"GPIO3":{"0":"None"},"GPIO4":{"0":"None"},"GPIO5":{"0":"None"},"GPIO9":{"0":"None"},"GPIO10":{"0":"None"},"GPIO12":{"0":"None"},"GPIO13":{"0":"None"},"GPIO14":{"0":"None"},"GPIO15":{"0":"None"},"GPIO16":{"0":"None"}}

• If using rules, provide the output of this command: Backlog Rule1; Rule2; Rule3:

  Rules output here:

• Provide the output of this command: Status 0:

  STATUS 0 output here:
20:47:19 MQT: stat/p1-smart-meter/STATUS = {"Status":{"Module":18,"FriendlyName":["P1 Smart Meter"],"Topic":"p1-smart-meter","ButtonTopic":"0","Power":0,"PowerOnState":3,"LedState":1,"LedMask":"FFFF","SaveData":1,"SaveState":1,"SwitchTopic":"0","SwitchMode":[0,0,0,0,0,0,0,0],"ButtonRetain":0,"SwitchRetain":0,"SensorRetain":0,"PowerRetain":0}}
20:47:19 MQT: stat/p1-smart-meter/STATUS1 = {"StatusPRM":{"Baudrate":115200,"SerialConfig":"8N1","GroupTopic":"tasmotas","OtaUrl":"https://thehackbox.org/tasmota/tasmota-lite.bin.gz","RestartReason":"Software/System restart","Uptime":"0T00:06:01","StartupUTC":"2020-05-03T19:41:18","Sleep":0,"CfgHolder":4617,"BootCount":16,"BCResetTime":"2020-04-12T12:02:53","SaveCount":62,"SaveAddress":"F9000"}}
20:47:19 MQT: stat/p1-smart-meter/STATUS2 = {"StatusFWR":{"Version":"8.2.0.6(d410420-lite)","BuildDateTime":"2020-05-03T19:00:51","Boot":31,"Core":"2_7_0","SDK":"2.2.2-dev(38a443e)","Hardware":"ESP8266EX","CR":"424/699"}}
20:47:19 MQT: stat/p1-smart-meter/STATUS3 = {"StatusLOG":{"SerialLog":0,"WebLog":2,"MqttLog":0,"SysLog":0,"LogHost":"","LogPort":514,"SSId":["***",""],"TelePeriod":300,"Resolution":"558180C0","SetOption":["00408009","2805C8000100060000005A00000000000000","00000000","00000000"]}}
20:47:19 MQT: stat/p1-smart-meter/STATUS4 = {"StatusMEM":{"ProgramSize":470,"Free":532,"Heap":26,"ProgramFlashSize":1024,"FlashSize":4096,"FlashChipId":"1640C8","FlashMode":3,"Features":["00000809","8F082783","041083A0","00000006","010001C0","00000000","00000024"],"Drivers":"1,2,3,4,9,10,16,20,21,37","Sensors":"3"}}
20:47:19 MQT: stat/p1-smart-meter/STATUS5 = {"StatusNET":{"Hostname":"p1-smart-meter-2108","IPAddress":"***","Gateway":"***","Subnetmask":"***","DNSServer":"***","Mac":"***","Webserver":2,"WifiConfig":4,"WifiPower":17.0}}
20:47:19 MQT: stat/p1-smart-meter/STATUS6 = {"StatusMQT":{"MqttHost":"***","MqttPort":1883,"MqttClientMask":"DVES_%06X","MqttClient":"DVES_DD683C","MqttUser":"DVES_USER","MqttCount":1,"MAX_PACKET_SIZE":1200,"KEEPALIVE":30}}
20:47:19 MQT: stat/p1-smart-meter/STATUS7 = {"StatusTIM":{"UTC":"2020-05-03T19:47:19","Local":"2020-05-03T20:47:19","StartDST":"2020-03-29T02:00:00","EndDST":"2020-10-25T03:00:00","Timezone":"+01:00","Sunrise":"05:25","Sunset":"20:08"}}
20:47:19 MQT: stat/p1-smart-meter/STATUS10 = {"StatusSNS":{"Time":"2020-05-03T20:47:19"}}
20:47:19 MQT: stat/p1-smart-meter/STATUS11 = {"StatusSTS":{"Time":"2020-05-03T20:47:19","Uptime":"0T00:06:01","UptimeSec":361,"Vcc":2.956,"Heap":26,"SleepMode":"Dynamic","Sleep":0,"LoadAvg":251,"MqttCount":1,"Wifi":{"AP":1,"SSId":"***","BSSId":"***","Channel":6,"RSSI":100,"Signal":-38,"LinkCount":1,"Downtime":"0T00:00:03"}}}

• Provide the output of the Console log output when you experience your issue; if applicable:
  (Please use weblog 4 for more debug information)

  Console output here:
20:51:50 MQT: tele/p1-smart-meter/RESULT = {"SerialReceived":"/ISK5\2M550E-1011"}

TO REPRODUCE

Steps to reproduce the behavior:

• Enter text through serial containing a \
• Check mqtt message
• Use something like https://jsonformatter.curiousconcept.com/ (disable the json fix option on the right!) to validate the message (other validators get really confused by the \ and report a general error about it not being a recognized type)

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen.
When non-json data from serial contains one of the forbidden characters (list can be found in ECMA-404 section 9), escape them so the resulting message contains valid json.

SCREENSHOTS

If applicable, add screenshots to help explain your problem.

ADDITIONAL CONTEXT

Add any other context about the problem here.

(Please, remember to close the issue when the problem has been addressed)

[sfromis]

To deal with this issue, I'd probably just switch the serial bridge to hex mode, and apply a conversion on the subscribing end of MQTT. This should allow any data to get through, irrespective of delimiters.

[ascillato2]

Closing this issue as it has been answered.

---

Support Information (Guide)

See Wiki for more information.
See FAQ for common questions/answers and links if none of your question is in the list.
See Chat for more user experience.
See Community for forum.
See Code of Conduct