Support for escape: :binary_safe

[josevalim]

Would you support a escaping mode that escapes anything that is not valid Unicode as \x?

Our use case is listed here: livebook-dev/livebook#2158 - the idea is that we need to send any user result or error messages to the client, and if the user gives or fetches malformed data, encoding it incorrectly and sending some feedback is better than crashing.

[michalmuskala]

I'm not sure I understand how would this be different from the escape: :unicode option. Could you elaborate a bit more?
I'm not sure escaping as \x would be useful since it would produce invalid JSON that can't be decoded by anything, but perhaps a unicode replacement char could be used in such cases

[josevalim]

I wanted to escape \xBD, which currently fails:

iex(2)> Jason.encode! "\xBD", escape: :unicode
** (Jason.EncodeError) invalid byte 0xBD in <<189>>
    (jason 1.4.1) lib/jason.ex:164: Jason.encode!/2
    iex:2: (file)

But if you tell me this is invalid JSON, then please go ahead and close this, as it wouldn't make sense :)

[michalmuskala]

Yes, JSON only accepts UTF8, and only escapes in the form of \uXXXX that denote valid unicode codepoints, so there's generally no compliant way of representing something like \xBD.
One option would be to encoding them as unicode replacement char, so we still produce something, though the conversion would be lossy.

[josevalim]

Well, the strings are indeed invalid, so it is either showing gibberish or a replacement char, might as well go for a replacement char, so we can close this as a duplicate of #12. :) Thank you!