Runtime checks for nan and inf values?

[liautaud]

While fuzzing the JSON encoder of Irmin, which uses jsonm as a backend, I came across some counterintuitive bugs while trying to encode non-standard floating-points values.

Although the documentation specifies that "Float lexemes must not be, Pervasives.nan, Pervasives.infinity or Pervasives.neg_infinity", there are no runtime errors when trying to encode such values. Instead, the encoder will comply and encode them using %.16g, which could generate the invalid tokens nan, inf or -inf, and the crash will only happen on decoding.

Is this absence of runtime checks motivated by performance? Otherwise, would it be possible to add checks to w_value to prevent such tokens from being produced in the first place?

[dbuenzli]

There are quite a few other things, that your fuzzer missed like trying to encode invalid UTF-8.

Yes the encoder does not check these things for performance reasons and expects these checks to be performed by the client. See the details of the contract here.

[liautaud]

@dbuenzli Alright, thanks! (The fuzzer did find bugs with invalid UTF-8, which we prevented by adding checks in the client, so we'll just do it for floats too then.)

[edwintorok]

FWIW this makes 'jsonm' produce non-standard JSON output that cannot be parsed by regular JSON clients.
This resulted in a bug where a client of a server using jsonm had to patch its JSON parsers to recognize the invalid output.

Yojson does produce valid output on NaN (although it is still up to the client to know how to interpret it, but it can be done at the application layer without having to patch the JSON parser grammar). However AFAICT Yojson doesn't validate that its input is UTF-8, so I would still prefer to use 'jsonm' due to its better input validation.

Due to 'jsonm' producing invalid JSON on output we had to switch our code to use yojson though (which AFAICT always produces valid JSON on output), it doesn't make sense to use 2 JSON libraries in the same application.

[dbuenzli]

Due to 'jsonm' producing invalid JSON on output we had to switch our code to use yojson though

jsonm does not produce invalid output. It's the duty of the client to check these things as there is nothing standard that can be done. What you want to do on nans and infinity that json cannot represent is for the client to decide. This is properly documented here.