BJData optimized binary array type #4513
Conversation
🔴 Amalgamation check failed! 🔴The source code has not been amalgamated. @nebkat |
|
Please run |
Introduces a dedicated `B` marker for bytes. This is used as the strong type marker in optimized array format to encode binary data such that it can also be decoded back to binary data (instead of decoding as an integer array). See NeuroJSON/bjdata#6 for further information.
| @@ -1514,11 +1542,8 @@ TEST_CASE("BJData") | |||
| // create expected byte vector | |||
| std::vector<std::uint8_t> expected; | |||
| expected.push_back(static_cast<std::uint8_t>('[')); | |||
| if (N != 0) | |||
There was a problem hiding this comment.
Seeing existing test cases change - is this a breaking change for existing clients?
There was a problem hiding this comment.
Yes - this is a breaking change for encoding, but fixes roundtrip encoding/decoding.
Before:
json::binarygets encoded as a typed array ofU- typed array of
Ugets decoded asjson::array❗does not match encoded data
After:
json::binarygets encoded as a typed array ofB- typed array of
Bgets decoded asjson::binary✅ matches encoded data
I could introduce a new flag to continue encoding binary as uint8 if the breaking change is undesirable?
There was a problem hiding this comment.
We must not break existing code, so yes, a parameter would make sense.
| @@ -847,11 +847,11 @@ class binary_writer | |||
| oa->write_character(to_char_type('[')); | |||
| } | |||
|
|
|||
| if (use_type && !j.m_data.m_value.binary->empty()) | |||
| if (use_type && (use_bjdata || !j.m_data.m_value.binary->empty())) | |||
There was a problem hiding this comment.
@nlohmann
And that specific test-case you commented on has changed due to this line. In order to ensure that encoding an empty json::binary still decodes to json::binary I force the insertion of the type here regardless of whether there are contents or not.
That said this problem will still exist if other libraries are used (e.g. python encodings an empty bytearray and doesn't include the type).
See NeuroJSON/bjdata#6 for further information.
Introduces a dedicated
Bmarker for bytes.This is used as the strong type marker in optimized array format to encode binary data such that it can also be decoded back to binary data (instead of wrongly decoding as an integer array).
Draft while awaiting the release of BJData draft 3.
Would a
legacy_binaryflag be desirable to continue supporting draft 2 expectations of uint8 typed arrays for binary?Pull request checklist
Read the Contribution Guidelines for detailed information.
include/nlohmanndirectory, runmake amalgamateto create the single-header filessingle_include/nlohmann/json.hppandsingle_include/nlohmann/json_fwd.hpp. The whole process is described here.