json parser fails to handle non-BMP unicode escapes

[ohmyarch]

#include <cpprest/json.h>

int main(int argc, char *argv[]) {
    ucout << web::json::value::parse(U("{\"test\":\"\\ud83c\\uddee\\ud83c\\udde9\"}"))
                 .at(U("test"))
                 .as_string()
          << std::endl;
}

terminate called after throwing an instance of 'std::range_error'
what(): UTF-16 string is missing low surrogate

[ohmyarch]

"\ud83c\uddee\ud83c\udde9" is a unicode string that represents 🇮🇩.

[ras0219-msft]

Which commit are you using? I do not get this behavior on the development branch.

[ohmyarch]

@ras0219-msft I have been using the development branch. I created a repository. And it went wrong with the same error on both Linux and OS X.

[ras0219-msft]

I see, thank you for the repro. I can confirm this is broken in the development branch.

I have checked in a fix to the branch json-utf8-3.0. On my (Linux) machine, this fixes the issue. Could you confirm that it fixes the problem on your side as well?

[ohmyarch]

@ras0219-msft I tried json-utf8-3.0 branch, it runs correctly. But I found out a new bug and I updated the repository. Please check it.

[bsjones]

I am also seeing this bug. Based on the commit above we have had some success, just need to check for the the upper bounds for the high surrogate

Tried with this https://github.com/minimaxir/big-list-of-naughty-strings/blob/4115c9deee71a7d732d4e50b814df33c4207789b/blns.txt#L209
EXPECT_NO_THROW( auto str = web::json::value::parse(""\u30fd\u0f3c\u0e88\u0644\u035c\u0e88\u0f3d\uff89 \u30fd\u0f3c\u0e88\u0644\u035c\u0e88\u0f3d\uff89""));

non-bmp-chars-patch.txt