Unicode U+2007 character treated as whitespace on macOS

[frenchy64]

macOS Sequoia 15.2 (Intel)

This test for the keyword :<U+2007><U+2007> fails on macOS.

===============================================================================
/Users/ambrose/Projects/jank-local-dev/main/compiler+runtime/test/cpp/jank/read/lex.cpp:1664:
TEST SUITE: lex
TEST CASE:  UTF-8
  Whitespace Characters

/Users/ambrose/Projects/jank-local-dev/main/compiler+runtime/test/cpp/jank/read/lex.cpp:1778: ERROR: CHECK( tokens == make_tokens({ { 0, 7, token_kind::keyword, "  "sv } }) ) is NOT correct!
  values: CHECK( [ err(error(0 - 0, "invalid keyword: expected non-whitespace character after :")) ok(token(1, 3, symbol,  )) err(error(4 - 4, "expected whitespace before next token")) ok(token(4, 3, symbol,  )) ] == [ token(0, 7, keyword,   ) ] )

===============================================================================

[jeaye]

I've seen this showing up on some people's machine. I suspected it was a locale issue on that machine, not having UTF-8 setup properly.

[elken]

I've hit this in CI also, specifically setting the std::locale in tests fixed one of the emoji tests failing but not this one

Most recent of which

[elken]

Huh okay jeaye is right, I've setup the locale and the tests pass now. https://github.com/elken/jank/actions/runs/13245748321/job/36971725739

Incorrectly assumed the image had set a locale by default, I'll look at the macos side next.

[elken]

Yep that fails as expected on macOS, so when I can work out what locale thing needs adjusting I'll update this issue 😄

[jeaye]

Try this.

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index ae4039a2..1f448f1c 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -51,6 +51,9 @@ jobs:
       JANK_SANITIZE: ${{ matrix.sanitize }}
       ASAN_OPTIONS: detect_leaks=0
       TERM: xterm
+      LC_ALL: en_US.UTF-8
+      LANG: en_US.UTF-8
+      LANGUAGE: en_US.UTF-8
     timeout-minutes: 30
     steps:
       - uses: actions/checkout@v4

If that doesn't work, we may need to do something more with the system's locale.

[elken]

Already added that yeah no change, it also should be set locally on macOS proper