build error: character too large for enclosing character literal type

[patrickt]

Running tree-sitter test produces the following error when compiling scanner.cc:

emcc command failed - src/scanner.cc:88:32: error: character too large for enclosing character literal type
           lexer->lookahead == '\uFEFF' ||
                               ^
src/scanner.cc:89:32: error: character too large for enclosing character literal type
           lexer->lookahead == '\u2060' ||
                               ^
src/scanner.cc:90:32: error: character too large for enclosing character literal type
           lexer->lookahead == '\u200B';

[simonrepp]

I've encountered this myself - if I remember correctly:

• You can comment out the affected lines for the time being if you just want to work on stuff (that's what I did so far), these are more exotic whitespace cases which of course matter, but are irrelevant for the overall implementation and general testing.
• It only occurrs in certain compilation scenarios (when building for the wasm target?)
• I don't have a solution yet!
• If you find one I'd be much obliged :)

In any case, thanks for the report! Glad about your interest in this, let me know if you have more questions and/or issues!

[patrickt]

I fixed it by removing the character syntax:

  inline bool is_horizontal_whitespace(TSLexer *lexer) {
    return lexer->lookahead == ' ' ||
           lexer->lookahead == '\t' ||
           lexer->lookahead == 0xFEFF ||
           lexer->lookahead == 0x2060 ||
           lexer->lookahead == 0x200B;
  }

[maxbrunsfeld]

Yeah, I don't think '\uFEFF' is a valid C/C++ expression. Single quoted character literals are of type char, which is almost always an 8-bit value. The largest numerical char value is 0xFF.

In more recent versions of C++, you can use a U prefix (e.g. U'\uFEFF'). I think the normal way to write numbers like this is to just use integer syntax, like @patrickt said ☝️ .

[simonrepp]

Very cool, thanks patrick for the fix and max for the additional insight, appreciate it!

Commited in c059967