Embedded null bytes in source files

[HertzDevil]

Crystal::Lexer uses '\0' to check for end-of-file conditions in multiple places, instead of Char::Reader#has_next?. A source file that contains a null byte will most likely fail to compile, e.g. when it appears within a string literal:

$ xxd test.cr
00000000: 7020 2261 0062 220a                      p "a.b".
$ crystal test.cr
In test.cr:1:5

 1 | p "ab"
         ^
Error: Unterminated string literal

Ruby seems to happily accept this:

$ ruby test.cr
"a\u0000b"

Should we allow those null bytes?

[asterite]

I think back then I checked against the null byte just because I didn't think of a use case for having null bytes. But I don't see anything wrong with allowing that either, for completeness.