Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix invalid UTF-8 handling in Char::Reader#previous_char #14013

Merged

Conversation

HertzDevil
Copy link
Contributor

Fixes #14010.

This relies on an explicit reverse DFA for the UTF-8 encoding, instead of actually "trying" 1 - 4 bytes for a valid byte sequence.

@HertzDevil HertzDevil added kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:stdlib:text labels Nov 23, 2023
spec/std/char/reader_spec.cr Outdated Show resolved Hide resolved
end
end

return yield (first << 18) &+ (second << 12) &+ (third << 6) &+ (fourth &- 0x3C82080), 4, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return yield (first << 18) &+ (second << 12) &+ (third << 6) &+ (fourth &- 0x3C82080), 4, nil
yield (first << 18) &+ (second << 12) &+ (third << 6) &+ (fourth &- 0x3C82080), 4, nil

@straight-shoota straight-shoota added this to the 1.11.0 milestone Nov 24, 2023
@straight-shoota straight-shoota merged commit c7202d4 into crystal-lang:master Nov 25, 2023
52 of 55 checks passed
@HertzDevil HertzDevil deleted the bug/char-reader-previous-char branch November 26, 2023 18:24
Blacksmoke16 pushed a commit to Blacksmoke16/crystal that referenced this pull request Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug A bug in the code. Does not apply to documentation, specs, etc. topic:stdlib:text
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unlimited backtracking in Char::Reader#previous_char
3 participants