Parser fuzz testing tools and fixes #185
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some fuzz testing tooling to check that the parser doesn't crash on randomly generated source strings. Fix several problems found with this: * `?` shouldn't be special in `parse_unary`. This was inherited from the a syntax hack used to support the ancient and questionable `@windows?` and other platform test macros in osutils.jl. This is long since gone and we shouldn't continue supporting this. * `<:` may be unary so `<: <: x` should parse as `(<: (<: x))`, even though this is kind of nonsense semantically. * Constructing a SyntaxNode tree shouldn't fail when there's malformed literals but when we've parsed using `ignore_errors=true`. Instead we use ErrorVal() for the leaf values in that tree. * The tokenizer should not crash when overlong UTF-8 character literals are encountered.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some fuzz testing tooling to check that the parser doesn't crash on randomly generated source strings.
Fix several problems found with this:
?shouldn't be special inparse_unary. This was inherited from the a syntax hack used to support the ancient and questionable@windows?and other platform test macros in osutils.jl. This is long since gone and we shouldn't continue supporting this.<:may be unary so<: <: xshould parse as(<: (<: x)), even though this is kind of nonsense semantically.ignore_errors=true. Instead we use ErrorVal() for the leaf values in that tree.