Skip to content

Commit

Permalink
Rewrite Lexer & Parser (#1)
Browse files Browse the repository at this point in the history
This PR rewrites the lexer to:

 - Half the size of tokens from 32B to 16B.
 - Skip whitespace & comments in the lexer, not the parser.
 - Simplify some tokens (e.g a Percentage token is not needed).
 - Move span data from the lexer to the parser.

It also rewrites the parser to:

- Move all individual parsing logic to hdx_ast, colocating it with the
AST nodes
 - Make use of more macros & Derives to massively DRY up parsing logic

The aim is to:

 - Speed up overall parsing
 - Make it easier to develop
 - Make the whole thing more robust.
  • Loading branch information
keithamus authored Feb 14, 2024
1 parent fe24a50 commit 85ad752
Show file tree
Hide file tree
Showing 835 changed files with 1,274,990 additions and 1,679,512 deletions.
1 change: 1 addition & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
root = true

[*]
max_line_length = 120
end_of_line = lf
insert_final_newline = true
charset = utf-8
Expand Down
Loading

0 comments on commit 85ad752

Please sign in to comment.