Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This PR rewrites the lexer to: - Half the size of tokens from 32B to 16B. - Skip whitespace & comments in the lexer, not the parser. - Simplify some tokens (e.g a Percentage token is not needed). - Move span data from the lexer to the parser. It also rewrites the parser to: - Move all individual parsing logic to hdx_ast, colocating it with the AST nodes - Make use of more macros & Derives to massively DRY up parsing logic The aim is to: - Speed up overall parsing - Make it easier to develop - Make the whole thing more robust.
- Loading branch information