-
-
Notifications
You must be signed in to change notification settings - Fork 128
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adds graph debug documentation to book
- Loading branch information
Showing
2 changed files
with
117 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
# Debug Graph | ||
|
||
It may be beneficial during debugging to be able to visualize how Logos has generated its graph. | ||
|
||
If we take this example: | ||
```rust,no_run,noplayground | ||
use logos::Logos; | ||
#[derive(Debug, Logos, PartialEq)] | ||
enum Token { | ||
// Tokens can be literal strings, of any length. | ||
#[token("fast")] | ||
Fast, | ||
#[token(".")] | ||
Period, | ||
// Or regular expressions. | ||
#[regex("[a-zA-Z]+")] | ||
Text, | ||
} | ||
fn main() { | ||
let input = "Create ridiculously fast Lexers."; | ||
let mut lexer = Token::lexer(input); | ||
while let Some(token) = lexer.next() { | ||
println!("{:?}", token); | ||
} | ||
} | ||
``` | ||
|
||
With our graph debugging enabled, we see the following output: | ||
``` | ||
graph = { | ||
1: ::Fast, | ||
2: ::Period, | ||
3: ::Text, | ||
4: { | ||
[A-Z] ⇒ 4, | ||
[a-z] ⇒ 4, | ||
_ ⇒ 3, | ||
}, | ||
7: [ | ||
ast ⇒ 8, | ||
_ ⇒ 4*, | ||
], | ||
8: { | ||
[A-Z] ⇒ 4, | ||
[a-z] ⇒ 4, | ||
_ ⇒ 1, | ||
}, | ||
9: { | ||
. ⇒ 2, | ||
[A-Z] ⇒ 4, | ||
[a-e] ⇒ 4, | ||
f ⇒ 7, | ||
[g-z] ⇒ 4, | ||
}, | ||
} | ||
``` | ||
|
||
Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`. We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input. | ||
|
||
We also can try to identify how `fast` works by looking at `@9` that `f` jumps to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (ie `[g-z] => 4`) | ||
|
||
## Enabling | ||
|
||
### Step 1 | ||
Start by pulling the Logos repo locally | ||
``` | ||
git clone [email protected]:maciejhirsz/logos.git | ||
``` | ||
|
||
|
||
### Step 2 | ||
In the [Generator::new](https://github.com/maciejhirsz/logos/blob/master/logos-codegen/src/generator/mod.rs#L47) function you can add `dbg!(graph);` like the following: | ||
|
||
```rust,no_run,noplayground | ||
impl<'a> Generator<'a> { | ||
pub fn new( | ||
name: &'a Ident, | ||
this: &'a TokenStream, | ||
root: NodeId, | ||
graph: &'a Graph<Leaf>, | ||
) -> Self { | ||
let rendered = Self::fast_loop_macro(); | ||
let meta = Meta::analyze(root, graph); | ||
// Add debug here | ||
dbg!(graph); | ||
Generator { | ||
name, | ||
this, | ||
root, | ||
graph, | ||
meta, | ||
rendered, | ||
fns: Set::default(), | ||
idents: Map::default(), | ||
gotos: Map::default(), | ||
tests: Map::default(), | ||
tables: TableStack::new(), | ||
} | ||
} | ||
``` | ||
|
||
### Step 3 | ||
Run `cargo build` | ||
|
||
|
||
### Step 4 | ||
Now in your code that uses Logos, point your Logos dependency in `Cargo.toml` to your local repo | ||
``` | ||
[dependencies] | ||
logos = { path = "../path/to/logos" } | ||
regex = "1.10.3" | ||
``` |