Skip to content

Commit

Permalink
Adds graph debug documentation to book
Browse files Browse the repository at this point in the history
  • Loading branch information
afreeland committed Feb 19, 2024
1 parent 81f923c commit 4af7244
Show file tree
Hide file tree
Showing 2 changed files with 117 additions and 0 deletions.
1 change: 1 addition & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
+ [Using `Extras`](./extras.md)
+ [Using callbacks](./callbacks.md)
+ [Common regular expressions](./common-regex.md)
+ [Debug Graph](./debug-graph.md)
+ [Examples](./examples.md)
+ [Brainfuck interpreter](./examples/brainfuck.md)
+ [JSON parser](./examples/json.md)
Expand Down
116 changes: 116 additions & 0 deletions book/src/debug-graph.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Debug Graph

It may be beneficial during debugging to be able to visualize how Logos has generated its graph.

If we take this example:
```rust,no_run,noplayground
use logos::Logos;
#[derive(Debug, Logos, PartialEq)]
enum Token {
// Tokens can be literal strings, of any length.
#[token("fast")]
Fast,
#[token(".")]
Period,
// Or regular expressions.
#[regex("[a-zA-Z]+")]
Text,
}
fn main() {
let input = "Create ridiculously fast Lexers.";
let mut lexer = Token::lexer(input);
while let Some(token) = lexer.next() {
println!("{:?}", token);
}
}
```

With our graph debugging enabled, we see the following output:
```
graph = {
1: ::Fast,
2: ::Period,
3: ::Text,
4: {
[A-Z] ⇒ 4,
[a-z] ⇒ 4,
_ ⇒ 3,
},
7: [
ast ⇒ 8,
_ ⇒ 4*,
],
8: {
[A-Z] ⇒ 4,
[a-z] ⇒ 4,
_ ⇒ 1,
},
9: {
. ⇒ 2,
[A-Z] ⇒ 4,
[a-e] ⇒ 4,
f ⇒ 7,
[g-z] ⇒ 4,
},
}
```

Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`. We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.

We also can try to identify how `fast` works by looking at `@9` that `f` jumps to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (ie `[g-z] => 4`)

## Enabling

### Step 1
Start by pulling the Logos repo locally
```
git clone [email protected]:maciejhirsz/logos.git
```


### Step 2
In the [Generator::new](https://github.com/maciejhirsz/logos/blob/master/logos-codegen/src/generator/mod.rs#L47) function you can add `dbg!(graph);` like the following:

```rust,no_run,noplayground
impl<'a> Generator<'a> {
pub fn new(
name: &'a Ident,
this: &'a TokenStream,
root: NodeId,
graph: &'a Graph<Leaf>,
) -> Self {
let rendered = Self::fast_loop_macro();
let meta = Meta::analyze(root, graph);
// Add debug here
dbg!(graph);
Generator {
name,
this,
root,
graph,
meta,
rendered,
fns: Set::default(),
idents: Map::default(),
gotos: Map::default(),
tests: Map::default(),
tables: TableStack::new(),
}
}
```

### Step 3
Run `cargo build`


### Step 4
Now in your code that uses Logos, point your Logos dependency in `Cargo.toml` to your local repo
```
[dependencies]
logos = { path = "../path/to/logos" }
regex = "1.10.3"
```

0 comments on commit 4af7244

Please sign in to comment.