Puppet syntax broken in a number of contexts

[alex-harvey-z3q]

I am using the Rouge gem in my Jekyll on Github blog and have found a number of issues with the Puppet syntax highlighter.

(I can see from the revision history that it hasn't really been changed since 2013, so at least some of the problems would be that it lacks Puppet >= 4 syntax support.)

Issue 1 (note the red equals sign), example 1:

Issue 1, example 2:

Issue 2:

These are from two posts online here and here.

[alex-harvey-z3q]

Geoff Williams wrote in the Puppet Community Slack:

@alex Just had a quick look at that library. Seems that the lexer for puppet goes to rule /[$]#{qualname}/, Name::Variable, I tried to find the definition of Name::Variable and closest I got was https://github.com/sinatra/mustermann/blob/master/mustermann/lib/mustermann/ast/compiler.rb I'd guess Name::Variable doesn't match Puppet's idea of a regex, although the library does seem to support hashes. What if you try assigning a hash with only one level, or an array, or just a simple variable? Could be that the regex used just doesn't support n-level nesting?

[alex-harvey-z3q]

@jneen if you could give me a clue what might be wrong I'd be happy to try to work on a fix. I did have a look but I can't get my head around the code. E.g. where does Name::Variable get defined?

[dblessing]

Hi @alex. You don’t need to worry about the definition of `Name::Variable‘. It’s a token that Rouge uses only to color items later on. You will need to adjust rules in the lexer (add, remove, rearrange) to fix the problem. Wiki has some instructions on debugging/testing lexers https://github.com/jneen/rouge/wiki/Setting-up-Ruby. Let me know if you have more questions and I can point you in the right direction.

…

On Mon, Sep 3, 2018 at 6:59 AM Alex Harvey ***@***.***> wrote: @jneen <https://github.com/jneen> if you could give me a clue what might be wrong I'd be happy to try to work on a fix. I did have a look but I can't get my head around the code. E.g. where does Name::Variable get defined? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#979 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACSKlO6WkXpf29crshubW_WrVaXLM2wDks5uXRmlgaJpZM4WXe5e> .

[jneen]

If you're curious where it's defined, it's in lib/rouge/token.rb. But as @dblessing already mentioned, it's nothing more than a fancy symbol - the type of tokens that are variables.

You can try running

rougify -L debug=1 -f null my_file.pp

to get a debug trace of the lex. (-L debug=1 sets debugging on, and -f null uses the "null" formatter, which suppresses normal output.)

[alex-harvey-z3q]

Weirdly enough the lexer simply wasn't recognising the = sign as an Operator. Fixed:

Example 1

Example 2

Example 3

[alex-harvey-z3q]

@jneen I raised a PR to fix it above. How long will it take for the fix to propagate to Jekyll on Github?

[dblessing]

Thanks for the PR! I’ll review today and get it merged. It will be in the next release which I hope to publish in the coming weeks.

…

On Tue, Sep 4, 2018 at 5:15 AM Alex Harvey ***@***.***> wrote: @jneen <https://github.com/jneen> I raised a PR to fix it above. How long will it take for the fix to propagate to Jekyll on Github? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#979 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACSKlDq7NE3iVlXoPfgakbvghYwqmQaUks5uXlKrgaJpZM4WXe5e> .