Apostrophes (single quotes) triggering weird highlighting.

[flowchartsman]

If I'm reading the documentation correctly, apotrophe (') characters between two word characters are detected and rendered as a single curved apostrophe. This seems to be correct in my rendering, but the extension seems to highlight them as if I'm writing a quoted phrase. This causes long blocks of colored text in my document every time I use a contraction.

Here is a contrived example screenshot, taken from my editor with the Light+ color scheme enabled.

[ggrossetie]

This is a limitation of the TextMate grammar. In this case, we might find a less greedy regular expression but we can only do so much...

[flowchartsman]

Actually, it seems as if this might be more to do with the embedded grammars. I just notices that the behavior does not start until after my first embedded code listing which declares a language (in this case, Go)

It seems as if the embedded grammar is not being fenced by the ---- listing delimiters correctly and is "leaking" out into the main body of the text.

[ggrossetie]

It seems related to #403

[flowchartsman]

Yes, that seems likely, especially given the bash block example which shows the same behavior with single quotes.

Normally I'd be inclined to ignore it--parsing is hard and TextMate's regexp approach is limited-- but contractions are common enough that this makes my text difficult to read. In any case, feel free to close as a duplicate.

[ggrossetie]

As a workaround, you can try to use:

[source,go,linenums]
----
include::example.go[]
----

It might help the regular expression.

[flowchartsman]

I tried this out, but unfortunately it's a non-starter since it would require a ton of changes since my entire document is written with inline code and some of it shows the evolution of a particular piece of code, which means it differs from the final version that is actually on disk. I could save multiple files I guess, but that would be yet more work, and I would need to structure the repository so that the intermediate files are ignored. It's a shame since this is otherwise a great extension, but it looks like I'll have to disable it for now.

Perhaps a faster workaround might be adding an option to disable the embedded grammar parsing? I don't really require it in the main editor window and highlighting would continue to work in preview and render. It might even be a good idea to disable this by default, since, presumably, it is broken everywhere, for everyone.

[ggrossetie]

Could you please provide a minimal reproduction case? We might be able to tighten the regular expression. Thanks!

[flowchartsman]

So, I have some interesting news and maybe even a fix. When I received the GH notification, I happened to be editing the beginning of the chapter where I first noticed the highlight problem, so I went to the first code listing to make the reproduction case, and found that it had an unterminated function (i.e. missing the final curly bracket). Upon fixing that, the rest of the document seems to work again, since every other listing appears to be valid (still testing, though).

So this seems to be exactly what is described in #403, where the first example was the result of a missing parenthesis and the second is likely because $ command is not valid Bash.

I did some digging, and ran across microsoft/vscode#20488, which lead me to a mention of the fix for markdown which encountered similar issues, along with a related fix in another project.

The upshot is that it seems like a possible fix would be to migrate from an end rule to a negated while rule, similar to how markdown did it. EDIT: Nope. See below comment.

In any case, here, is the minimal repro:

This line isn't a problem.

.Code Listing
[source,go,linenums]
----
func MyTest(t *Testing.T) {
----

This doesn't format correctly

[flowchartsman]

Update: wasn't able to get this working in testing. I think the while rule solution might be more specific for non-fenced rules that involve a closing tag. I'll continue to plug away at it, but I'm relatively green on TM grammars. I'm thinking your initial solution of a less-greedy regexp might be more correct.

[ggrossetie]

@flowchartsman Let me know if you need help and thanks for looking into it, greatly appreciated 🙌🏻