Overly aggressive fontification for within-token underscores

[rriegs]

CommonMark specifies that, while asterisk * can establish within-token emphasis, underscore _ cannot.

Expected Behavior

Input

underscore_containing_token

should not have any emphasis, even though

asterisk*containing*token

emphasizes containing.

Actual Behavior

underscore_containing_token

emphasizes containing the same as with asterisks.

Software Versions

markdown-mode version 2.4-dev (installed via MELPA)
GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21) of 2017-09-22, modified by Debian
Ubuntu 18.04.4 LTS

[syohex]

markdown-mode does not conform to all CommonMark spec. While markdown-mode conforms original markdown specification because markdown-mode was written before CommonMark birthed. _ specification is different between original spec and common mark. Original spec requires back-slash in such case like underscore\_containing\_token for avoiding emphasis.

I'm not sure markdown-mode.el should conform CommonMark specification because it may breaks many documents which were written in markdown-mode.

[rriegs]

I just noticed that markdown-mode also offers gfm-mode, which actually does refrain from fontification for within-token underscores, however, it also doesn't fontify within-token asterisks, which it should. I would argue that, whether or not markdown-mode strives to match CommonMark, gfm-mode should definitely strive to match GFM, as this is what it's name and documentation suggest it does.

Input:

In gfm-mode:

*emphasis*

em*pha*sis

*emph*asis

emph*asis*

em*ph as*is

_emphasis_

no_emph_asis

_no_emphasis

no_emphasis_

no_em ph_asis

(Note the double newlines are significant.)

Expected output:

Lines with text "emphasis" are (at least partially) emphasized. Lines with text "no emphasis" are not emphasized at all, i.e.:

emphasis

emphasis

emphasis

emphasis

emph asis

emphasis

no_emph_asis

_no_emphasis

no_emphasis_

no_em ph_asis

Actual output:

Only *emphasis* and _emphasis_ are emphasized.

[syohex]

I have merged #495. Please check latest version.