Editing code block content in visual editor, replaces line breaks with br tags, output is on single line now.

[oldrup]

Description

In the Gutenberg version shipped with WP 6.4, I can edit my content of the code block using the visual editor, and it's being presented fine on front end. I can apply syntax highlighting like Prism.JS for prettyness. Sample https://oldrup.dk/en/external-links/

Now I'm testing WP 6.5 beta 3, and/or the current version of the Gutenberg plugin, and when using the "visual editor" (not HTML editor), my linebreaks are being replaced with br tags (screenshot below)

This breaks my front end representation of the contents in the code block, everything is now on one line.

Only people using the code block for sharing code, in my case css, appears to be affected by this change, and there might be a good reason for the new way of outputting code this way. A workaround might be to edit code in "code view" moving forward. It's just something to be avare of.

Step-by-step reproduction instructions

1. Have a code block with multible lines of CSS
2. Updated to WP 6.5 beta 3 or latest Gutenberg plugin
3. Edit that code block using the visual editor view
4. Edit that code block using the code editor view
5. Notice that line breaks are replaced with br tages
6. Notice that output on the front end is now on one line instead of before multiple lines.

Screenshots, screen recording, code snippet

Content of code block before - multible lines:

Content of code block after a "visual edit" - everything on one line

Environment info

No response

Please confirm that you have searched existing issues in the repo.

Yes

Please confirm that you have tested with all plugins deactivated except Gutenberg.

Yes

[oldrup]

Additional info:

But downgrading to WP 6.4 and uninstalling Gutenberg, it's actually possible to fix the content of the code blocks, by editing them in visual mode and saving the post. That indicates that some change has been made to the code block editing experience with the current version of GB or WP 6.5 beta.

Please let me know if I can do anything else to troubleshoot this issue. I like being able to share code snippets on my WordPress blog.

[t-hamano]

Thanks for the report. I was also able to reproduce this problem. I believe this issue is similar to the issue reported in #56855.

First, enter the following CSS in the code block.

@media only screen {
    body {
        --link-suffix-content: "\25E5";
    }
}

In the front end it displays correctly.

Next, enable a plugin called Prismatic that highlights code on the front end. The front end will then display as follows.

Looking at the HTML source, it appears that the br tag is escaped.

@ellatrix My guess is that the library that extends the code block always escapes the br tag and relies on newline codes. I'm concerned that this problem probably affects many plugins, do you have any good solutions?

[t-hamano]

I would like to add this issue to the WP6.5 project board as it may be impacting various consumers.

[oldrup]

Thanks for the report. I was also able to reproduce this problem. I believe this issue is similar to the issue reported in #56855.

@ellatrix My guess is that the library that extends the code block always escapes the br tag and relies on newline codes. I'm concerned that this problem probably affects many plugins, do you have any good solutions?

Read the issue mentioned @t-hamano, and you are right—it is definitely related, nice catch. For funsies, I tested Prismatic with WP 6.4 and 6.5 beta 3, and it's the same problem.

Once you edit the content of a code block (in 6.5 beta), code is escaped and no longer renders as expected on front-end. Absolutely a breaking change for me, and I hope this new approach will be reconsidered or at least documented so we can take our precautions.

Because, once all preformatted code is being altered by Gutenberg and served in a single line with added br tags, is a nuisance to fix.

Thank you again for connecting the dots :)
Bjarne

[t-hamano]

@getdave @youknowriad @ellatrix @dmsnell

I believe this issue will impact many developers who are extending core code blocks, so I think something needs to be done in WP6.5.

---

Summary

I think the root cause is that the newline character included in RichText is replaced with the br tag (perhaps introduced by #58659?)

This issue is caused by the following scenarios:

• In WP6.4, enter some code in the code block.
• As of WP6.4, newline characters are maintained.
• User updates to WP6.5 (or activates the latest Gutenberg plugin).
• When a user updates a post, all newline characters in the code block are replaced with br tags.
• A library that extends the code block on the front end escapes br tags in the content. Or they ignore the br tag and rely on newline characters.
• As a result, the code will be displayed on one line.

As far as I have tested, the following plugins are affected:

• Syntax-highlighting Code Block (with Server-side Rendering) (1,000+ active installations)
• Prismatic (2,000+ active installations)
• Code Syntax Block (8,000+ active installations)

The screencast below reproduces the issue occurring with the Code Syntax Block.

76303117f919e7a075269ea5c0263582.mp4

[ellatrix]

Tbh, that sounds like a bug in those plugins, but now it seems like we'll have to fix it 😞
Btw, the code block can contain any HTML formatting like bold, links, etc., and it sounds like any of that will break these plugins as well? These plugins seems to rely on the code content being entirely plain text.

[ellatrix]

And the code syntax block uses RichText even though it doesn't really accept rich text, but only plain text?

[t-hamano]

the code block can contain any HTML formatting like bold, links, etc., and it sounds like any of that will break these plugins as well?
And the code syntax block uses RichText even though it doesn't really accept rich text, but only plain text?

I tested it with WP6.4.:

• Code Syntax Block: HTML formatting appears to be disabled in the editor. If I activate this plugin after applying some formats, those formats seem to be removed in the front end.
• Syntax-highlighting Code Block, Prismatic: HTML formatting is available on the editor side, but on the front end, those formats are escaped.

I thought it would be a good idea to publish a post as a dev note encouraging existing plugins to add a filter like the one below so that they can continue to rely on newline characters:

function replace_new_line_character( $block_content ) {
	$block_content = str_replace( '<br>', "\n", $block_content );
	return $block_content;
}
add_filter( 'render_block_core/code', 'replace_new_line_character' );

For some plugins this hook works, for others it doesn't. The br tag might have been escaped earlier in another filter...

[oldrup]

These plugins seems to rely on the code content being entirely plain text.

I'm not to say whether we are doing it the wrong way 🤷‍♂️

The WP implementation of the code block wraps the code element in the pre element, like this:

<pre> <code> This content is being shown on three lines </code> </pre>

"The <pre> HTML element represents preformatted text which is to be presented exactly as written in the HTML file. The text is typically rendered using a non-proportional, or monospaced, font. Whitespace inside this element is displayed as written."

MDN also states that

"If you have to display reserved characters such as <, >, &, and " within the <pre> tag, the characters must be escaped using their respective HTML entity."

And the current code block implementation does exactly that, escaping < and > etc, but otherwise, it adds nothing to the code pasted in. No br tags are being added in replacement for the pre-formatted code; it's already formatted.

Here is another odd example from MDN.

So it's true, I'm used to any br tags I paste into the code block, to be rendered as br tags on the frontend, like in the current WP 6.4 implementation:

Notice that the code blocks, in the two columns, are almost identical. The only difference is the language-html class added to the code block where I'd like syntax highlighting. I'm using PrismJS for highlighting, a small piece of JS and CSS. It's commonly used and supports every type of code imaginable. No plugin needed, so this change in WP 6.5, will affect more people than the plugin install counts mentioned.

This approach is, or was, very robust because it just enhances the existing code block, and if I run into trouble, I can simply disable the PrismJS code. No harm done, besides me losing a nice, widespread option for syntax highlighting.

My sample post, now updated to WP 6.5

Maybe we are syntax highlighting in a wrong way, relying on our pre tag wrapped code is actually pre-formated. But I also find it difficult to understand what is being gained, really, by then new approach with line breaks being replaced with br tags. 🤔

Just my two cents.

WP 6.5 is looking very promising; can't wait to play with the new "content override" feature of patterns; that is a huge deal.

Bjarne

[youknowriad]

Given that 6.5 RC1 is happening today, only very critical bugs are allowed to remain in the milestones. I think the current one is a bit unclear. I'd love more discussions for it and potentially a solution which could be a dev note as well.

[timnolte]

As a Dev using the Code Block for my site, I don't think this issue, having already been identified as something breaking, should just be allowed to roll out to sites without a fix. This is not just a security patch release so knowingly pushing out a major release with breaking bugs is not good form. I sure hope that isn't the decision being made here for the sake of an arbitrary release schedule.

[youknowriad]

The release schedule is not arbitrary. I'm happy to reconsider adding this to 6.5 if there's agreement and a fix is made on time for RC2. @ellatrix would you mind taking a second look here.

[timnolte]

The release schedule, IMO, is a guideline, there is nothing demanding that the releases go out at all cost.

[youknowriad]

The release schedule, IMO, is a guideline,

I can assure you that is not the case.

Keeping the release on schedule. Deadlines are not arbitrary. WordPress releases should strive to stay on schedule and the release lead and deputies are responsible for this schedule. (See “On Scheduling,” above.) There are many aspects to maintaining the release schedule, many of which are listed as responsibilities here.

There are cases where deadlines are not respected but there needs to be a very strong case for it.

For this particular bug, as I said again, I think it's something we can sanction for the next RCs I just need more clarity from contributors and have fix on the work.

[timnolte]

Has a Trac ticket been opened for this yet?

[youknowriad]

It seems to be a Gutenberg issue, so a trac ticket is not needed. As a fix for this will make it to through package update.

[timnolte]

I will take a look and see if I can determine the cause and provide a patch.

[timnolte]

So it looks like this might have broken when converting the Code Block to use RichText. It's not clear to me why it was changed to use RichText vs keeping it as plain text.

Well, so seeing as how that conversion was done longer ago I'm suspecting that isn't actually the direct cause but this was maybe caused by recent changes to the RichText block that didn't take into account the impacts on the Code Block. Seems we may need some additional tests for the Code Block in order to flag breaking affects to the Code Block.

[timnolte]

sigh so this issue ends up being a rather larger rabbit hole. Unfortunately, this isn't just a fix in WP 6.5 that's needed it's need in Gutenberg and then WP 6.5 will need to update it's version of Gutenberg. What is the likelihood that WP 6.5 will even allow pulling in a new version of Gutenberg in the RC?

[ellatrix]

I'm working on a fix

[ellatrix]

There's really two separate issues here:

• One is for plugins that filter the content on the front-end, without extending or creating a block.
• The other is for plugins extending or creating a block.

The first we can address by again making the Code block serialise line breaks as character new lines instead of HTML BR elements. I'm ok with doing that, but you really shouldn't escape non-escaped HTML in code blocks. That doesn't make any sense to me.

So it's true, I'm used to any br tags I paste into the code block, to be rendered as br tags on the frontend, like in the current WP 6.4 implementation:

Create a Code block and format text with bold and links. Your plugin shouldn't be escaping any of this formatting. :)

For the second case, it's a bit harder because we are dealing with plugins that are completely overriding the edit and save functions. I feel a bit less inclined to fix these cases. The content attribute even has an HTML source, so if you have a custom save function, it should be able to handle that. I don't feel like these block have enough active installs to make really drastic changes? The only fix I can think of is to revert to serialising HTML inside the edit function/RichText component when the preserveWhiteSpace props is set, as opposed to serialising in the save function. Maybe we can work with @westonruter and @mkaz to copy this save function again to their plugin.

Something I noticed with these two plugins as well: when you adds Code blocks with these plugins enabled, and later de-activate them, the Code blocks become invalid.

[justintadlock]

Noting that the Code block change breaks several JS-only syntax highlighters. I've mostly been testing with PrismJS, which is used on WordPress.org in some places and in several WP plugins. For Prism in particular, it strips all HTML within the <code> tag, so it's as if there are no line breaks at all.

These plugins are fundamentally broken because valid HTML within <code> is still valid HTML and should be rendered correctly (i.e., HTML not escaped or stripped).

Unfortunately, this will affect the WP ecosystem at a larger scale than what's presented in this ticket, which is why a fix (for line breaks) should be shipped with WP 6.5.

We should do some outreach with plugin authors to let them know that—while the line break issue is being fixed in a PR—they need to account for HTML within the Code block. Otherwise, it's going to break things like links, bold, italic, and other Rich Text.

CC: @ndiego - This needs to be noted in the March What's new for developers? post on the Developer Blog.

[justintadlock]

For plugin authors using PrismJS, I recommend bundling the Keep Markup extension: https://prismjs.com/plugins/keep-markup/

This will preserve the Rich Text markup, so it won't be stripped from the output.

[dmsnell]

I feel like the spirit of this old PR is being invoked again.

[t-hamano]

Something I noticed with these two plugins as well: when you adds Code blocks with these plugins enabled, and later de-activate them, the Code blocks become invalid.

One reason is probably because the plugin completely overrides the save function and adds a class called language-{XXX} to the code element inside the block:

<!-- wp:code -->
<pre class="wp-block-code"><code lang="css" class="language-css"></code></pre>
<!-- /wp:code -->

Therefore, block validation will fail when the plugin is disabled.

I think this comes from the fact that in Prism.js, the basic usage is to add a CSS class to the code element inside the pre element.

Additionally, perhaps Prism.js removes the br tag by default on the front end.

For example, create the following HTML block to apply Prism.js on the front end:

<!-- wp:html -->
<pre><code class="language-css">@media only screen {<br>    body {<br>        --link-suffix-content: "\25E5";<br>    }<br>}</code></pre>
<!-- /wp:html -->

The editor will of course respect the br tag and display line breaks correctly:

And load the basic Prism.js library:

function add_prism() {
	wp_enqueue_script( 'prism', 'https://cdn.jsdelivr.net/npm/[email protected]/prism.min.js', array(), null, true );
	wp_enqueue_style( 'prism', 'https://cdn.jsdelivr.net/npm/[email protected]/themes/prism.min.css', array(), null );
}
add_action( 'wp_enqueue_scripts', 'add_prism' );

The code appears in one line because Prism.js removes the br element:

Therefore, I think it makes sense to preserve newlines, at least in the Code block, as suggested by #59627.

[oldrup]

Just tested with WP 6.5 RC2, Prism.JS and a code highlight plugin. Everything looks great now. Content of code block survives edit and deactivating of the syntax highlighting.

Thank you ✨

[thphuccoder]

Was this issue fixed in Wordpress 6.5? I'm still facing the bug today (My website runs Wordpress 6.5)

[t-hamano]

Hi @thphuc,

Could you tell me the details of the problem?

• Does that problem occur in the core code block?
• Are you using any plugins that extend the core code block?

[thphuccoder]

Hi @t-hamano

Yes, the problem occurs in the core code block. I use Code Syntax Bloc plugin: https://wordpress.org/plugins/code-syntax-block/

My website was (auto) updated recently to Wordpress 6.5. After that, the problem started appearing.

All the posts we created before are still showing the code correctly. I checked the HTML, in those old posts, there are no
tags inserted.
However, for all new code blocks, the
tags are inserted automatically when there is a new line break. If I remove the
tags, everything works correctly.

Thanks!

[t-hamano]

@thphuc Thank you for the reply.

We have resolved this issue with #59627. However, as stated in this comment, the Code Syntax Block overrides the save function, so the solution by #59627 may not be reflected.

Ideally, I would like the save function to be fixed in a plugin update, but if not, the hook below should be able to resolve the issue.

// Replace br tags with new line characters
function replace_new_line_character( $block_content ) {
	$block_content = str_replace( '<br>', "\n", $block_content );
	return $block_content;
}
add_filter( 'render_block_core/code', 'replace_new_line_character' );

When the filter is applied

When the filter is not applied

[thphuccoder]

Thanks @t-hamano I ended up using the above hook to fix it!