RMarkdown child documents not rendering properly: ```{=html} shows in the output

[hugocosh]

See SO 68225360
https://stackoverflow.com/questions/68225360/rmarkdown-child-documents-not-rendering-properly-html-shows-in-the-output

---

We have a big interactive Rmarkdown document using shiny that we split into a parent/child structure to make it easier to manage, as per this guidance. We use tabsetPanel to pull different child documents into different tabs.

This used to work well, but it has recently broken when using R 4.0.4. Still works on my old 3.6.2 version of R.
When you run the parent document, ```{=html} appears at the top of the output, the tabs don't work any more and the text has disappeared. There are also three backticks at the bottom of the output.

screenshot of the output with unexpected bits highlighted

So far I have tried doing this in a different way using the following code from the guide linked above, but I don't think this will work with shiny and the layout we want.

res <- knitr::knit_child('child.Rmd', quiet = TRUE)
cat(res, sep = '\n')

Here's the code for the three R Markdown documents:
a) parent.Rmd
b) child1.Rmd
c) child2.Rmd

a) parent.Rmd

---
output:
  html_document

runtime: shiny
---

```{r, echo = FALSE}

library(tidyverse)

tabsetPanel(type="tabs", id="x"

, tabPanel("child1", value="y"
  , HTML(knitr::knit_child('child1.Rmd', quiet = TRUE))
  )
  
, tabPanel("child2", value="z"
  , HTML(knitr::knit_child('child2.Rmd', quiet = TRUE))    
  )

)
```

b) child1.Rmd

---
output:
  html_document

runtime: shiny
---


Here is some text describing the chart below

And this is some more text

```{r, echo = FALSE}

fluidPage(

fluidRow(column(6,
                
renderPlot(
  iris %>% 
    ggplot(aes(Sepal.Length, Sepal.Width)) + 
    geom_point()
)

)))


```

c) child2.Rmd

---
output:
  html_document

runtime: shiny
---


Here is some text describing the chart below

And this is some more text


```{r, echo = FALSE}

fluidPage(

fluidRow(column(6,

renderPlot(
  mtcars %>%
      ggplot(aes(disp, mpg)) +
      geom_point()
)

)))

```

And here's my session info:

xfun::session_info('rmarkdown')
R version 4.0.4 (2021-02-15)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042), RStudio 1.2.1335

Locale:
  LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252   
  LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C                           
  LC_TIME=English_United Kingdom.1252    

Package version:
  base64enc_0.1.3   digest_0.6.27     evaluate_0.14     glue_1.4.2       
  graphics_4.0.4    grDevices_4.0.4   highr_0.9         htmltools_0.5.1.1
  jsonlite_1.7.2    knitr_1.33        magrittr_2.0.1    markdown_1.1     
  methods_4.0.4     mime_0.11         rlang_0.4.11      rmarkdown_2.9.1  
  stats_4.0.4       stringi_1.6.2     stringr_1.4.0     tinytex_0.32     
  tools_4.0.4       utils_4.0.4       xfun_0.24         yaml_2.2.1       

Pandoc version: 2.6

Checklist

When filing a bug report, please check the boxes below to confirm that you have provided us with the information we need. Have you:

• [Y] formatted your issue so it is easier for us to read?

• [Y] included a minimal, self-contained, and reproducible example?

• [Y] pasted the output from xfun::session_info('rmarkdown') in your issue?

• [Y] upgraded all your packages to their latest versions (including your versions of R, the RStudio IDE, and relevant R packages)?

• [Y] installed and tested your bug with the development version of the rmarkdown package using remotes::install_github("rstudio/rmarkdown")?

[cderv]

This special syntax with backtick is appearing due to a change in htmltools. This is used to protect specific HTML part during Pandoc conversion. rmarkdown now ask htmltools to use this special Raw Block syntax for Pandoc. It should not be seen in the output after Pandoc conversion so you may have found a bug.

You can deactivate for now this behavior to get the old one that should still work by setting options(htmltools.preserve.raw = FALSE) somewhere in your document or global R script / Rprofile. This should be a workaround to make it work. It could be specific to runtime shiny

In the meantime we need to understand why this happens.

[cderv]

Let me makes a few comments on what is going on here.

You are calling HTML(knitr::knit_child('child1.Rmd', quiet = TRUE) inside the main Rmd. So you are explicitly asking to preserve the content of HTML() as raw HTML content, the content being the result of knit_child().

First problem is that knit_child() will knit() the Rmd document in the context of the current knitting process. And using knit() on a Rmd file will output a .md file - so Markdown content, and not HTML content. By passing the result of knit_child() into HTML(), I believe there is something not quite right. Which in a way is the source of the issue. The guidance you link to in the R Markdown Cookbook is explaining how to use knit_child() to create markdown content to insert into a main R Markdown document. Here you are passing the result to a function (HTML()) that does not expect Markdown but HTML.

But that being said, let's dig to understand why you see what you see.

In a knit rendering, htmltools::htmlPreserve() will be used in the htmltools:::knit_print.html methid. We can emulate this to see what happens

# with htmltools.preserve.raw = FALSE
> knitr::knit(text = c("```{r, echo = FALSE}", "htmltools::HTML('<h1>Hello</h1>')", "```"), quiet = TRUE)
[1] "<!--html_preserve--><h1>Hello</h1><!--/html_preserve-->"

# with htmltools.preserve.raw = TRUE set by default by rmarkdown
> withr::with_options(list(htmltools.preserve.raw = TRUE), 
+                     cat(knitr::knit(text = c("```{r, echo = FALSE}", "htmltools::HTML('<h1>Hello</h1>\n<p>Content</p>')", "```"), quiet = TRUE))
+                     )

```{=html}
<h1>Hello</h1>
<p>Content</p>
```

You can find above the specific syntax you observed.

As said above, I believe the conflict comes from the fact that knit_child() is called to create the content of HTML(). As the child document will contained some shiny code, with HTML content, htmlPreserve() will also be called as part of htmltools::knit_print.html methods to create the Markdown content, that is usually pass to Pandoc.

Lets look at this by rendering one of your child from another dummy main Rmd doc

---
title: "test"
output: html_document
---

```{r, comment=""}
library(shiny)
cat(knitr::knit_child('child1.Rmd', quiet = TRUE))
```

The last chunk will output this:

Here is some text describing the chart below

And this is some more text

```{=html}
<div class="container-fluid">
<div class="row">
<div class="col-sm-6">
<div id="outb494fc4cc8988858" class="shiny-plot-output" style="width:100%;height:400px;"></div>
</div>
</div>
</div>
```

As you can see, the raw block bits added for HTML preservation are also added as part of the knit_child() rendering.

If we combine the two together, we will have a double preservation by raw block meaning only the outer one will be processed as such by Pandoc

```{=html}
this will be treated as HTML by pandoc, will the outer backticks removed.

```{=html}
And this all block with line above and below will be kept
```
```

We can emulate what happens with this

• test.Rmd content

---
title: "test"
output: html_document
---

```{r, comment=""}
library(shiny)
HTML(knitr::knit_child('child1.Rmd', quiet = TRUE))
```

• R code to render

withr::with_options(list(htmltools.preserve.raw = TRUE), 
                    cat(knitr::knit("test.Rmd", quiet = TRUE))
                   )
xfun::file_string("test.md")

• Resulting test.md file

---
title: "test"
output: html_document
---


```r
library(shiny)
HTML(knitr::knit_child('child1.Rmd', quiet = TRUE))
```

```{=html}









Here is some text describing the chart below

And this is some more text

```{=html}
<div class="container-fluid">
<div class="row">
<div class="col-sm-6">
<div id="outa01a12c987172259" class="shiny-plot-output" style="width:100%;height:400px;"></div>
</div>
</div>
</div>
```
```

The nested raw blocks can be seen above.

For me, this explains why you see the inner raw block backticks syntax in your output - they are kept because they are inside of a raw block which tells pandoc to process the content as raw HTML.

I think this issue needs to be solve by rethinking the way you split you content and put in back together. If you want to use child document to use HTML() you need to have HTML and not markdown.
I don't think knit_child() can be used this way, and probably interactive Rmd document with runtime shiny cannot be constructed this way.

I am not a shiny expert so I don't have a correct solution right now. If we want to support this, we need to think more about how this could be done (and possibly have different way to create shiny apps from Rmd that allow splitting content of the UI in several files.)

Hope it helps understand.

[cderv]

I look into what shiny offers to include Markdown in an app.

There are

• htmltools::includeMarkdown() which takes a md path and use the markdown package to convert to HTML fragment
• shiny 1.5 as a new shiny::markdown() function which will use commonmark package to convert md strings to HTML

None of the above uses Pandoc, so the raw block syntax will not be support

```{=html}
content
```

Building on the above, and using simple Rmd content, this should work

---
output:
  html_document
runtime: shiny
---

```{r, include = FALSE}
render_child <- function(path) {
  withr::local_options(list(htmltools.preserve.raw = FALSE))
  markdown(knitr::knit_child(path, quiet = TRUE))
}
```


```{r, echo = FALSE}
library(ggplot2)
tabsetPanel(
  type = "tabs", id = "x",
  tabPanel("child1",
    value = "y",
    HTML(render_child("child1.Rmd"))
  ),
  tabPanel("child2",
    value = "z",
    HTML(render_child("child2.Rmd"))
  )
)
```

```

• htmltools option htmltools.preserve.raw needs to be set to false
• knit_child() will render to Markdown not using the Pandoc syntax
• shiny::markdown() will convert the markdown to HTML
• HTML() will preserve this HTML content in the main Rmd doc

I also experimented with using rmarkdown directly to convert the result of knit_child(). This would also work

---
output:
  html_document
runtime: shiny
---

```{r, include = FALSE}
render_child <- function(path) {
  tmp <- tempfile(fileext = ".md")
  xfun::write_utf8(knitr::knit_child(path, quiet = TRUE), tmp)
  res <- rmarkdown::render(tmp, 'html_fragment', quiet = TRUE) # should we add `envir = knitr::knit_global()` ?
  xfun::read_utf8(res)
}
```


```{r, echo = FALSE}
library(ggplot2)
tabsetPanel(
  type = "tabs", id = "x",
  tabPanel("child1",
    value = "y",
    HTML(render_child("child1.Rmd"))
  ),
  tabPanel("child2",
    value = "z",
    HTML(render_child("child2.Rmd"))
  )
)
```

• knit_child() would render the child document in the current knitting context as a child doc.
• The result is written back to a temp .md file
• This file is then render to HTML by rmarkdown::render(). This means Pandoc will be used to HTML conversion. The file won't be reprocessed by knitr because it is already a .md file. The output format used is html_fragment() so that we don't have a full HTML doc to include.
• The resulting .html file is read and passed to HTML() to be added as-is in the shiny app.

The above solution are only experiment from me to understand better how this could / should work. They could work for anyone having this issue but there may be limitation, side effects, conflict, or anything happening. I think it would highly depend on what is inside the child document.

Anyway, this is useful to understand better all this child document behavior.