keep_md option not used by blogdown::html_page

[lazappi]

Hi

I have noticed that blogdown::html_page() does not make use of the keep_md option. For example this YAML frontmatter does not result in a .md file as well as the .html file.

output:
  blogdown::html_page:
    keep_md: true

I believe this is related to this rmarkdown issue rstudio/rmarkdown#1558.

Not sure if this is the desired behaviour or not but there are two workarounds I have come up with.

Option 1 is create a new output format that checks for keep_md and passes other options on to blogdown::html_page().

html_page_with_md <- function(..., number_sections = FALSE,
                                self_contained = FALSE, highlight = NULL,
                                template = NULL, pre_knit = NULL,
                                post_processor = NULL) {
    opts <- list(...)
    keep_md <- ifelse(!is.null(opts$keep_md), opts$keep_md, FALSE)

    rmarkdown::output_format(
        knitr       = NULL,
        pandoc      = NULL,
        keep_md     = keep_md,
        base_format = blogdown::html_page(
            ...,
            number_sections = number_sections,
            self_contained  = self_contained,
            highlight       = highlight,
            template        = template,
            pre_knit        = pre_knit,
            post_processor  = post_processor
        )
    )
}

The other option is to use rmarkdown::resolve_output_format() to get the output format and then manually set the keep_md option.

output_format <- rmarkdown::resolve_output_format(input)
output_format$keep_md <- TRUE

I think this is probably the easiest option as long as you always want the .md file. The best solution would be for blogdown::html_page() to pass on the keep_md argument like above (if that is something you want to be able to do).

If there is a better solution please let me know.

xfun::session_info("blogdown") 
                                       
R version 3.6.2 (2019-12-12)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Catalina 10.15.3

Locale: en_GB.UTF-8 / en_GB.UTF-8 / en_GB.UTF-8 / C / en_GB.UTF-8 / en_GB.
UTF-8

Package version:
  base64enc_0.1.3 BH_1.72.0.3     blogdown_0.18   bookdown_0.18  
  digest_0.6.25   evaluate_0.14   glue_1.3.1      graphics_3.6.2 
  grDevices_3.6.2 highr_0.8       htmltools_0.4.0 httpuv_1.5.2   
  jsonlite_1.6.1  knitr_1.28      later_1.0.0     magrittr_1.5   
  markdown_1.1    methods_3.6.2   mime_0.9        promises_1.1.0 
  R6_2.4.1        Rcpp_1.0.3      rlang_0.4.5     rmarkdown_2.1  
  servr_0.16      stats_3.6.2     stringi_1.4.6   stringr_1.4.0  
  tinytex_0.20    tools_3.6.2     utils_3.6.2     xfun_0.12      
  yaml_2.2.1

---

By filing an issue to this repo, I promise that

• I have fully read the issue guide at https://yihui.org/issue/.
• I have provided the necessary information about my issue.
  • If I'm asking a question, I have already asked it on Stack Overflow or RStudio Community, waited for at least 24 hours, and included a link to my question there.
  • If I'm filing a bug report, I have included a minimal, self-contained, and reproducible example, and have also included xfun::session_info('blogdown'). I have upgraded all my packages to their latest versions (e.g., R, RStudio, and R packages), and also tried the development version: remotes::install_github('rstudio/blogdown').
  • If I have posted the same issue elsewhere, I have also mentioned it in this issue.
• I have learned the Github Markdown syntax, and formatted my issue correctly.

[yihui]

Thanks a lot for the investigation! I'm curious why you need the intermediate .md file, though. I can't think how it could be useful.

[lazappi]

I have been playing around with a custom build script that caches the .md file and knits from that if the .Rmd hasn't been updated more recently. The idea was to preserve the code output in case there was anything time dependent that might change if the everything was run again. From some of your comments in other places this might not be necessary but I did learn a lot about how {blogdown} works in the process 😸. The script is here if you are interested https://github.com/lazappi/lazappi_blog/blob/master/R/build.R and I was planning to write a blog post explaining it (mostly so I don't forget).

[yihui]

Thanks for the explanation! If you need the .md for the caching purpose (by comparing the timestamps of .md and .Rmd?), I guess the .html file might serve the same purpose. No?

[lazappi]

I guess that I want to cache the output of any code chunks so that I can avoid any problems with re-running the code in the future. You could probably get that from the HTML I suppose but not as easily. The blog post I wrote about it is here if you are interested https://lazappi.id.au/post/2020-04-09-caching-blogdown/. There are probably some things I don't fully understand so happy to hear feedback/corrections if you have any.

[cderv]

Great blog post !
I am just curious about something you said

One thing that blogdown does which isn’t necessarily desirable is re-knit every R Markdown document whenever the site is built.

I understand what you are looking for is the implicit caching mechanism blogdown offer with local = TRUE but for building the website. This parts in the book mentions it : https://bookdown.org/yihui/blogdown/local-preview.html#local-preview

Did you try any of the solution mentioned ?

• Turn on knitr’s caching for time-consuming code chunks
• using blogdown::hugo_build() instead of blogdown::build_site() ?

I wonder what are your experience on those ? Thanks !

[lazappi]

@cderv I haven't really tried either of those, I probably rushed into this without researching properly what the options were.

That said I'm not sure either of those is ideal for what I wanted to do. Caching would need to be turned on for every chunk and it is harder to maintain and transfer than a .md file which can be tracked using git. The blogdown::hugo_build() option might work but I would need to make sure the .Rmd files don't need to be re-rendered and I don't think I trust myself to keep track of that.

(Apologies if this discussion has strayed too far from the original issue)

[giocomai]

A new package, Hugodown, still marked as experimental, seems to address the main concerns raised by @lazappi, in particular in reference to keeping md files, and rendering from .Rmd only when the user explicity requests it.

[yihui]

The next version of blogdown will address this problem and allow you to re-render only the Rmd file that you want. Thanks!

[yihui]

@lazappi I just read your blog post, and I feel there might be some misunderstanding. As @cderv quoted:

One thing that blogdown does which isn’t necessarily desirable is re-knit every R Markdown document whenever the site is built.

This is only true for blogdown::build_site(), but you'd almost never need to call this function, because it re-renders all Rmd posts. If you only want to re-render the posts that you have modified, you need blogdown::build_site(local = TRUE), which is what blogdown::serve_site() does under the hood.

That's why I didn't even mention build_site() at all in the workflow, but highlighted blogdown::serve_site() instead: https://bookdown.org/yihui/blogdown/workflow.html

The approach that you mentioned in your blog post is essentially exactly what build_site(local = TRUE) does: build_site(local = TRUE) compares the timestamps of the .html output file and the .Rmd source file, and renders the .Rmd file only if its modification time is newer than .html. As @cderv mentioned above, this is explained in the blogdown book: https://bookdown.org/yihui/blogdown/local-preview.html If I understand your post correctly, you were comparing the timestamps of .md.cached and .Rmd. That's why I originally asked if .html could serve the same purpose.

Anyway, I still don't think the option keep_md: true could be practically useful, but I did add it to blogdown::html_page. Your original request has been fulfilled. Thanks!

BTW, if you have both .md and .html output files, you probably need to tell Hugo to ignore one of them (use ignoreFiles in config.toml/.yaml), otherwise Hugo might render both, and one will override the other (I'm not sure about the precedence).

I'm push another commit to provide an option for blogdown not to re-render any Rmd files at all, unless users explicitly click the Knit button.

[yihui]

With the current development version (remotes::install_github('rstudio/blogdown')), build_site() no longer renders any Rmd files at all (unless you explicitly ask for it; see help page ?blogdown::build_site). The Knit button will just work.

(cc @apreshill)

[apreshill]

Re: precedence, I cannot find Hugo documentation now on this, but I believe if there is an .html file in a bundle, that will take precedence over an .md file when the site is built.