Output directory in quarto-cli

[adomingues]

(this discussion started in the Rstudio community but I got asked to move it here)

I am trying quarto as a replacement for my Rmd-based rmarkdown::render workflow to render reports into specific folders and encountered what I think is a missing feature in quarto. The usual project structure has the Rmd files in a scripts folder and then I render them to specific results folders :

project_base:

• scripts/
  • hello.Rmd
• results/
  • reports/
    • hello.html

I do this with:

rmarkdown::render(
  "scripts/hello.Rmd",
  output_dir = "results/reports"
)

this is particularly useful when rendering separate parametrized reports for either different datasets or arguments:

for (word in c("great", "lovely", "tea")){
	rmarkdown::render(
	  "scripts/hello.Rmd",
	  params(
	  	word = word
	  )
	  output_dir = glue::glue("results/reports/{word}")
	)
}

Thus generating the files:

• scripts/
  • hello.Rmd
• results/
  • reports/
    • great/hello.html
    • lovely/hello.html
    • tea/hello.html

Very flexible and part of my daily set-up.

Trying to recreate this in quarto-cli with a simple example, I thought thought the way to go was the option --output-dir:

quarto render scripts/hello.qmd --output-dir report/

However this places all outputs in the same directory as the qmd file (scripts/) and not in the output-dir:

ls scripts/
hello_files  hello.html  hello.qmd

A few things I tried but did not mimic the expected behaviour:

• created a project with quarto create and then rendered the script with quarto render scripts/hello.qmd --output-dir results. It sort of works, but the output is sent to "results/scripts/"
• post-render: xcopy *_files output is not a solution beacuse it would have to be reweiteen for every loop iteration in the example above.

Christophe Dervieux mentioned that this is not planned feature and so I would like to get the discussion started.

Cheers-

[mcanouil]

Define a _quarto.ymlat the root of your project following https://quarto.org/docs/projects/quarto-projects.html#shared-metadata.
In your case, it should be something like:

project:
  output-dir: results/reports/

There is also the directory in which the execution happens, by default this is where the Quarto document lives, see https://quarto.org/docs/projects/code-execution.html#working-dir.

Also you could use the quarto R package and do all in R as you did for Rmarkdown.
For example, https://github.com/mcanouil/publishing-templates/blob/main/_build.R.

[adomingues]

Thanks @mcanouil. I had already tried the _quarto.yml suggestion and (1) the output is not what is expected and (2) does not allow easy looping to generate multiple reports. Output:

quarto render scripts/hello.qmd
(..)
Output created: ../results/reports/scripts/hello.html

As for using the quarto R package, I guess I could, but is there any advantage over using rmarkdown::render? The only reason for me to try quarto was to compile documents from the command-line, which would simplify some operations (and also to advocate to my python colleagues). If I need to run R -e "quarto::quarto_render" it doesn't make sense to switch from rmarkdown - unless I am missing something.

[mcanouil]

Well, ok the for the CLI, although would the Python users want to use a Python interface (https://github.com/quarto-dev/quarto-python) rather than switching back to shell/bash or whatever?

[famuvie]

I strongly support this request.
I'd love to switch to quarto, with such many awesome features. However, it's disappointingly difficult to keep input and output separate.

I have a solid data-analysis workflow based on targets, and there is some support for rendering quarto reports.
However, there is no way to render reports on some output directory (e.g. for publishing on the web).

Resorting to some ad-hoc code for copying over the resulting files is cumbersome, error prone and breaks the workflow and the chain of dependencies in the pipeline.

Best,

[massisenergy]

I second this! Best things is that quarto(-cli) can greatly simplify the rendering, as well as give tremendous control in a single line – which knit couldn't provide IMO. An option such as --output-dir is very much needed to achieve this goal!

[djhurio]

I second this! In my case the process rendering qmd files do not have write access to the script folder (only read). So it is not able to render the qmd files in the script folder.

The current workaround is to:

• copy qmd file to the output folder
• render the qmd file in the output folder (remember to set the correct working directory with the execute_dir argument in the quarto::quarto_render function, otherwise it will be changed by default to the output folder)
• delete qmd file from the output folder

Hope this helps.

[benscarlson]

A more flexible option to specify the output directory would be really useful. I render a lot of ad-hoc reports that are stored in different parts of my code base, and I want the output to be stored in various sub-folders that I use to organize my results. Right now I use some minor shell scripting to put the results where I want them, but this would be greatly simplified if there were a working option to set the output directory.

qmd=$src/reports/my_report.qmd
out=output_dir/reports/my_report.html

mkdir -p ${out%/*}

quarto render $qmd
mv ${qmd%.*}.html $out
open $out

[benscarlson]

Hello mcanouil. When I attempt to use --output-dir as you mention it does not seem to have any effect. The final report remains in the same directory as the qmd file and not in the folder specified by --output-dir.

I output reports from the same project into different folders in an ad-hoc manner, so I can't use a solution like _quarto.yml, since it seems that will set one output directory for all reports.

It would be great to be able to specify an arbitrary output directory at the command line (I suspect that is what most people think --output-dir does in the first place).

[mcanouil]

I don't know what's your setup, but it works just fine.

Edit: To be accurate, it works only as long as you do run quarto without parent directories of the Quarto document you try to render and if the quarto document lives alongside _quarto.yml.

[blabbree]

I have been having a similar issue. I can specify an output directory child directory, but I have not gotten the output directory to be entirely at a different location. (For example I can save it to C:/~/child, but not to X:/shared_dir).

In my case, I am trying to generate a report on my local machine and output it to a shared drive for my team.

[cderv]

I can specify an output directory child directory, but I have not gotten the output directory to be entirely at a different location. (For example I can save it to C:/~/child, but not to X:/shared_dir).

You can't do that. Quarto we'll handle output-dir as a child directory for project.

You need to have another step after your quarto render to take the whole output folder to move it in its final destination. In a sense, this is what happens when you need to publish it. So perfectly fine to move the output folder on another step, as a publishing step

[d33bs]

I also am seeking the capability mentioned by others in this thread - to specify an output-dir which is "above" the directory where a .qmd or similar file is located (using a relative path such as ../../some-other-dir). Has it already been decided this won't be something that Quarto will implement? If not, would it be worth opening feature / enhancement issue to help develop?

[jhelvy]

On a related note, I'm having a similar issue using quarto::quarto_render(). If I specify the output_file to be a relative or absolute path somewhere, it won't render.

For example, this will work:

quarto::quarto_render(
    input = "document.qmd",
    output_file = "template.pdf"
)

But this will produce an error:

quarto::quarto_render(
    input = "document.qmd",
    output_file = file.path("pdf", "template.pdf")
)

ERROR: --output option cannot specify a relative or absolute path
Error in `processx::run(quarto_bin, args, echo = TRUE)`:
! System command 'quarto' failed
---
Exit status: 1
stdout & stderr: <printed>
---

This is really unfortunate because one of the only reasons I might use quarto::quarto_render() is when I'm batch-rendering a .qmd file with parameters, in which case I'd like to render the resulting set of PDFs into a separate folder.

Edit: A work around is to simply render these and then add another line to move the files into a folder, which is easy enough, but I still would expect that I should be able to render them into a sub folder.

[asadow]

my_quarto_render <- function(input, output_file) {
    output_file_name <- basename(output_file)
    # First render to the current working directory
    quarto::quarto_render(input = input, output_file = output_file_name)
    # Now copy it to the desired path and delete the original file
    file.copy(from = output_file_name, to = output_file)
    file.remove(output_file_name)
}

Unfortunately this isn't working for me. I get repeat warnings of "Could not fetch resource [my-qmd-name]__files/libs/quarto-html/[random-css-and-js-file.js/css]"

[jhelvy]

Yeah I posted my original function prematurely. After further testing and improving it, I came up with quarto_render_move(), which I export in my personal R package jph.

[asadow]

Yeah I posted my original function prematurely. After further testing and improving it, I came up with quarto_render_move(), which I export in my personal R package jph.

That was quick - thank you! Some notes: it errors if input is specified using here() due to quarto_inspect(). Also will error if output_file is NULL, output_path has the full file path, and its file name already exists at this path.

[jhelvy]

Thanks! Good feedback. It's not perfect. I played around with different ways to try to handle every situation, but stopped once it addressed most of my use cases. At some point the Quarto team may take this on, so I'm going to leave it to them (or if anyone else finds a better solution).

[jrosell]

I'm not 100% happy, but here another workaround I have found.

You can run a render.R script with this contents:

render_penguin_reports = function() {
    unique(palmerpenguins::penguins$species) |>
        as.character() |>
        walk(render_penguin_report)
}

render_penguin_report = function(x) {
    output_file <- glue::glue("penguins-{x}.html")
    final_output_file <- here::here("results", "reports", output_file)
    setwd(here::here("scripts"))
    quarto::quarto_render(
        input = "penguins.qmd",
        execute_params = list(species = x),
        output_file = output_file
    )
    file.copy(from = output_file, to = final_output_file)
    file.remove(output_file)
    setwd(here::here())
}

render_penguin_reports()

My quarto document YAML:

---

title: "Penguin reports"
editor: source
date: today
params:
species: Adelie
editor_options:
chunk_output_type: console
format:
html:
embed-resources: true

[mcanouil]

For anyone wondering about this issue, here is an update as of Quarto CLI v1.5.23 (pre-release), about using --output-dir.
TL;DR: It is behaving exactly as initially described when rendering from a sub-directory in something that isn't a book or a website.

---

Quarto CLI v1.5.23

By default, for simple document, you actually can't use --output-dir:
No longer true using v1.5.23.

mkdir demo
echo "---\ntitle: "Untitled"\nformat: html\nembed-resources: true\n---" > demo/demo.qmd
quarto render demo/demo.qmd  --output-dir output

tree
#> .
#> └── demo
#>     ├── demo.qmd
#>     └── output
#>         └── demo.html
#> 
#> 3 directories, 2 files

But, for many reasons you can end up with a _quarto.yml which turns your directory into a "Quarto project", thus allowing you (the user) to use --output-dir.
This leads to a "weird behaviour" where the parent directory/ies of the Quarto document gets added to the output directory.

touch _quarto.yml
mkdir demo
echo "---\ntitle: "Untitled"\nformat: html\nembed-resources: true\n---" > demo/demo.qmd
quarto render demo/demo.qmd  --output-dir output

tree
#> .
#> ├── _quarto.yml
#> ├── demo
#> │   └── demo.qmd
#> └── output
#>     └── demo
#>         └── demo.html
#> 
#> 4 directories, 3 files

Note that trying to render from a sub-directory of a Quarto project will still "fail" as it will have the same result as the example above (kind of expected since Quarto resolves the root using _quarto.yml).

touch _quarto.yml
mkdir demo
echo "---\ntitle: "Untitled"\nformat: html\nembed-resources: true\n---" > demo/demo.qmd
cd demo
quarto render demo.qmd  --output-dir output

tree ../
#> ../
#> ├── _quarto.yml
#> ├── demo
#> │   └── demo.qmd
#> └── output
#>     └── demo
#>         └── demo.html
#> 
#> 4 directories, 3 files

[davidzgli]

Is it possible to just put demo.html under output directory, i.e., output/demo.html, not under output/demo, i.e., output/demo/demo.html? Also, is it possible to render a qmd file that is not in current working directory. Let's say I am in a directory named analysis. It has qmd file ./demo/demo.qmd. Is it possible to run the command quarto render demo/demo.qmd --output-dir output?

[mcanouil]

No for "simple project" right now as illustrated in my message.
(Opening a new discussion #8846 won't change that fact unfortunately.)
Quarto team is planning to improve this in the future, meanwhile, you'll have to use a post-render script to copy/move the rendered file.

[davidzgli]

Got it. I will stay with file-moving scripts after rendering the qmd files. Hope Quarto team will implement this feature soon. Thank you!