Handle `application/pdf` mimetypes #136

MichaelHatherly · 2024-05-23T13:12:00Z

No description provided.

codecov · 2024-05-23T13:37:26Z

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 56.30%. Comparing base (224d278) to head (21bb813).

Files	Patch %	Lines
...NotebookWorker/ext/QuartoNotebookWorkerPlotsExt.jl	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #136      +/-   ##
==========================================
+ Coverage   55.98%   56.30%   +0.32%     
==========================================
  Files          25       25              
  Lines        1145     1142       -3     
==========================================
+ Hits          641      643       +2     
+ Misses        504      499       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

MichaelHatherly · 2024-05-23T14:40:28Z

src/QuartoNotebookWorker/ext/QuartoNotebookWorkerPlotsExt.jl

-    else
-        Plots.gr(; size_kwargs..., fmt = fm.fig_format, dpi_kwargs...)
-    end
+    Plots.gr(; size_kwargs..., dpi_kwargs...)


Format wasn't really relevant to pass here, since we call show on each valid mimetype rather than relying on the default format set here. The addition of application/pdf support seems to resolve this.

Yeah that's what I gathered from the linked Plots code as well. I'm not sure what backends would actually use pdf assets if we offer them, maybe latex-based ones.

MichaelHatherly · 2024-05-23T14:46:35Z

@lrnv if you have a chance to confirm this resolves it for you that would be good.

To point quarto render at this branch you can

clone this branch
instantiate and precompile it
set QUARTO_JULIA_PROJECT environment variable to that path of the clone so that it uses that runner rather than the quarto-managed one
run quarto render file.qmd

lrnv · 2024-05-23T17:15:24Z

I was able to install QNR, dev it and move the deved verion to this branch, but not to tell quarto to use it. Could you guide me a bit more into the environement thing ? I tried using a _enrivonement file but that did not work.

MichaelHatherly · 2024-05-23T17:29:24Z

The below should hopefully work for you. QUARTO_JULIA_PROJECT=/Users/mike/personal/QuartoNotebookRunner.jl/ should be set to whatever path you cloned this repo to.

~/personal 
❯ git clone --branch mh/plots-pdf https://github.com/PumasAI/QuartoNotebookRunner.jl^C

~/personal 
❯ cd QuartoNotebookRunner.jl/

QuartoNotebookRunner.jl on  mh/plots-pdf [?] is 📦 v0.10.2 via ஃ v1.10.3 via ⨁ v1.5.39 took 4s 
❯ julia --project -e 'import Pkg; Pkg.instantiate(); Pkg.precompile()'

QuartoNotebookRunner.jl on  mh/plots-pdf [?] is 📦 v0.10.2 via ஃ v1.10.3 via ⨁ v1.5.39 
❯ QUARTO_JULIA_PROJECT=/Users/mike/personal/QuartoNotebookRunner.jl/ quarto render PATH/TO/FILE.qmd
Running [1/1] at line 17:  using Random, Plots
pandoc 
  to: beamer
  output-file: file.tex
  standalone: true
  pdf-engine: xelatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf
  
metadata
  block-headings: true
  title: title
  julia:
    exeflags:
      - '--project=plots'
  

Rendering PDF
running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999995 (TeX Live 2023) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode
  
running xelatex - 2
  This is XeTeX, Version 3.141592653-2.6-0.999995 (TeX Live 2023) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode
  

Output created: file.pdf

lrnv · 2024-05-23T17:39:24Z

i'm sorry I was still not able to reproduce my previous result, now it complains about not having Plots.jl in its environement although it is (in the local env of the qmd)..

MichaelHatherly · 2024-05-23T17:42:41Z

Ah, the file.qmd I was using had explicitly set the project, see https://github.com/PumasAI/QuartoNotebookRunner.jl/pull/136/files#diff-e002873e0c60e777197ce755872fc8a5d32041b7dd3b21062807548c4ef1f85dR7-R8 for how to do that. Set it to whatever environment has Plots installed.

(The entire error message you're seeing would be useful though.)

lrnv · 2024-05-23T17:49:37Z

Ok it finnaly worked, but it looks like I have the sam ebug as before :

lrnv@laptop slides_juliacon2024 $ QUARTO_JULIA_PROJECT="C:\\Users\\lrnv\\.julia\\dev\\QuartoNotebookRunner" quarto render examples.qmd
Running [1/1] at line 17:  using Random, Plots#|
ERROR: Julia server returned error after receiving "run" command:
Failed to run notebook: C:\Users\lrnv\Documents\slides_juliacon2024\examples.qmd       
ERROR: EvaluationError: Encountered 1 error during evaluation

Error 1 of 1
@ C:\Users\lrnv\Documents\slides_juliacon2024\examples.qmd:17
Error showing value of type Plots.Plot{Plots.GRBackend}
only png or svg allowed. got: :pdf
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:35
 [2] _show(io::IOContext{IOBuffer}, ::MIME{Symbol("text/html")}, plt::Plots.Plot{Plots.GRBackend})
   @ Plots C:\Users\lrnv\.julia\packages\Plots\ju9dp\src\output.jl:203
 [3] #invokelatest#2
   @ .\essentials.jl:887 [inlined]
 [4] invokelatest
   @ .\essentials.jl:884 [inlined]
 [5] show(io::IOContext{IOBuffer}, m::MIME{Symbol("text/html")}, plt::Plots.Plot{Plots.GRBackend})
   @ Plots C:\Users\lrnv\.julia\packages\Plots\ju9dp\src\output.jl:232
 [6] show(io::IOContext{IOBuffer}, m::String, x::Plots.Plot{Plots.GRBackend})
   @ Base.Multimedia .\multimedia.jl:123

ERROR: Internal julia server error

Stack trace:
    at writeJuliaCommand (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:41666:19)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async executeJulia (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:41575:22)
    at async Object.execute (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:41326:20)
    at async renderExecute (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:77864:27)
    at async renderFileInternal (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:78032:43)
    at async renderFiles (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:77900:17)
    at async render (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:82728:21)
    at async Command.actionHandler (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:82876:32)
    at async Command.execute (file:///C:/Users/lrnv/AppData/Local/Programs/Quarto/bin/quarto.js:8015:13)
lrnv@laptop slides_juliacon2024 $

MichaelHatherly · 2024-05-23T17:55:13Z

Hmm, lets confirm that that is actually using this branch.

Can you swap out the contents of the notebook with

```{julia}
names(Main)
```

# Intro

## Slide working 
bla bla 

## Slide not working 

```{julia}
#| echo: false
#| fig-cap: "my caption"
# using Random, Plots
# x = rand(2, 1000)
# scatter(x[1, :], x[2, :])
```

I'm interested in what is printed out by the names(Main) cell.

lrnv · 2024-05-23T17:58:27Z

prints out

59-element Vector{Symbol}:
:BUFFER_SIZE
:Base
:Core
:IOCapture
:InlineDisplay
:MIMETYPE_WRAPPERS
:MSG_BOUNDARY
:Main
:MsgID
:MsgType
�
:main
:ojs_define
:pkg_hook_creator
:refresh!
:render
:render_mimetypes
:serve
:with_context
:with_inline_display

lrnv · 2024-05-23T18:00:34Z

All of them :

names(Main)[1:20]
20-element Vector{Symbol}:
:BUFFER_SIZE
:Base
:Core
:IOCapture
:InlineDisplay
:MIMETYPE_WRAPPERS
:MSG_BOUNDARY
:Main
:MsgID
:MsgType
:Notebook
:NotebookInclude
:OPTIONS
:PACKAGE_LOADING_HOOKS
:PACKAGE_REFRESH_HOOKS
:PKG_VERSIONS
:PNG
:POST_ERROR_HOOKS
:POST_EVAL_HOOKS
:PROJECT


names(Main)[21:40]
20-element Vector{Symbol}:
:PlotlyBasePlot
:PlotlyJSSyncPlot
:RCall_temp_files
:SVG
:WrapperType
:_CairoMakie_hook
:_Makie_hook
:_PlotlyBase_hook
:_PlotlyJS_hook
:_Plots_hook
:_RCall_hook
:_RCall_refresh_hook
:_buffer_writes
:_channel_cache
:_discard_until_boundary
:_figure_metadata
:_flatmap
:_getproperty
:_mimetype_wrapper
:_parseall


names(Main)[41:end]
19-element Vector{Symbol}:
:_pkg_version
:_register_wrapper
:_render_thunk
:_serialize_msg
:_transform_output
:clean_bt_str
:format_error
:handle
:include_str
:interrupt
:main
:ojs_define
:pkg_hook_creator
:refresh!
:render
:render_mimetypes
:serve
:with_context
:with_inline_display

MichaelHatherly · 2024-05-23T18:02:03Z

Thanks. quarto isn't using this branch, I'd expect that printout to not contain IOCapture, or many of the others. It appears to be somehow still using the quarto-managed environment...

What shell are you using? Setting the environment variable with the bash syntax might not be working.

lrnv · 2024-05-23T18:04:01Z

It's git-bash on windows, usually quite compliant with linux bash

MichaelHatherly · 2024-05-23T18:05:53Z

Let me see if I can find a more reliable way to make this kind of stuff testable. I'll ping you once I've worked that out :) probably won't be til tomorrow, thanks for taking the time to try this now though, much appreciated.

lrnv · 2024-05-23T18:07:29Z

Nevermind, thnaks for taking care of this -- this bug is basically making the whole engine unusable for me, and i have juliacon slides to write ;)

jkrumbiegel · 2024-05-24T06:27:17Z

src/QuartoNotebookWorker/ext/QuartoNotebookWorkerPlotsExt.jl

-    else
-        Plots.gr(; size_kwargs..., fmt = fm.fig_format, dpi_kwargs...)
-    end
+    Plots.gr(; size_kwargs..., dpi_kwargs...)


Yeah that's what I gathered from the linked Plots code as well. I'm not sure what backends would actually use pdf assets if we offer them, maybe latex-based ones.

Handle application/pdf mimetypes

21bb813

MichaelHatherly commented May 23, 2024

View reviewed changes

MichaelHatherly linked an issue May 23, 2024 that may be closed by this pull request

Pdf plotting is broken (worked on iJulia) #128

Closed

MichaelHatherly marked this pull request as ready for review May 23, 2024 14:41

MichaelHatherly requested a review from jkrumbiegel May 23, 2024 21:00

MichaelHatherly mentioned this pull request May 23, 2024

Add test_pr for testing PRs #137

Merged

jkrumbiegel approved these changes May 24, 2024

View reviewed changes

MichaelHatherly merged commit 65ab142 into main May 24, 2024
9 of 10 checks passed

MichaelHatherly deleted the mh/plots-pdf branch May 24, 2024 06:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle `application/pdf` mimetypes #136

Handle `application/pdf` mimetypes #136

MichaelHatherly commented May 23, 2024

codecov bot commented May 23, 2024 •

edited

Loading

MichaelHatherly May 23, 2024

jkrumbiegel May 24, 2024

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024 •

edited

Loading

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024

MichaelHatherly commented May 23, 2024 •

edited

Loading

lrnv commented May 23, 2024

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024

lrnv commented May 23, 2024 •

edited

Loading

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024 •

edited

Loading

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024

jkrumbiegel May 24, 2024

Handle application/pdf mimetypes #136

Handle application/pdf mimetypes #136

Conversation

MichaelHatherly commented May 23, 2024

codecov bot commented May 23, 2024 • edited Loading

Codecov Report

MichaelHatherly May 23, 2024

Choose a reason for hiding this comment

jkrumbiegel May 24, 2024

Choose a reason for hiding this comment

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024 • edited Loading

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024

MichaelHatherly commented May 23, 2024 • edited Loading

lrnv commented May 23, 2024

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024

lrnv commented May 23, 2024 • edited Loading

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024 • edited Loading

MichaelHatherly commented May 23, 2024

lrnv commented May 23, 2024

jkrumbiegel May 24, 2024

Choose a reason for hiding this comment

Handle `application/pdf` mimetypes #136

Handle `application/pdf` mimetypes #136

codecov bot commented May 23, 2024 •

edited

Loading

lrnv commented May 23, 2024 •

edited

Loading

MichaelHatherly commented May 23, 2024 •

edited

Loading

lrnv commented May 23, 2024 •

edited

Loading

lrnv commented May 23, 2024 •

edited

Loading