Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quarto v1.5 installs a nonexistent package hyphen-chinese-hans when compiling PDF #10291

Closed
leovan opened this issue Jul 14, 2024 · 35 comments · Fixed by #10614
Closed

Quarto v1.5 installs a nonexistent package hyphen-chinese-hans when compiling PDF #10291

leovan opened this issue Jul 14, 2024 · 35 comments · Fixed by #10614
Assignees
Labels
bug Something isn't working latex LaTeX engines related libraries and technologies
Milestone

Comments

@leovan
Copy link

leovan commented Jul 14, 2024

Bug description

For a book type project with lang: zh, Quarto v1.5 installs a nonexistent package hyphen-chinese-hans when compiling PDF. hyphen-chinese is a valid package, but hyphen-chinese-hans is not. Quarto v1.4 works well. Error output is shown as below:

Rendering PDF
running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode
  
> 1 package to install
> installing hyphen-chinese-hans (1 of 1)
ERROR: tlmgr returned a non zero status code
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.


Stack trace:
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

    at Object.onComplete (file:///Applications/quarto/bin/quarto.js:92847:64)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async renderFiles (file:///Applications/quarto/bin/quarto.js:78077:16)
    at async renderProject (file:///Applications/quarto/bin/quarto.js:78479:25)
    at async Command.actionHandler (file:///Applications/quarto/bin/quarto.js:83077:32)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8017:13)
    at async Command.parseCommand (file:///Applications/quarto/bin/quarto.js:7907:20)
    at async quarto (file:///Applications/quarto/bin/quarto.js:118219:9)
    at async file:///Applications/quarto/bin/quarto.js:118238:9
    at async mainRunner (file:///Applications/quarto/bin/quarto.js:118123:9)

Steps to reproduce

Repo: https://github.com/leovan/quarto-issue

Expected behavior

No response

Actual behavior

No response

Your environment

No response

Quarto check output

Quarto 1.5.54
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.5.54
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /Library/TeX/texbin
      Version: 2024

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.10.14
      Path: /Users/leo/Applications/Python/python310-dev/bin/python
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with python3 -m pip install jupyter

[✓] Checking R installation...........OK
      Version: 4.4.1
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library
      knitr: 1.47
      rmarkdown: 2.27

[✓] Checking Knitr engine render......OK
@leovan leovan added the bug Something isn't working label Jul 14, 2024
@mcanouil
Copy link
Collaborator

mcanouil commented Jul 14, 2024

@leovan Thanks for the report. unfortunately we currently can't use your report as you did not provide us with the means to reproduce the issue as requested in the template.
Please update your report (by editing it) with the required information. This includes your environment.
Thank you for your cooperation.

@mcanouil mcanouil added the needs-repro Issues that are blocked until reporter provides an adequate reproduction label Jul 14, 2024
@leovan
Copy link
Author

leovan commented Jul 14, 2024

I've added a minimum repo to reproduce the issue: https://github.com/leovan/quarto-issue

@mcanouil
Copy link
Collaborator

Thanks!

I can even reproduce with the following:

---
tile: "范叶亮"
format: pdf
lang: zh
---

# 范叶亮

The issue is not in the LaTeX part but in the processing after that to render the document, in particular the automatic package discovery/install.

The change comes from an attempt to fix issues with hyphenation:

The following might need another exception for chinese (in addition to german):

  • //ngerman gets special cased
    const filterLang = (lang: string) => {
    // NOTE Although the names of the corresponding lfd files match those in this list,
    // there are some exceptions, particularly in German and Serbian. So, ngerman is
    // called here german, which is the name in the CLDR and, actually, the most logical.
    //
    // See https://ctan.math.utah.edu/ctan/tex-archive/macros/latex/required/babel/base/babel.pdf
    if (lang === "ngerman") {
    return "hyphen-german";
    }
    return `hyphen-${lang.toLowerCase()}`;
    };

@mcanouil mcanouil added latex LaTeX engines related libraries and technologies and removed needs-repro Issues that are blocked until reporter provides an adequate reproduction labels Jul 14, 2024
@leovan
Copy link
Author

leovan commented Jul 14, 2024

Thanks for checking the issue, look forward to the fix.

@Jay-kkk
Copy link

Jay-kkk commented Jul 15, 2024

I had the same error, look forward to the fix

@leovan
Copy link
Author

leovan commented Jul 15, 2024

I think the quick fix is to make latex-auto-install option (https://quarto.org/docs/output-formats/pdf-engine.html#package-installation) works for install hyphen package too.

if (success) {
// See whether there are warnings about hyphenation
// See (https://github.com/rstudio/tinytex/commit/0f2007426f730a6ed9d45369233c1349a69ddd29)
const logText = Deno.readTextFileSync(response.log);
const missingHyphenationFile = findMissingHyphenationFiles(logText);
if (missingHyphenationFile) {
if (await pkgMgr.installPackages([missingHyphenationFile])) {
// We installed hyphenation files, retry
continue;
} else {
writeError("missing hyphenation file", "", response.log);
return Promise.reject();
}
}
} else if (pkgMgr.autoInstall) {

For line 182, it can be revised to:

   if (missingHyphenationFile && pkgMgr.autoInstall) { 

@mcanouil
Copy link
Collaborator

The suggested change won't change the fact that Quarto tries to determine the hyphen file and will try to install it. Making the auto install will only make sure the correct package is installed but won't correct the bad name so it should lead to the exact same error.

@leovan
Copy link
Author

leovan commented Jul 15, 2024

OK, it seems adding another exception for chinese is a better way to fix this.

@cderv
Copy link
Collaborator

cderv commented Jul 15, 2024

This is indeed a problem of Quarto trying to guess which package needs to be installed when the babel warning is encounter

Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil
(babel)             (\language10). Reported on input line 143.

We build the package name based on value from the log.

hyphen-chinese is a valid package, but hyphen-chinese-hans is not.

Installing hyphen-chinese does not solve the problem as the warning message is still present. So it seems to be another package, or a different file / fonts that would need to be installed when not present already. This is required to add the correct exception.

@cderv cderv self-assigned this Jul 15, 2024
@cderv cderv added this to the v1.6 milestone Jul 15, 2024
@leovan
Copy link
Author

leovan commented Jul 15, 2024

According to https://ctan.math.utah.edu/ctan/tex-archive/macros/latex/required/babel/base/babel.pdf, in section List of locales available in \babelprovide, chinese (zh) is used for:

chinese-simplified (zh-Hans)
chinese-hans (zh-Hans)
chinese-traditional (zh-Hant)
chinese-hant (zh-Hant)
chinese-simplified-hongkongsarchina (zh-Hans-HK)
chinese-hans-hk (zh-Hans-HK)
chinese-simplified-macausarchina (zh-Hans-MO)
chinese-hans-mo (zh-Hans-MO)
chinese-simplified-singapore (zh-Hans-SG)
chinese-hans-sg (zh-Hans-SG)
chinese-hant-hk (zh-Hant-HK)
chinese-traditional-hongkongsarchina (zh-Hant-HK)
chinese-hant-mo (zh-Hant-MO)
chinese-traditional-macausarchina (zh-Hant-MO)

Will the following code works?

if (lang === "ngerman") {
    return "hyphen-german"; 
} else if (lang.startsWith("chinese") {
    return "hyphen-chinese";
}

return `hyphen-${lang.toLowerCase()}`; 

I think this will install hyphen-chinese everytime, becase the error message Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil still exists after installing hyphen-chinese. But it can continue the compling and produce the final PDF.

Really need a quick fix, otherwise Quarto v1.5 won't compile doc with lang: zh to PDF correctly.

@cderv
Copy link
Collaborator

cderv commented Jul 15, 2024

I did try that. But installing hyphen-chinese does not prevent the warning to show in the document, at least in my test.
You could try on your side, by installing the package from CTAN and re-render. If the warning from babel is still in log file, Quarto will still attempt to install it.

@leovan
Copy link
Author

leovan commented Jul 15, 2024

OK, I will try later to see whether we can get a quick fix for this issue.

@cderv
Copy link
Collaborator

cderv commented Jul 15, 2024

hyphen-chinese is a valid package

I can't find it on CTAN. Are you sure this is the name of the package ?

@leovan
Copy link
Author

leovan commented Jul 15, 2024

Yes, cmd output:

❯ sudo tlmgr install hyphen-chinese
Password:
tlmgr: package repository https://mirrors.pku.edu.cn/ctan/systems/texlive/tlnet (verified)
tlmgr install: package already present: hyphen-chinese

And in TeX Live Utility, it is shown installed too.

SCR-20240715-pnbm

@cderv
Copy link
Collaborator

cderv commented Jul 15, 2024

Oh great thanks. This is a sytem package then. It does not solve the warning though, so not sure if we should ask for installation for this warning problem.

Another problem we have is that if this is already install the rendering should not stop... 🤔

@cderv
Copy link
Collaborator

cderv commented Jul 15, 2024

I think the quick fix is to make latex-auto-install option (quarto.org/docs/output-formats/pdf-engine.html) works for install hyphen package too.

@leovan I do think this is the right thing to do here. At least to unblock. Quarto LaTeX engine does auto install missing hyphenation package based on log warnings, but the latex-auto-install configuration does not applied. IMO it should. Thanks for pointing that out !

Quarto v1.5 installs a nonexistent package hyphen-chinese-hans when compiling PDF

Regarding this problem, Quarto does have a recipe to guess which package to install based on log warning. Here, it builds a wrong name so we need an exception. Though even when hyphen-chinese is present, the warning is still there. So we keep trying to install, and at the second time we fail.

And we do prevent PDF rendering for a hyphenation warning in the log, not resolved.

// See whether there are warnings about hyphenation
// See (https://github.com/rstudio/tinytex/commit/0f2007426f730a6ed9d45369233c1349a69ddd29)
const logText = Deno.readTextFileSync(response.log);
const missingHyphenationFile = findMissingHyphenationFiles(logText);
if (missingHyphenationFile) {
if (await pkgMgr.installPackages([missingHyphenationFile])) {
// We installed hyphenation files, retry
continue;
} else {
writeError("missing hyphenation file", "", response.log);
return Promise.reject();
}
}

We could also relax this constraint, and continue the rendering anyway... 🤔

@leovan
Copy link
Author

leovan commented Jul 16, 2024

I made a test, code changes:

diff --git a/src/command/render/latexmk/pdf.ts b/src/command/render/latexmk/pdf.ts
index 52062aa3f..ec9eb0554 100644
--- a/src/command/render/latexmk/pdf.ts
+++ b/src/command/render/latexmk/pdf.ts
@@ -179,7 +179,7 @@ async function initialCompileLatex(
       // See (https://github.com/rstudio/tinytex/commit/0f2007426f730a6ed9d45369233c1349a69ddd29)
       const logText = Deno.readTextFileSync(response.log);
       const missingHyphenationFile = findMissingHyphenationFiles(logText);
-      if (missingHyphenationFile) {
+      if (pkgMgr.autoInstall && missingHyphenationFile) {
         if (await pkgMgr.installPackages([missingHyphenationFile])) {
           // We installed hyphenation files, retry
           continue;

qmd example:

---
tile: "范叶亮"
format: pdf
lang: zh
keep-tex: true
latex-auto-install: false
---

# 范叶亮

Without latex-auto-install: false:

Quarto version: 99.9.9
pandoc --verbose --trace
  to: latex
  output-file: zh.tex
  standalone: true
  pdf-engine: xelatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf

metadata
  documentclass: scrartcl
  classoption:
    - DIV=11
    - numbers=noendperiod
  papersize: letter
  header-includes:
    - '\KOMAoption{captions}{tableheading}'
  block-headings: true
  tile: 范叶亮
  lang: zh

[execProcess] /Users/leo/Documents/Projects/TypeScript/quarto-cli/package/dist/bin/tools/aarch64/pandoc +RTS -K512m -RTS --defaults /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/quarto-defaults3307782351324eff.yml /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/quarto-inputf75c4f14f272bf5c.md --metadata-file /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/quarto-metadata5f0552045cd04d13.yml --verbose --trace --data-dir /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/pandoc/datadir
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/template.patched from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/template.patched
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/doc-class.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/doc-class.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/pandoc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/pandoc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tightlist.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tightlist.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tables.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/tables.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/graphics.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/graphics.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/citations.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/citations.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-body.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/toc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/toc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-bib.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/before-bib.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/biblio.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/biblio.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/after-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionaa09b94676a0f145/5539a4b7f6934776/643eacc3cb17c578/after-body.tex
[trace] Parsed [Plain [Str "DIV=11"]] at line 1
[trace] Parsed [Plain [Str "numbers=noendperiod"]] at line 1
[trace] Parsed [Plain [Str "scrartcl"]] at line 1
[trace] Parsed [Plain [Str "zh-Hans"]] at line 1
[trace] Parsed [Plain [Str "blue"]] at line 1
[trace] Parsed [Plain [Str "letter"]] at line 1
[trace] Parsed [Plain [Str "\33539\21494\20142"]] at line 1
[trace] Parsed [Plain [Str "\30446\24405"]] at line 1
[trace] Parsed [] at line 6
[trace] Parsed [Header 1 ("\33539\21494\20142",[],[]) [Str "\33539\21494\20 at line 9
[INFO] Running filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua
[INFO] Completed filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua in 34 ms
[execProcess] Success: true, code: 0

Rendering PDF
running xelatex - 1
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] xelatex -interaction=batchmode -halt-on-error zh.tex
[execProcess] Success: true, code: 0
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

> 1 package to install
> installing hyphen-chinese-hans (1 of 1)
[execProcess] tlmgr info --list --only-installed --data name hyphen-chinese-hans
[execProcess] Success: false, code: 1
[execProcess] tlmgr install hyphen-chinese-hans
[execProcess] Success: false, code: 1
[NotebookContext]: Starting Cleanup
ERROR: [non-error-thrown] tlmgr returned a non zero status code
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.


Stack trace:
tlmgr install: package hyphen-chinese-hans not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

    at Command.handleError (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/vendor/deno.land/x/[email protected]/command/command.ts:2202:11)
    at Command.parseCommand (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/vendor/deno.land/x/[email protected]/command/command.ts:1782:12)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async quarto (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/quarto.ts:159:5)
    at async file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/quarto.ts:180:5
    at async mainRunner (file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/core/main.ts:35:5)
    at async file:///Users/leo/Documents/Projects/TypeScript/quarto-cli/src/quarto.ts:170:3

With latex-auto-install: false, it compiles the PDF successfully.

Quarto version: 99.9.9
pandoc --verbose --trace
  to: latex
  output-file: zh.tex
  standalone: true
  pdf-engine: xelatex
  variables:
    graphics: true
    tables: true
  default-image-extension: pdf

metadata
  documentclass: scrartcl
  classoption:
    - DIV=11
    - numbers=noendperiod
  papersize: letter
  header-includes:
    - '\KOMAoption{captions}{tableheading}'
  block-headings: true
  tile: 范叶亮
  lang: zh

[execProcess] /Users/leo/Documents/Projects/TypeScript/quarto-cli/package/dist/bin/tools/aarch64/pandoc +RTS -K512m -RTS --defaults /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/quarto-defaults5f94d52e3d71252c.yml /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/quarto-input30f068c454c9ee7.md --metadata-file /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/quarto-metadata95f92bc2a715e2bf.yml --verbose --trace --data-dir /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/pandoc/datadir
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/template.patched from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/template.patched
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/doc-class.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/doc-class.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/pandoc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/pandoc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tightlist.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tightlist.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tables.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/tables.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/graphics.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/graphics.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/citations.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/citations.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/title.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/title.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-body.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/toc.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/toc.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-bib.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/before-bib.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/biblio.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/biblio.tex
[INFO] Loaded /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/after-body.tex from /var/folders/6j/_g2nn8gj1b5_c0kw6c48_4fr0000gn/T/quarto-sessionae6c4759c2f1ea77/ed5f845b90d10d67/6dbf47c10b3ea69c/after-body.tex
[trace] Parsed [Plain [Str "DIV=11"]] at line 1
[trace] Parsed [Plain [Str "numbers=noendperiod"]] at line 1
[trace] Parsed [Plain [Str "scrartcl"]] at line 1
[trace] Parsed [Plain [Str "zh-Hans"]] at line 1
[trace] Parsed [Plain [Str "blue"]] at line 1
[trace] Parsed [Plain [Str "letter"]] at line 1
[trace] Parsed [Plain [Str "\33539\21494\20142"]] at line 1
[trace] Parsed [Plain [Str "\30446\24405"]] at line 1
[trace] Parsed [] at line 6
[trace] Parsed [Header 1 ("\33539\21494\20142",[],[]) [Str "\33539\21494\20 at line 9
[INFO] Running filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua
[INFO] Completed filter /Users/leo/Documents/Projects/TypeScript/quarto-cli/src/resources/filters/main.lua in 33 ms
[execProcess] Success: true, code: 0

Rendering PDF
running xelatex - 1
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] tlmgr --version
[execProcess] Success: true, code: 0
[execProcess] xelatex -interaction=batchmode -halt-on-error zh.tex
[execProcess] Success: true, code: 0
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

running xelatex - 2
[execProcess] xelatex -interaction=batchmode -halt-on-error zh.tex
[execProcess] Success: true, code: 0
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode


[NotebookContext]: Starting Cleanup
Output created: zh.pdf

Thus, would it be possible to make a quick fix with if (pkgMgr.autoInstall && missingHyphenationFile) to make Quarto v1.5 works with chinese language document.

@leovan
Copy link
Author

leovan commented Jul 16, 2024

By the way, the fix won't change the defaut behavior since latex-auto-install is true by default. But it can give us a way to fix this problem by setting latex-auto-install to false.

@cderv
Copy link
Collaborator

cderv commented Jul 16, 2024

Thus, would it be possible to make a quick fix with if (pkgMgr.autoInstall && missingHyphenationFile) to make Quarto v1.5 works with chinese language document.

Yes we do agree on this. Definitely something we would need to improve, i.e. not blocking the pdf rendering because of a warning in the log file.

Though, I would still like to understand what needs to be done for this specific warning

Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil

This triggers our "package to install" detection, but wrongly as we don't know which is the required package to solve it. If we can't find it, I wonder if we should make an exception to skip this warning completely.

Do you have insight on this ? or does the PDF produced anyway is useful and you stop there ?

Any insight would be helpful !

@leovan
Copy link
Author

leovan commented Jul 16, 2024

I guess it's a upstream problem. In babel package, for English:

https://github.com/latex3/babel/blob/dcd551427599879303b7a047b6fd3aef0a77120e/locale/en/babel-en.ini#L147-L149

hyphenrules = english

for Chinese:

https://github.com/latex3/babel/blob/dcd551427599879303b7a047b6fd3aef0a77120e/locale/zh/babel-zh.ini#L196-L198

hyphenrules = 

hyphenrules is set to empty for all babel-zh-Hans-*.ini. I think this leads to the warning message:

Package babel Info: Hyphen rules for 'chinese-hans' set to \l@nil

Should we just ignore the warning message from babel package about hyphen rules?

@leovan
Copy link
Author

leovan commented Jul 16, 2024

Also there is indeed no hyphen for Chinese. For a Chinese word like 你好, line breaking is like:

xxx xxx xxx xxx 你
好 xxx xxx

No need for a hyphen like:

xxx xxx xxx xxx 你-
好 xxx xxx

Why hyphen-chinese package still exits. According to https://github.com/hyphenation/tex-hyphen/blob/master/hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-zh-latn-pinyin.tex, hyphen is only used for Chinese Pinyin https://en.wikipedia.org/wiki/Pinyin.

@cderv
Copy link
Collaborator

cderv commented Jul 16, 2024

So you mean we should ignore the Hyphen rules warning for chinese-hans ?

Seems like the right think to do, if there is no hyphen expected anyway

@leovan
Copy link
Author

leovan commented Jul 16, 2024

Yes, but not only for Chinese. More precisely, I think we should check all languages with empty value for hyphenrules in bable package https://github.com/latex3/babel/tree/main/locale, at least for CJK languages.

@cderv
Copy link
Collaborator

cderv commented Jul 16, 2024

Oh interesting. I don't know very well in detail how babel works and it seems you know better. Can you detail you idea ?
Empty hyphenrules fields in the .ini means that no package should try to be installed and warning is expected in log file ? Is that what you mean ?

Thanks for your help !

@leovan
Copy link
Author

leovan commented Jul 17, 2024

Not quite sure, just guessed from its document and source code.

@cderv
Copy link
Collaborator

cderv commented Jul 17, 2024

ok thanks. I'll search about this. If we do code some logic about this, it needs to be relevant to expected babel behavior

@cscheid
Copy link
Collaborator

cscheid commented Jul 17, 2024

As an immediate workaround, I think we should add a special-case rule to not attempt to install the bad hyphenation package in this specific case.

@cderv
Copy link
Collaborator

cderv commented Jul 17, 2024

As an immediate workaround, I think we should add a special-case rule to not attempt to install the bad hyphenation package in this specific case.

💯 Agreed. This is exactly what I was going to do if not further information on babel package.

With adding support also for latex-auto-install: false for those installed package.

@cderv
Copy link
Collaborator

cderv commented Jul 24, 2024

Just for reference, related notes I made in the past about hyphenation:

@leovan
Copy link
Author

leovan commented Aug 5, 2024

Hi, @cderv any updates with this issue? Would it be possible to make a quick fix with latex-auto-install: false? It truely influence many Chinese language projects.

@mcanouil
Copy link
Collaborator

mcanouil commented Aug 5, 2024

Asking for updates and pressuring with things like "It truely influence many Chinese language projects." does not really help.
This issue is not the only one opened. There are more than 1200 issues and only a few humans to investigate and fix the issues properly (while also adding new features).
Note also that it's summer holidays.

The team is pretty much transparent and communicates/replies more than average projects (see the thread itself), so if you don't see any update, it means there is nothing to communicate about.

Thanks for your interest in this issue and Quarto but please be patient.

@leovan
Copy link
Author

leovan commented Aug 5, 2024

Here is a quick fix like:

diff --git a/src/command/render/latexmk/pdf.ts b/src/command/render/latexmk/pdf.ts
index 52062aa3f..ec9eb0554 100644
--- a/src/command/render/latexmk/pdf.ts
+++ b/src/command/render/latexmk/pdf.ts
@@ -179,7 +179,7 @@ async function initialCompileLatex(
       // See (https://github.com/rstudio/tinytex/commit/0f2007426f730a6ed9d45369233c1349a69ddd29)
       const logText = Deno.readTextFileSync(response.log);
       const missingHyphenationFile = findMissingHyphenationFiles(logText);
-      if (missingHyphenationFile) {
+      if (pkgMgr.autoInstall && missingHyphenationFile) {
         if (await pkgMgr.installPackages([missingHyphenationFile])) {
           // We installed hyphenation files, retry
           continue;

I'm not sure will you accept this as a fix. If do, I can make the PR.

@MaxforCherubim
Copy link

My problem seems like closely to this.

Rendering PDF
running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999996 (TeX Live 2024) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode

> 1 package to install
> installing hyphen-chinese (1 of 1)

And loading forever, no returning the trace of error.
But it is hyphen-chinese not hyphen-chinese-hans, is there someing difference between both?
So, i also render my another previous good project, found it is the same question.

Here i found a workaround:

  1. Disable the yaml lang: zh-CN
  2. add the ctex
# lang: zh-CN
format:
  pdf:
    include-in-header:
      text: |
        \usepackage{ctex}

But using this method, I can not custom Chinese fonts.

@mcanouil
Copy link
Collaborator

@MaxforCherubim Your issue is unrelated. The issue here causes a complete crash.
Please open a separate GitHub discussion with a fully reproducible example, see see https://quarto.org/bug-reports.html.

@MaxforCherubim
Copy link

OK. Let me try

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working latex LaTeX engines related libraries and technologies
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants