Compilation errors using MiKTeX and lualatex on Windows

[l0th3r]

Hi,
I am new to Tex and LaTex, especially on Windows.

Recently I needed to be able to compile a project I had on Linux, on Windows.
(never done Tex on Windows before)

On Linux, I was using a portable install of the markdown package.
Linux compilation command: lualatex --shell-escape -interaction=nonstopmode --output-directory=_build main.tex
(0 error and PDF file content is as expected)

Windows installation process:

• Installed MiKTeX latest version.
• Started MiKTeX Console program
  • Chose https://ctan.mines-albi.fr as package repository
  • Checked for updates and Installed updates, restart program
  • Installed the markdown package
• Removed all files from the portable installation in my project directory

Executed in the directory with powershell: lualatex --shell-escape -interaction=nonstopmode --output-directory=_build main.tex
Some dialog window showed up to ask to install some packages and said yes to all.
(6 errors and no PDF file produced)

Errors:

! Undefined control sequence.
\markdownOptionOutputDir ->Z:\shared 
                                     \testTEx \_build 
l.6 \end{markdown}
                
The control sequence at the end of the top line
of your error message was never \def'ed. If you have
misspelled it (e.g., `\hobx'), type `I' and the correct
spelling (e.g., `I\hbox'). Otherwise just continue,
and I'll forget about whatever was undefined.

! Undefined control sequence.
\markdownOptionOutputDir ->Z:\shared \testTEx 
                                              \_build 
l.6 \end{markdown}
                
The control sequence at the end of the top line
of your error message was never \def'ed. If you have
misspelled it (e.g., `\hobx'), type `I' and the correct
spelling (e.g., `I\hbox'). Otherwise just continue,
and I'll forget about whatever was undefined.

! Undefined control sequence.
\markdownOptionOutputDir ...ared \testTEx \_build 
                                                  
l.6 \end{markdown}
                
The control sequence at the end of the top line
of your error message was never \def'ed. If you have
misspelled it (e.g., `\hobx'), type `I' and the correct
spelling (e.g., `I\hbox'). Otherwise just continue,
and I'll forget about whatever was undefined.

! You can't use `\endgroup' after \the.
<argument> \endgroup 
          
l.6 \end{markdown}
                
I'm forgetting what you said and using zero instead.


! Package markdown Error: Markdown file \markdownOptionOutputDir
(markdown)                /\markdownOptionInputTempFileName  does not exist

Type <return> to continue.
 ...                                              
                                                  
l.6 \end{markdown}
                

LaTeX does not know anything more about this error, sorry.

Try typing <return> to proceed.
If that doesn't work, type X <return> to quit.



! LaTeX Error: \begin{markdown} on input line 4 ended by \end{document}.

See the LaTeX manual or LaTeX Companion for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.7 \end{document}
                
Your command was ignored.
Type  I <command> <return>  to replace it with another command,
or  <return>  to continue without it.

Complete log file

main.tex:

\documentclass{article}
\usepackage{markdown}
\begin{document}
\begin{markdown}
Hello *world*!
\end{markdown}
\end{document}

Also tried with the pdflatex command but similar errors aswell.

Thanks in advance for taking the time to answer.

[Witiko]

This is an interesting Windows-specific bug. You should be able to hotfix it by typing \def\markdownOptionOutputDir{_build} above \usepackage{markdown}. Typing \usepackage[outputDir=_build]{markdown} should also work, please confirm.

The proper fix is to deactivate tokens in the macro \markdownOptionOutputDir. We may also need to apply the fix in lt3luabridge.

[l0th3r]

Thank you for your answer @Witiko.

Maybe it's important to notice I am using a virtual machine

With main.tex as such:

\documentclass{article}
\def\markdownOptionOutputDir{_build}
\usepackage{markdown}
\begin{document}
\begin{markdown}
Hello *world*!
\end{markdown}
\end{document}

Error:

./markdown.lua:3237: bad argument #1 to 'open' (string expected, got nil)
stack traceback:
        [C]: in function 'io.open'
        ./markdown.lua:3237: in function <./markdown.lua:3235>
        ./markdown.lua:3235: in main chunk
        [C]: in function 'require'
        [\directlua]:1: in main chunk.
\lua_now:e #1->\__lua_now:n {#1}

complete log file "main.log"
(1 error, no PDF file produced)

The exact same error when using \usepackage[outputDir=_build]{markdown}.

If you need more information do not hesitate.
Thank you for your time !

[Witiko]

According to the logs, you used Markdown v3.6.2-0-g6c30af7e. In this version of the package, line 3237 of markdown.lua loads file UnicodeData.txt. As previously discussed with @Yggdrasil128 in #458 (comment) and below, this is an issue with MikTeX whose KPathSea library fails to locate the file UnicodeData.txt, see also the upsteam report in MiKTeX/miktex#1515. You can fix this issue by downloading the file and placing it into the same directory as your document.

[l0th3r]

@Witiko Thank you for your answer.

I did read this issue #458 before opening this issue and tested.

By adding a file named "UnicodeData.txt" with the content you give.
Errors:

luaotfload | load : FATAL ERROR
luaotfload | load :   × Failed to load "luaotfload" module "database".
luaotfload | load :   × Error message:
luaotfload | load :     × "...rams/MiKTeX/tex/luatex/luaotfload/luaotfload-unicode.lua:108: assertion failed!".

This issue happens now on every environment (Windows, Ubuntu, Arch) since I added UnicodeData.txt

[Witiko]

At lines 107–109 in the file luaotfload-unicode.lua, the file UnicodeData.txt is located, read, and parsed:

local f = io.open(kpse.find_file"UnicodeData.txt")                        
assert(file:match(f:read'*a'))                                            
f:close()

Apparently, your file UnicodeData.txt contains something that cannot be parsed.

Can you take a peek at what's inside your file UnicodeData.txt? Mine starts like this:

0000;<control>;Cc;0;BN;;;;;N;NULL;;;;
0001;<control>;Cc;0;BN;;;;;N;START OF HEADING;;;;
0002;<control>;Cc;0;BN;;;;;N;START OF TEXT;;;;
0003;<control>;Cc;0;BN;;;;;N;END OF TEXT;;;;
0004;<control>;Cc;0;BN;;;;;N;END OF TRANSMISSION;;;;
0005;<control>;Cc;0;BN;;;;;N;ENQUIRY;;;;

Regardless, we will be getting rid of the dependency on UnicodeData.txt in the upcoming release, see also #486.

[l0th3r]

Apparently, your file UnicodeData.txt contains something that cannot be parsed.

Can you take a peek at what's inside your file UnicodeData.txt? Mine starts like this:

I have used the content from this link, you sent. Thus, the content of my file starts the same.

Would it be easier to update my markdown package (which is installed in "portable") with the update you made in #486 ?

[Witiko]

I have used the content from this link, you sent. Thus, the content of my file starts the same.

Strange! How does it end? Mine ends as follows:

E01EF;VARIATION SELECTOR-256;Mn;0;NSM;;;;;N;;;;;
F0000;<Plane 15 Private Use, First>;Co;0;L;;;;;N;;;;;
FFFFD;<Plane 15 Private Use, Last>;Co;0;L;;;;;N;;;;;
100000;<Plane 16 Private Use, First>;Co;0;L;;;;;N;;;;;
10FFFD;<Plane 16 Private Use, Last>;Co;0;L;;;;;N;;;;;

It's weird that luaotfload-unicode.lua would not be able to parse the file. I suspect incomplete download or an incorrect encoding of line endings.

You should be able to hotfix the issue by renaming the file UnicodeData.txt to e.g. UnicodeDataMarkdown.txt and replacing all occurences of UnicodeData.txt in markdown.lua with the new name. This way, libraries such as luautfload should be unaffected.

Would it be easier to update my markdown package (which is installed in "portable") with the update you made in #486?

Later today, I will release version 3.7.0, which includes #486. In this version, markdown.lua is also renamed to markdown-parser.lua, as discussed in #487. Therefore, you would need to apply your changes to markdown-parser.lua.

Regardless, we will be getting rid of the dependency on UnicodeData.txt in the upcoming release, see also #486.

When I say "upcoming release", I mean version 3.7.1, to be released by the end of September.

[l0th3r]

Mine ends with the same lines.

It's weird that luaotfload-unicode.lua would not be able to parse the file. I suspect incomplete download or an incorrect encoding of line endings.

Do you mean from my part ? Could it be coming from me ?

You should be able to hotfix the issue by renaming the file UnicodeData.txt [...]

It works. I can compile on Ubuntu now with this fix. I have other Issues on Windows but not this one anymore.
Renaming the file in and renamed the file also in markdown.lua fixed it.

Later today, I will release version 3.7.0 [...]

I will test when updating my package in 3.7.0 and close the issue if it works.

Thank you for your answers !

[Witiko]

Do you mean from my part ? Could it be coming from me ?

Possibly, depending on how you downloaded it. For example, if you opened the file in your browser and then copy-and-pasted its contents into the Notepad and saved, then this would have saved the file with an incorrect encoding of line endings, which might confuse the parser.

I am not saying that this is what happened but since both @Yggdrasil128 and I were able to use the downloaded file UnicodeData.txt with the Markdown package 3.6.2 without issues, it may indicate that there is something wrong with your file. Of course, there are countless other potential reasons.

[l0th3r]

Okay I understand. For now it is fixed, thanks to you.
I have other issues with cross compilation but I am not sure they are directly linked with the markdown package.

Since in the next version UnicodeData.txt will not be a dependency anymore, there is no point investing further in my opinion.

Thanks a lot for the help ! @Witiko

[Witiko]

I am happy to help. However, let's keep this ticket open, please. #492 (comment) and #492 (comment) mention a bug that should be fixed before closing the ticket.

[Witiko]

The issue from #492 (comment) should be fixed after PR #501, to be released in Markdown 3.7.1 by the end of September.

@l0th3r: I tested the changes on a Windows VM but I will appreciate your feedback before the release. Here are the compiled sources of the Markdown package after the change.