'Dimension too large' error when \setmathfont has the option Scale=MatchLowercase

[sammotic]

As discussed at tex.stackexchange.com/q/475792/29127, the code below produces an error saying 'Dimension too large'.

\documentclass{article}
\usepackage{unicode-math}
\setmainfont{Georgia}
\setmathfont[Scale=MatchLowercase]{Cambria Math}
\begin{document}
n $n \sqrt[n]{n}$
\end{document}

The fix has been suggested.

[wspr]

Sorry, I think I'm doing something wrong but I can't reproduce it. Can you use fonts that are provided in TeX Live? E.g., does the following work for you?

\documentclass{article}
\usepackage{unicode-math}
\setmainfont{texgyretermes-regular.otf}
\setmathfont[Scale=MatchLowercase]{XITSMath-Regular.otf}
\begin{document}
n $n \sqrt[n]{n}$
\end{document}

[ysalmon]

Experiencing the same, but only with beamer.

This does not compile :

\documentclass{beamer}
\usepackage{fontspec}
\usepackage[math-style=ISO, bold-style=ISO]{unicode-math}
\setmathfont{texgyrepagella-math.otf}[Scale = MatchLowercase]
\setmathfont{Asana-Math.otf}[range={\varnothing}, Scale = MatchLowercase]
\setmathfont{texgyrepagella-math.otf}[range={\int}, Scale = MatchLowercase] %%

\begin{document}
$a\models b$
\end{document}

neither does this

\documentclass{beamer}
\usepackage{fontspec}
\usepackage[math-style=ISO, bold-style=ISO]{unicode-math}
\setmathfont{texgyrepagella-math.otf}[Scale = MatchLowercase]
\setmathfont{Asana-Math.otf}[range={\varnothing}]
\setmathfont{texgyrepagella-math.otf}[range={\int}] %%

\begin{document}
$a\models b$
\end{document}

But this does

\documentclass{article}
\usepackage{fontspec}
\usepackage[math-style=ISO, bold-style=ISO]{unicode-math}
\setmathfont{texgyrepagella-math.otf}[Scale = MatchLowercase]
\setmathfont{Asana-Math.otf}[range={\varnothing}, Scale = MatchLowercase]
\setmathfont{texgyrepagella-math.otf}[range={\int}, Scale = MatchLowercase] %%

\begin{document}
$a\models b$
\end{document}

This also does

\documentclass{beamer}
\usepackage{fontspec}
\usepackage[math-style=ISO, bold-style=ISO]{unicode-math}
\setmathfont{texgyrepagella-math.otf}[Scale = MatchLowercase]

\begin{document}
$a\models b$
\end{document}

[RuixiZhang42]

Quite possibly due to my previous proposal to move 65536 to the inside in the definition of \@@_fontdimen_to_percent:nN (to fix something else). I never saw this could lead to overflow so easily (sorry!).

[RuixiZhang42]

The behavior of the overflow is quite bizarre: In all my documents, where I have Scale=1.059248555 for Pagella Math, Scale=1.105029586 for Termes Math, everything compiles without any errors.

For the combination in the OP, the x-heights for Georgia and Cambria Math are 986/2048 and 956/2048, respectively, which result in Scale=1.031380753 for Cambria Math. And sure enough, this seemingly smaller scale factor causes an overflow error:

\documentclass{article}
\usepackage{unicode-math}
\setmathfont[Scale=1.031380753]{latinmodern-math.otf}
\begin{document}
n $n \sqrt[n]{n}$
\end{document}

with error message:

! Dimension too large.
<recently read> \l__um_font 

[RuixiZhang42]

Oh dear! Even Ulrike Fischer’s proposed solution in the linked TeX.SX post cannot deal with Scale=1.031380753:

\documentclass{article}
\usepackage{unicode-math}

\ExplSyntaxOn
\cs_set:Nn \__um_fontdimen_to_percent:nN
  {
    \fp_eval:n { \dim_to_decimal_in_sp:n { \fontdimen #1 #2 } / 100 }
  }
\ExplSyntaxOff

\setmathfont[Scale=1.031380753]{latinmodern-math.otf}
\begin{document}
n $n \sqrt[n]{n}$
\end{document}

I still get an error of ! Dimension too large.

However, neither Scale=1.03139 nor Scale=1.03137 produces errors, but Scale=1.03138 does.

[wspr]

I'm glad I asked for the additional validation cases — like you say @RuixiZhang42 I suspect this is quite a subtle problem and might need some creative approaches to executing the calculations with branching depending on the inputs.

[RuixiZhang42]

@wspr This was exactly what I feared in my comment from #287

\documentclass{article}
\usepackage{unicode-math}
\newcommand*\temp{0.7}% failed at 0.7, 0.8, 0.9, 1.1, and of course, 1.03138
\setmathfont[Scale=\temp]{latinmodern-math.otf}
\begin{document}
$n \sqrt[n]{n}$
\end{document}

The Dimension too large error appears precisely when two of the three families share the same size. I shall not repeat the table in my comment from #287 but instead provide here further evidence: The following table lists the em size in sp unit for Latin Modern Math under Scale=1.03137/1.03138/1.03139:

[RuixiZhang42]

We can be sloppy and apply ScaleAgain = 1.0001 and ScaleAgain = 0.9999, each with one less significant digit, to families 2 and 3, which seems to solve all the problems.

[u-fischer]

What is \__um_fontdimen_to_percent:nN meant to calculate? percent of what? The values I get sound completely senseless. E.g. a value of 4.57936pt for fontdimen 10 leads to a percent value of 3001.13.

[wspr]

@u-fischer & @RuixiZhang42 — sorry, I don't have time tonight to look into this further but thanks for your input!

Ulrike, here's what this is supposed to be doing:

\documentclass{article}
\usepackage{unicode-math}
\begin{document}
\font\x="[texgyrepagella-math.otf]" at 10pt
Font dimen 10:
\ExplSyntaxOn
\__um_fontdimen_to_percent:nN {10} \x
\ExplSyntaxOff
\end{document}

I.e., the "script size font" is supposed to be 74% of the "text size" font. (Also see Appendix D of the unicode-math package docs.)

[wspr]

@RuixiZhang42 — I suspect you're correct that the problem is the unintended overlap of sizes with fam2 & fam3. Instead of multiplying by 1+epsilon and 1-epsilon, I should instead by adding an incremental value to avoid the rounding. Which was entirely your suggestion originally, I apologise for not seeing the rationale then!

[RuixiZhang42]

@u-fischer I believe when unicode-math scans OTF Math Table, fontdimen’s 10, 11 and 65 are stored in units of sp. For instance, with the Georgia + Cambria Math (Scale=MatchLowercase) example, \ExplSyntaxOn \the \fontdimen 10 \g__um_sqrt_font_cmd_tl \par \the \fontdimen 11 \g__um_sqrt_font_cmd_tl \par \ExplSyntaxOff produces 0.00111pt and 0.00092pt. Could you please point out why there was a 4.57936pt for fontdimen 10?

When I proposed to pre-multiply 65536 to avoid rounding error, I was sure that fontdimen’s 10, 11 and 65 are “percent” as they should be, and therefore they were no more than 100sp. Thus \__um_fontdimen_to_percent:nN will at most produce 6553600, which is less than 0.62% of the TeX limit of 2^30.

[RuixiZhang42]

@wspr I came up with an algorithm which allows a dynamic ScaleAgain factor:

1. Define global macros \g_@@_ScaleAgain_fam_two: and \g_@@_ScaleAgain_fam_three: to be 1.00001 and 0.99999, respectively.
2. Extract Scale from the loading of “fam1”; i.e., the font without any ScaleAgain.
3. Compare if \g_@@_ScaleAgain_fam_two: * Scale equals Scale. If equal, proceed to Step 4; Otherwise proceed to Step 5.
4. \g_@@_ScaleAgain_fam_two: <- \g_@@_ScaleAgain_fam_two: * 1.00001, then go back to Step 3.
5. Compare if \g_@@_ScaleAgain_fam_three: * Scale equals Scale. If equal, proceed to Step 6; Otherwise proceed to Step 7.
6. \g_@@_ScaleAgain_fam_three: <- \g_@@_ScaleAgain_fam_three: * 0.99999, then go back to Step 5.
7. Pass \g_@@_ScaleAgain_fam_two: and \g_@@_ScaleAgain_fam_three: to ScaleAgain to load fam2 and fam3.

As you can see, I am instructing unicode-math to perform two loops until the ScaleAgain factors are able to make a difference in font size. This is supported by the following observation:

Round( 1.00001 * 1.03138 * 2^16 ) = 67593
Round(           1.03138 * 2^16 ) = 67593
Round( 0.99999 * 1.03138 * 2^16 ) = 67592

In this case, we only loop once to get \g_@@_ScaleAgain_fam_two: = 1.00001^2 = 1.00002, which is enough:

Round( 1.00002 * 1.03138 * 2^16 ) = 67594
Round(           1.03138 * 2^16 ) = 67593
Round( 0.99999 * 1.03138 * 2^16 ) = 67592

[u-fischer]

@RuixiZhang42 Inside \r@@t the fontdimen has this value:

\documentclass{article}
\usepackage{unicode-math}

\ExplSyntaxOn


\cs_set:Nn \__um_fontdimen_to_percent:nN
  {
    \fp_eval:n { \dim_to_decimal_in_sp:n { \fontdimen #1 #2 } / 100 }
  }


\makeatletter
\cs_set:Nn \__um_redefine_radical:
  {
    \cs_set_nopar:Npn \r@@t ##1 ##2
      {
        \hbox_set:Nn \l_tmpa_box
          {
            \c_math_toggle_token \m@th
            ##1 \sqrtsign { ##2 }
            \c_math_toggle_token
          }
        \__um_mathstyle_scale:NnnN ##1 { \kern } { \fontdimen 63 \g__um_sqrt_font_cmd_tl } \g__um_sqrt_font_cmd_tl
        \box_move_up:nn
          {
            (\box_ht:N \l_tmpa_box - \box_dp:N \l_tmpa_box) * \number \fontdimen 65 \g__um_sqrt_font_cmd_tl / 100
          }
          { \box_use:N \rootbox }
          \typeout{fontdimen 10~
          \the\fontdimen10\g__um_sqrt_font_cmd_tl}
          \fp_set:Nn\l_tmpa_fp {\__um_fontdimen_to_percent:nN {10}
          \g__um_sqrt_font_cmd_tl}
          \fp_show:N\l_tmpa_fp
          \typeout{fontdimen 11~
          \the\fontdimen11\g__um_sqrt_font_cmd_tl}
          \fp_set:Nn\l_tmpa_fp {\__um_fontdimen_to_percent:nN {11}
          \g__um_sqrt_font_cmd_tl}
          \fp_show:N\l_tmpa_fp
        \__um_mathstyle_scale:NnnN ##1 { \kern } { \fontdimen 64 \g__um_sqrt_font_cmd_tl } \g__um_sqrt_font_cmd_tl
        \box_use_drop:N \l_tmpa_box
      }
  }
\ExplSyntaxOff

\setmathfont[Scale=1.031380753]{latinmodern-math.otf}
\begin{document}

$\sqrt[n]{n}$
\end{document}

[RuixiZhang42]

@u-fischer Thank you for providing the additional information. Your newest example illustrates the clearly wrong fontdimen 10 used, and confirms my point that \__um_fontdimen_to_percent:nN isn’t broken but the way ScaleAgain operates is. Nonetheless, I fully agree with your improvement of using \dim_to_decimal_in_sp:n.

You see, with your improvement, \setmathfont[Scale=0.7]{latinmodern-math.otf} compiles successfully. But it shouldn’t and the current \dim_to_decimal:n { 65536 \fontdimen #1 #2 } exposes the bug by throwing out a ! Dimension too large.

@wspr I can confirm that this is caused by overlapping font dimension parameters.

---

• Why does \setmathfont[Scale=1.03138]{latinmodern-math.otf} break down?

  Because overlapping fam2 parameters. Since Round( 1.00001 * 1.03138 * 2^16 ) = 67593 = Round( 1.03138 * 2^16 ), the fontdimen 10 in family 2 (and together with 1st font) is remapped to fontdimen 32 (StackTopShiftUp).

  This explains the 4.57936pt reported by @u-fischer — 4.57936/1.03138 = 4.44003 and I found this in FontForge:

  

---

• Why does \setmathfont[Scale=0.7]{latinmodern-math.otf} break down?

  Because overlapping fam3 parameters. Since Round( 0.99999 * 0.7 * 2^16 ) = 45875 = Round( 0.7 * 2^16 ), the fontdimen 10 in family 3 (and together with 1st font) is remapped to fontdimen 30 (LowerLimitGapMin). The example by @u-fischer reports a fontdimen 10 of 1.16899pt, which is from 1.16899/0.7 = 1.66999:

  

[u-fischer]

Why are the fontdimen remapped like this, if the families overlap?

[RuixiZhang42]

@u-fischer The legacy TeX fontdimens’ assume parameters come from different fonts: fam2 comes from OMS (\times, \subset) and fam3 comes from OMX (\sum, \int). OpenType math font bundles all parameters in a single file, so fontdimens’ have to be setup by loading the OTF three times. Since TeX forbids loading the same font at the same size twice, unicode-math needs to use slightly different sizes.

@wspr The current fixed ScaleAgain factors did not compensate TeX’s binary arithmetic. So under some special user requested Scale, the two ScaleAgain=1.00001 and ScaleAgain=0.99999 may fail to generate really different sizes. I took the liberty of coming up with a theorem, which tries to describe which user requested scale may fail.

IMHO, to fully overcome the problem in Theorem 1 below, unicode-math must find a way to apply dynamic ScaleAgain factors (see my suggested algorithm). Any fixed ScaleAgain factors will create some discrete ranges inside which the Scale will lead to ! Dimension too large.

I strongly encourage anyone who are interested to try the boundary values in Corollary 1.2. These special boundaries are obtained by setting the integer k in Theorem 1 to 67591, 67592 and 67593.

• \setmathfont[Scale=1.031369386]{latinmodern-math.otf} — fails.
• \setmathfont[Scale=1.031369387]{latinmodern-math.otf} — works.
• \setmathfont[Scale=1.031374754]{latinmodern-math.otf} — works.
• \setmathfont[Scale=1.031374755]{latinmodern-math.otf} — fails.
• \setmathfont[Scale=1.031384644]{latinmodern-math.otf} — fails.
• \setmathfont[Scale=1.031384645]{latinmodern-math.otf} — works.
• \setmathfont[Scale=1.031390013]{latinmodern-math.otf} — works.
• \setmathfont[Scale=1.031390014]{latinmodern-math.otf} — fails.

[tail-reversion]

Excuse the naïve question, but is TeX’s prohibition of reloading a font at the same size a fundamental limitation of how TeX works, or is it just a restriction imposed to prevent redundancy? I ask because, if the latter is the case, it sounds like a lot of headaches could be prevented by patching XeTeX and LuaTeX to remove the restriction.

[wspr]

@tail-reversion — it's just an optimisation from the '70s to save memory. (I actually thought it was different in LuaTeX but my test file shows the same behaviour as XeTeX.)

It's outside my area of expertise to comment on whether it would be feasible to replace. There are other internal parts of TeX the Program that survive into the modern era (such as a relatively poor hash algorithm) that would be good to replace but no-one has attempted upgrades like this as far as I know.

[wspr]

@u-fischer — @RuixiZhang42 has it right, the only reason to load fam2 and 3 remapped with "legacy" font dimensions is to avoid rewriting every part of LaTeX & amsmath & assorted packages that use fontdimens from those families for various calculations.

@RuixiZhang42 — thank you so much for your extended analysis here, fascinating stuff. I took a look in the NFSS again to remind myself what my code was doing, and there's no (easy) way to add an absolute increment to the pt size being loaded (i.e., it has to be a scale value like we have). So something along the lines of your dynamic algorithm is the best approach for us to take IMO.

Of course... we should probably be a little conservative in that if a maths font is set up at 12pt with the smallest scaling such that fam2 & fam3 have unique sizes, that scaling will be too small if the maths font setup is then loaded for (say) a 7pt text size.

Maybe more pragmatically at this point, does ScaleAgain = 1.0001 and ScaleAgain = 0.9999 like you suggest above fix the problem for all reasonably-sized text fonts? Or even ScaleAgain = 1.001 and ScaleAgain = 0.999? A 0.1% error on some \fontdimen value is probably not too bad :)

[u-fischer]

@wspr Do you have an example that demonstrates the problem if the same font is used for all families? I always wondered about this. Here Joseph did it https://tex.stackexchange.com/questions/308749/unicode-math-at-tex-primitive-level.

Did I understand correctly that the problem with fontdimen 10 (and 11) is, that it has a different meaning (and value) with legacy fonts than with open type? How could this happen?

[RuixiZhang42]

@wspr Please don’t go to the extreme 1.001 and 0.999 I beg ;-) Resorting to lower accuracy is indeed an alternative.

ScaleAgain = 1.0001 and ScaleAgain = 0.9999, both with 4 decimals, are able to cover all reasonably-scaled fonts. And I can prove it!!!

This means, in theory, the last problematic range end at

( 9999 + 1.5 ) / ( 1.0001 * 2^16 ) = 0.15258026199333191680831916808319 = 15.258 %

Therefore, as long as the user Scale is greater than 0.152581 (roughly 15.3%), the two 4-decimal ScaleAgain’s are able to produce real differences in font sizes. I don’t even see how one would require to scale the math font down to 20%. So 4-decimal is the way to go.

Using 3-decimal corresponds to a user Scale lower bound allowance of roughly 1.525%. I believe no one actually scales their math fonts at 1.525%. Also, this is too rough.

Using 5-decimal (the current implementation) corresponds to 152.587%, which is Scale = 1.52587. Below this threshold, one has to be lucky enough to be in one of the “good zone”. However, normal usage from the user Scale almost always falls below this threshold.

[ysalmon]

I do not know if this helps, but I encountered a case where the error only triggers at some point in the file, not at the beginning.

Attached is a tex file to generate a beamer presentation for explaining balancing of AVL trees. The trees are typeset by package forest from generated descriptions. Everything goes well until line 929 of aleat.tex (which corresponds to page 199 of the document). I can ignore these errors, and they will repeat for lines that correspond to pages 200 to 220 (pages 221 to 245 are generated without error).

This is baffling because the textual content of pages 221 to 245 is a superset of the content of pages 200 to 220 (see attached PDF, were all pages seem to be rendered correctly despite the errors).

One can eschew inclusion of croissant.tex to speed up compilation somewhat.
Arbres.zip

[RuixiZhang42]

@ysalmon Like I said in this thread, the temporary fix in your code containing \dim_to_decimal_in_sp:n is not enough (and it hides the real problem).

Gentium Basic has an x-height of 930/2048, while TeX Gyre Pagella Math has 469/1000. These metrics will result

Scale = 0.968234 (or so) for TeX Gyre Pagella Math

If you plug in k = 63454 in my theorem, then you will see that the following range will cause problem:

0.968 233 255 < Scale < 0.968 244 406 — will cause problem!

The safe Scale zones are

0.968 229 149 < Scale < 0.968 233 254 (but not attainable here)
or
0.968 244 407 < Scale < 0.968 248 513 (okay)

Here is what happens near Scale=0.968234:

All red line segments represent the problematic Scale factors, while green line segments represent the safe Scale factors.

The ScaleAgain=1.00001 is by trial and error, meant to bring the Scale factor to the nearest green segment. The numbers are subject to other rounding errors. So the real fix to your AVL trees file is to issue (this won’t be necessary in the next release)

\setmathfont{texgyrepagella-math.otf}[Scale = MatchLowercase, ScaleAgain=1.00001]
% Then proceed with Asana-Math.otf and texgyrepagella-math.otf
% No more ScaleAgain required

Also, the problem appears in 11pt (beamer default, with calculated Scale 0.9682347732816738 as in your log), but not in 10pt (article default, with calculated Scale 0.9682331047227392) — these are the “other rounding errors” I’ve mentioned. The followed behavior matches the theorem predication perfectly.