Bug with rendering $\hat$ with superscripts

[geomblog]

Consider the form $\hat{f}^p$. Mathjax renders it in HTML-CSS mode with the superscript p above f, rather than to the right, as you'd get with $f^p$.

[geomblog]

Here's a screen shot. This was rendered in mathim.com (which uses mathjax for latex rendering). Viewed on Chrome version 31.0.1650.63 on a Macbook running OS X 10.9.1.

[pkra]

Thanks for the report. I can confirm this across browsers.

[geomblog]

And while the bug was observed on mathim.com, I was able to reproduce it on cstheory.stackexchange.com (a Stackexchange site that also uses mathjax)

[pkra]

@geomblog Yes, it's not a browser bug or bad CSS-interaction. MathJax is just laying it out "wrong".

@dpvc This does not seem to be a regression. Is it intentional?

[dpvc]

The natural MathML translation of \hat{f}^p is

<msup>
  <mover>
    <mi>f</mi>
    <mo>^</mo>
  </mover>
  <mi>p</mi>
</msup>

That means the base of the power is the f with the hat. Superscripts are placed in relation to the height of the base, and that height includes the hat in this case, so the p is placed higher than it is when the f doesn't have the hat. (This is complicated by the fact that the OP has the STIX fonts, and it appears that the wrong hat is being used -- when used as an accent, the U+005E should be replaced by one of the smaller versions intended for accents -- so the height is even larger than usual; it looks better with the MathJax fonts, though still higher than for a plain f.)

To get the same output as TeX, the layout for superscripts would have to ignore the height of the accent, and just use the height of the f. That would be possible, but would require making special cases in the treatment of <msup>. It is complicated by the fact that TeX is inconsistent about this. For example \bar{f}^p is positioned as though the bar is not there, but \overline{f}^p is positioned taking the overline into account. Both of these are handled through <msup><mover> constructs like the one above in MathJax, and there is no natural way to distinguish which is which (other than using a list of characters to ignore when they are used as accents). It also depends on what is below the hat. For example, \hat{ff}^p has the p in the higher position, as does \hat{\rule{1em}{1em}}^p or \hat{\hbox{f}}^p or \hat{+}^p.

Finally, note that native MathML in Firefox currently treats the <msup><mover> as MathJax does, so even if we change the HTML-CSS and SVG output, it will still be "wrong" in the NativeMML output.

[pkra]

Thanks, I suspected as much. I'll ask David Carlisle for some thoughts.

[davidcarlisle]

A TeX example is

\documentclass{article}

\showoutput

\begin{document}

$$ f^p \hat{f}^p  {{}\hat{f}}^p  F^p   \hat{F}^p$$

\end{document}

as Davide commented neither TeX nor MathML really offers any user-level control over this, TeX just has some heuristics in appendix G that tweak the superscript position in some simple cases (basically if the base is a single letter and the accent is a single glyph placed with the \accent primitive, even adding {} to the base is enough to prevent this.

I don't think the high p is wrong, it's "just" a quality of rendering issue, similar to font specific kerning or ligatures. It would be nice (and conformant) if renderers could use similar special cases to TeX but I don't think it is strictly a bug if they don't (or if different MathJax back ends end up with different design choices here) I suppose if you really wanted to force the low p you could use mpadded to hide the height of the accent, but that would require the generated markup knowing the size of the accent (or base) and seems rather fragile in general (although I would guess mathjax has that information to hand?)

[dpvc]

@davidcarlisle, thanks for your comments on this, as always.

if the base is a single letter and the accent is a single glyph placed with the \accent primitive

I think this is correct, except that there is one more restriction, which is that the class of the base has to be ORD. For example, \hat{+}^p and +^p differ, as do \hat{(}^p and (^p, etc. On the other hand, \hat{\hat{f}}^p ignores the height of both accents. So the code needed to determine the exceptions using the internal MathML would have to be fairly complicated. I suppose it could be simplified if we only wanted to handle this for TeX input rather than arbitrary MathML (in that case, the TeX input processor could mark the internal MathML to indicate what came from \accent, but I've been trying to avoid that as much as possible).

In terms of using <mpadded>, you can get the desired effect from

<msup>
  <mover accent="true">
    <mi>f</mi>
    <mpadded height="0">
      <mo stretchy="false">^</mo>
    </mpadded>
  </mover>
  <mi>p</mi>
</msup>

where we make the height of the hat 0), but this would not be appropriate for all uses of \hat, since you want \sqrt{\hat{f}} to place the root over the hat, not the f. Also, Firefox doesn't render this as expected, because the <mpadded> causes Firefox to place the accent too high (apparently it only removes the space below the glyph if it is the direct child of the <mover>; I think this is probably a bug, as the <mpadded> should make it an embellished operator, and it should still position the result using its core).

that would require the generated markup knowing the size of the accent (or base) ... (although I would guess mathjax has that information to hand?)

Actually, the TeX input processor has no idea about the size of the output (intentionally), and so it could not be done at the time \accent is processed. Certainly the HTML-CSS and SVG output processors know the sizes, but it is difficult to determine when to take this into account at this point (my claim that the algorithm would be complicated above). The NativeMML output processor doesn't know the sizes of things, but I suppose that since we already tweak the output for Firefox in other ways, it would be possible to modify the MathML at output time for this as well.

I don't think the high p is wrong, it's "just" a quality of rendering issue

I agree, and am not sure it is worth trying to make it match the TeX output in this case. I suspect that it would be difficult to get it to handle exactly the situations that TeX does.

[davidcarlisle]

By coincidence the W3C Math WG was having a telecon as this came up, and the consensus was as above that while in some cases it would be nice to have the superscripts aligned that at the mathml level there are no plans to add functionality for this. Patrick commented that he (in TeX) often changes fontdimen16/17 to change (raise) the default height of superscripts so that you do get more even superscript position over more varied base constructs in MathML this would amount to setting the superscriptshift length attribute on the unaccemted base so that the superscript was raised by a minimum amount that accounted for accented letters.

[dpvc]

Thanks for bringing it up. I wasn't expecting that MathML would add anything to accommodate this. You are right that superscriptshift could be used to help normalize the space, but isn't that on the <msup> element, not the base?

I guess it will be up to the renderers how they want to treat this situation. I'm going to mark this as a feature request for now.

[davidcarlisle]

Thanks for bringing it up. I wasn't expecting that MathML would add
anything to accommodate this. You are right that superscriptshift could
be used to help normalize the space, but isn't that on the element, not the base?

yes it is no msup. sorry I meant use it on the expression with the
unaccented base (or on both) to force the p's to be the same height.

[dpvc]

Ah, yes, now I see what you meant. Sorry for the confusion.

[dpvc]

This will (finally) be resolved in v3 by PR mathjax/MathJax-src#618.