Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

supported type should include MathML types #141

Open
polx opened this issue May 7, 2021 · 21 comments
Open

supported type should include MathML types #141

polx opened this issue May 7, 2021 · 21 comments

Comments

@polx
Copy link

polx commented May 7, 2021

This issues proposes that the MathML media-types also be included as mandatory media-types for paste and copy events:

Indeed the use case includes mathematical information as one of the targets. But as of today, only application/xml would be allowed while this type is much too generic. A few applications exposed the platform specific "clipboard flavors" which are also specified in the media-types declarations.

@marcoscaceres
Copy link
Member

cc @bkardell for input... wondering what browsers do with the media types?

@NSoiffer
Copy link

I just checked copy FireFox and Chrome (with experimental MathML turned on) with and neither put out any of those on the clipboard in Windows.

@bkardell
Copy link

cc @bkardell for input... wondering what browsers do with the media types?

@marcoscaceres - Do you mean "at all" or specifically on this question?

@marcoscaceres
Copy link
Member

Yeah, "at all" (with respect to the media type)... and if browsers do something when performing a clipboard operation with MathML data, then what do they do? ... thanks to @NSoiffer, we know at least on Windows nothing happens #141 (comment)

@bkardell
Copy link

If I set the content-type in the response to these, it changes the parsing behavior in FF from HTML (creating the host doc, etc), in WebKit/Chromium (Linux) it causes the document to be downloaded instead of displayed. I think some of this is underspecified and not all of these observations should necessarily be the case, but some it'll also change the application of some things, I guess. In FF, if set:

  • the parser is xml style/throws on errors (not the case with core on an element in HTML)
  • the document root will be <math> not generated in a host <html> doc, and all that follows from that in terms of different trees (:root, document.documentElement, etc are diff).
  • scripts and styles in tokens will be displayed, and not run
  • an <a href=""> inside a token will work differently - without the mime these are regular HTML links, with them they are MathML links on effectively 'unknown element' (tho there wasn't really IDL for these even until recently with core, but MathML allowed any element to have an href which were really 'somewhat link like') That is, these aren't entirely well specified, or compatible. An example though is attributes like rel/target, etc are not also supported and default tab indexes aren't the same. Probably there is also more on the relation of linked resources.
  • There might be more, but that's all I can find/think of atm

@polx
Copy link
Author

polx commented Jun 17, 2021

@marcoscaceres : On macOS (and on windows I think), browsers that are invoked with the clipboard operation e.clipboardData.setData('application/mathml+xml', '<math><mfrac><mn>3</mn><mn>5</mn></mfrac></math>'); just put this content as an exotic org.mozilla.custom-clipdata or similarly private types with Chrome or Safari.
Test page here: http://direct.hoplahup.net/tmp/tryclipboardOps.html .
You will also see that HTML comes through.
This call is the object of this issue.

When you have MathML in a web-page and try to copy it, you obtain HTML with, sometimes the MathML in it.

@polx
Copy link
Author

polx commented Jun 17, 2021

@bkardell : Re: your comment: for me this is a different story that is not bound to clipboard-apis. I would move this to an issue in our MathML spec.

@rniwa
Copy link

rniwa commented Jun 18, 2021

@marcoscaceres : On macOS (and on windows I think), browsers that are invoked with the clipboard operation e.clipboardData.setData('application/mathml+xml', '<math><mfrac><mn>3</mn><mn>5</mn></mfrac></math>'); just put this content as an exotic org.mozilla.custom-clipdata or similarly private types with Chrome or Safari.
Test page here: http://direct.hoplahup.net/tmp/tryclipboardOps.html .
You will also see that HTML comes through.

So on macOS and other Apple platforms, there is system declared UTI for MathML or SVG. As a result, this isn't really possible in a way that's interoperable across different applications: https://developer.apple.com/library/archive/documentation/Miscellaneous/Reference/UTIRef/Articles/System-DeclaredUniformTypeIdentifiers.html#//apple_ref/doc/uid/TP40009259

We could declare it as a generic XML content but I don't think that'd help the situation.

@polx
Copy link
Author

polx commented Jun 19, 2021

@rniwa : I don't get the point of being impossible.
UTIs for MathML supported types are already defined.
Are you claiming that it is not possible because Apple does not define it as system-wide type?
Microsoft also does not, they try to define the least possible. Linux does none.

Why would this prevent the clipboard-APIs implementations to copy the appropriate type as expressed in a way that is specific to the platform?

@rniwa
Copy link

rniwa commented Jun 19, 2021

@rniwa : I don't get the point of being impossible.
UTIs for MathML supported types are already defined.

MIME type is defined. UTI in AppKit / UIKit is a slightly different concept, and that's what applications on Apple platforms would use to identify the type of a content. MIME types don't matter much on Apple platforms since that's not the primary mechanism by which the content type is identified by applications.

Are you claiming that it is not possible because Apple does not define it as system-wide type?

That's right although I would not rule out the possibility that Apple may define the UTI type for MathML in the future.

Why would this prevent the clipboard-APIs implementations to copy the appropriate type as expressed in a way that is specific to the platform?

Well, because if different applications don't agree which UTI to use to identify MathML, then it's not possible for different applications to recognize the content as such. Say, WebKit / Safari uses public.mathml to expose MathML. If other applications were expecting public.xml to contain MathML then such an application won't find it.

Microsoft also does not, they try to define the least possible. Linux does none.

While Windows SDK also has a small set of the standard clipboard format types defined, format types are often defined using MIME type strings themselves so it would not be entirely incompatible to use application/mathml+xml as the format type string.

GTK+ also uses MIME type to define GdkContentProvider for example so having MIME type defined works there too.

@polx
Copy link
Author

polx commented Jun 19, 2021

Thank you @rniwa for the extra links. I don't see a disagreement except for "it will not work". We had already spotted at the time that media-types are not exactly the same as UTIs. That is why the MathML specification includes UTI specifications:

  • MathML: Macintosh Universal Type Identifier code public.mathml conforming to public.xml
  • MathML-presentation: Macintosh Universal Type Identifier code public.mathml.presentation conforming to public.mathml (described above) conforming to public.xml
  • MathML-content: Macintosh Universal Type Identifier code: public.mathml.content conforming to public.mathml (described above) conforming to public.xml

That is also why I sometimes recommend media-type declarations to include UTIs and Windows Clipboard flavour names when I see them passing by on the media-types mailing-list.

I would like to note that this issue is about "supported typed" and includes UTIs and Clipboard flavour names.
Do you still see that "This isn't really possible in a way that's interoperable across different applications" ?

paul

@rniwa
Copy link

rniwa commented Jun 19, 2021

Thank you @rniwa for the extra links. I don't see a disagreement except for "it will not work". We had already spotted at the time that media-types are not exactly the same as UTIs. That is why the MathML specification includes UTI specifications:

  • MathML: Macintosh Universal Type Identifier code public.mathml conforming to public.xml
  • MathML-presentation: Macintosh Universal Type Identifier code public.mathml.presentation conforming to public.mathml (described above) conforming to public.xml
  • MathML-content: Macintosh Universal Type Identifier code: public.mathml.content conforming to public.mathml (described above) conforming to public.xml

Presumably you're referring to the names of MathML encodings. I wasn't aware that this section existed but I don't think this recommendation is sound. public. prefix is reserved for the (operating) system. It does follow the general pattern of what Apple defined for other markup languages but it doesn't mean that it's officially sanctioned. Had I been aware of this, I would have raised a concern about this before the spec advanced to the recommendation state.

@polx
Copy link
Author

polx commented Jun 19, 2021

I'm afraid that the fact that the public subtree of UTIs is from Apple was not known when we did this (probably in 2009). The same holds for other types actually.

The wikipedia page about UTIs indicates that the public sub-tree is managed by Apple. But that page started in 2010. I do not remember the UTI documentation stating that exclusivity. Another aspect which I don't remember reading is that UTIs would be close to reversed domain names.

What can we do from now?

  • Go for some archeology into the different versions of Apple's documentation?
  • Change the UTIs? (they are currently implemented by MS Word and MathPlayer).
  • Try to involve an Apple active person to define the public mathml types? (who?)

Thanks in advance.

PS: On the MicroSoft side, we had in the Math Working Group a member of MicroSoft which assured that names are free to create by apps. An example of situation where a standard such as MathML is useful to propose converging names.

@rniwa
Copy link

rniwa commented Jun 20, 2021

Since I'm an engineer in the WebKit team at Apple, I can bring up this up with the relevant folks. I'm just pointing out that getting the spec changed isn't enough to make it work on Apple platforms.

For now, it might be advisable to say that UA MAY support MathML instead of MUST or SHOULD.

@polx
Copy link
Author

polx commented Jun 22, 2021

Allow me to indicate that MathML is by far not the only content-type which has allowed itself a public UTI. SVG and TAR are also there and not specified by Apple. Wikipedia's links to UTI shows a big list.

Maybe you could ask relevant persons what has been the process and what is the current process? If a process existed to (ask to) register the MathML UTI, we would have used it when MathML3 was written.

@snianu
Copy link
Contributor

snianu commented Jan 27, 2022

PS: On the MicroSoft side, we had in the Math Working Group a member of MicroSoft which assured that names are free to create by apps. An example of situation where a standard such as MathML is useful to propose converging names.

While it is true that the OS allows custom formats to be defined by the app, it is not, however, guaranteed to be supported by other apps. The apps have to add explicit support for these types which makes me wonder if Pickling API would be useful here so we don't have to define MathML types in the platform standard clipboard formats which is standardized by the OS. @BoCupp-Microsoft @annevk @whsieh @mbrodesser FYI.

@polx
Copy link
Author

polx commented Feb 2, 2022

Super interesting the Pickling API. Where do comments go for this?
Even in the math-world it will not be rare that sub-types or alternative types will emerge.

@snianu
Copy link
Contributor

snianu commented Feb 2, 2022

@polx You can open issues in this repo for Pickling API.

@polx
Copy link
Author

polx commented Oct 19, 2022

Hello @rniwa ,

is there a way to resolve the debate about public.* UTIs that originated from other sources than Apple's applications?

It seems to me that either Apple establishes a process (just as there was one for the four-letter codes) or tacitly accepts older UTI-indications within specifications and applications (probably what's currently happening but the issue is broader than MathML) or declares explicitly a problem with such UTIs.

Thanks in advance.
Paul

@polx
Copy link
Author

polx commented Sep 2, 2023

Hello @rniwa , any chance that someone of Apple shows up in the MathML working group's meeting (Wednesday 17:00-18:30) at TPAC?

@polx
Copy link
Author

polx commented Sep 2, 2023

(to discuss it the debate about the UTI's naming)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants