[FEATURE] - Styling Toolbar

[WildYellowfin]

I was planning on building this in my project, but since it might be of use to others I wanted to make a proposal here.

Font color, background color, font size, math variant. Also environments (align, gather, cases, matrix...), this would be a good place to add controls as described in #600. Something like this:

This is really for desktop, since the virtual keyboard is great on mobile.

I would approach this by having users supply DOM elements (similar to custom keyboard container) in which I insert an array of controls. These can then be styled as desired (eg. vertical flexbox).

[arnog]

That's an interesting proposal.

This seems like it could fit in with the virtual keyboard, though. The virtual keyboard already has a toolbar (for cut/copy/paste/undo/redo). I could see that being a "second row" toolbar (above or below the current one).

And I think I would go with a JSON description of the toolbar, like for the custom keyboard layouts. It could be based on the JSON format for the keyboard layout, since those are effectively keycaps with commands attached.

[arnog]

(BTW I'm in the middle of making some improvements to the VKB, in particular so it can be used as a "pop-up" keyboard, and not always at the bottom of the screen)

[WildYellowfin]

Cool! Is it possible to show / hide keys based on the position of the cursor? That way the matrix buttons could only show up when relevant.

I also think it would be nice to be get an indication of the current selection size / color / variant. If not, I suppose it is hackable.

[arnog]

Yes, the undo/redo buttons already change their state appropriately. And cut/copy only appear where the selection is not empty. So, the state of the "style" toolbar could definitely change based on the selection or position of the caret.

[WildYellowfin]

I've been reading through the keyboard logic and I think I understand a bit better how I would do this. Basically, each style attribute would be a key, and its various values would be key cap variants. I see where I would conditionally render the keys, and I also think it will be possible to modify their appearance based on the selection style.

There are two features I would like to build that I am not sure how to approach:

Firstly, I prefer having the variant scrim open on click for color / font size. I know I can make the key call showAlternateKeys, but I believe the scrim automatically closes on mouseup. I wonder if I can set some attribute on altContainer to allow some variant popovers to behave like menus.

Secondly, it would be nice to have an on-hover preview of style changes.

Thoughts?

[arnog]

I'll have a look, but I think you should be able to control when the scrim gets closed. That said, it would probably make sense to add a new type of panel other than the variant panel (or 'alt-keys' as it's currently called), that would pop-up on click. I can look at that.

Tell me more about what you're thinking for the hover preview. Are you thinking about showing the selection with the style applied in a tooltip next to the styling button? Or something else?

[WildYellowfin]

Yes I suppose that’s best.

The hover preview would work like fonts in many rich text editors, where the style is temporarily applied to the selection. So the style would be applied on mouse enter, and then reverted on mouse leave.

[WildYellowfin]

@arnog I'm working on this at the moment, and so far I have added methods to ModelPrivate and VirtualKeyboard to allow my makeStylebar to retrieve the intersected style of the current selection. This works great - except for shared keyboards...

Is there a way for me get information from the focused mathfield? As far as I can tell, the regular toolbar is built on selection change, which is fired from within the model. But I think the stylebar should be there before any selection changes take place.

[arnog]

That's great!

You have probably seen and modified VirtualKeyboard.updateToolbar()? It gets called whenever the content or selection changes and is passed the currently focused mathfield. There is indeed a bug right now where it doesn't get called initially. You can ignore this bug for now.

I'm in the process of rearchitecting the virtual keyboard so that there is only a shared virtual keyboard. This will simplify the architecture, and will also simplify usage when there are multiple mathfields on the page. It will also take care of this bug, so that updateToolbar() does get called initially.

Make sure you stay in sync with head. I pushed an update this morning. Also, if you'd like to push WIP updates, that would be fine by me.

[WildYellowfin]

Ahh I was specifically working around the updateToolbar since I thought it was intended not to show initially, but this makes my life easier.

It looks like the remote keyboard passes a public Mathfield to updateToolbar, so I will move the selectionStyle getter to the public class (it probably ought to be exposed anyways). Should be easy enough to work around you, since most of what I need to do is a couple new functions. I'll push to my fork and stay up to date.

[arnog]

OK, turns out it's a bit more complicated than I was hoping. In order to deal with iframes, the keyboard and the mathfield are isolated and the mathfield instance cannot be passed to updateToolbar(). Instead, a proxy mathfield will be passed that will exposed the properties and functions necessary for the toolbar, a subset of the full MathfieldElement. I presume that selection, value, undo state and styling information will all be relevant and I'll create a proxy that deals with that (by using cross-frame messaging). Let me know if there's any other info that you would need exposed for your purpose and I'll expand the proxy as needed.

[WildYellowfin]

Ah, no matter. I’ve mostly worked on the color and font styling for now (with a fun little color picker), so I haven’t yet looked into the environment manipulation features. Without having looked at the keyboard handlers, I don’t know how the logic for adding row / column work, but I will need to know when to show the buttons, and similarly know when the selection is in a matrix.

For the styles, I have added a method on MathfieldElement that gets the intersection of styles on the current selection (and the adopted style if collapsed).

[WildYellowfin]

Hey @arnog, just a little check in on this after the keyboard changes. I've added a selectionStyle getter to MathfieldPrivate, and I see how I would add it to the proxy. The menus (color, font variants etc) I am building are similar to key variants, but in many ways simpler since the styling toolbar won't be customizable like the key-caps. are (which means I don't need to mess with serialized functions).

For the colors, I was thinking of adding a static themeColors to MathfieldElement, which would allow developers to sync with their site's color scheme. This also enables users to add color swatches (w ` to the styling menu, and have them saved statically.

Also, in this new keyboard paradigm I'm thinking it should be possible to forward actions of a rich-text-editor toolbar to a math field via the keyboard. This is our use case at Perplex, and it would be nice if our users could control mathfield styles with the header toolbar. Could I for example call
window.mathVirtualKeyboard.executeCommand(["applyStyle", someStyle])?

If so, it might be nice for the styling toolbar to be an API for integrating with broader toolbars.

Last comment, re the onhover preview. I can't tell whether changes made via executeCommand get a snapshot. If not, I think I could write a mouseenter handler that executes a command, and a mouseleave event that reverts that command.

[arnog]

@WildYellowfin

themeColors or something like it makes sense. Maybe colorPalette? Having it as a static makes sense, although I realized that I've kept the colorMap and backgroundColorMap properties as per-instance. Perhaps they should be static as well and work in conjunction with colorPalette? Also, you'll want to distinguish "ink" from "background" colors. A color "red" for example should have different values for use as a background color (less saturated, lighter) or as a foreground color (more saturated, darker).

You should be able to apply formatting to a mathfield using mathVirtualKeyboard.executeCommand(). That's certainly the intent anyways. It hasn't been heavily tested yet, so if there are bugs or undesirable behavior using that code path they should be addressed.

Commands executed via executeCommand interact with the undo manager so that commands are undoable as you would expect. That means that most commands are undoable (but some aren't or have a more complicated algorithm, for example when inserting character each character is not undoable individually). In the case of styling, they should be undoable. If they're not, that's a bug. So your idea to use the undo manager to temporarily apply the command should work.

[WildYellowfin]

Awesome @arnog! I wonder if the ink / paper colors should have the same base, with a shift to saturation applied where necessary. If I add an ink color to my palette as a user, I might want the "same" color in the background palette.

For the undo, could I write a command to pop one item from the history? It would be incorrect in my view for a "redo" to reapply a temporary style after a mouseleave.

[arnog]

@WildYellowfin unfortunately calculating the background/ink from a base color is... complicated. You do have to adjust the saturation, but also the lightness, although how much you do depends on the hue because the human eye is more sensitive the lightness of some hues than others, so if you do it mathematically, it will look wrong.

Ah, yes, there's a mf.popUndoStack() function, but having it as a command would be what you want in your case.

[arnog]

BTW, I should add that the colors from FOREGROUND_COLORS and BACKGROUND_COLORS, which are mapped via colorMap() and backgroundColorMap() have been very carefully tuned to match several criteria:

• they are distinctive from each other, including for people with color vision deficiency
• they are equally distributed according to human vision perception (not physical reality)
• they offer sufficient contrast when foreground applied to background

It is very difficult when picking a set of random colors to meet all those criteria.

[WildYellowfin]

A comment here on what has been done:

• Array editing is implemented via the environmentPopover
• No styling toolbar has been built, but it is possible to build one using the mathVirtualKeyboard.style getter (returns the style of the selection) and the preexisting applyStyle command.

[arnog]

The MathLive menu now provides options to style the selection, including color, variant, and matrix editing.