What is up with $BoxForms?

[rocky]

One of the things that caught my attention was that we don't have a Doc example for $BoxForms and it is always a good idea to add an example even if it is something as simple as

>> $BoxForms
 = ...

This is good because in the Django interface the user has an example to run and see what is currently set by clicking the = on the side. And in TeX documentation you see the example as well.

I started to add this and then noticed that $BoxForms is not documented in WMA. Is it defined?

And then I notice this comment:

# TODO: Differently from the current implementation, MakeBoxes should only
# accept as its format field the symbols in `$BoxForms`. This is something to
# fix in a following step.

What is the status of $BoxForm?

[mmatera]

@rocky, indeed, this is another thing that is not clearly documented in WMA. See for instance
https://reference.wolfram.com/language/ref/message/$BoxForms/formset.html?view=all

Something that I discovered digging into the way in which WMA converts an expression into a Box structure, is that (at least in the current version of WMA), MakeBoxes just accepts (in principle) two "forms" as a second parameter: StandardForm and TraditionalForm, which are the elements in $BoxForms. I am not sure if this was always the behavior, or not, but in Mathics, we had implemented it in a different way. So, for instance, we accept MakeBoxes[expr, InputForm], which is how MakeBoxes[InputForm[expr]] is evaluated.
On WMA, on the other hand,

In[1]:=MakeBoxes[F[x,"a"], InputForm]
MakeBoxes::boxfmt: 
   InputForm in MakeBoxes[F[x, a], InputForm]
     is not a box formatting type. A box formatting type is any member of
     $BoxForms.

Out[1]= MakeBoxes[F[x, a], InputForm]

Also, to add InputForm to the list $BoxForms it is not enough to make it work: https://reference.wolfram.com/language/ref/message/ParentForm/deflt.html

In[2]:= AppendTo[$BoxForms, InputForm]
Out[2]= {StandardForm, TraditionalForm, InputForm}
In[3]:=MakeBoxes[F[x,"a"], InputForm]

ParentForm::deflt: 
   The ParentForm of ParentForm[InputForm] is not defined on $BoxForms.

ParentForm::deflt: 
   The ParentForm of ParentForm[InputForm] is not defined on $BoxForms.

ParentForm::deflt: 
   The ParentForm of ParentForm[InputForm] is not defined on $BoxForms.

General::stop: Further output of ParentForm::deflt
     will be suppressed during this calculation.

Out[8]= RowBox[{F, [, RowBox[{x, ,, "a"}], ]}]

The good part of that is so obscure, is that it minimizes the risk of falling in incompatibilities with 3rd party packages, so we could keep for a while the way in which Mathics interprets MakeBoxes.
My plan now is

• Check what we need to do in order to make that all the broken test cases get fixed.
• Build an infrastructure like mathics.core.formatters, to implement the MakeBoxes work.

Once this part is done, we could evaluate if we want to change the behavior of this part of the interpreter to make in closer to WMA.

[rocky]

I have very little understanding in this area. Why would one want to evaluatie MakeBoxes[InputForm[expr]]?

InputForm is the form that something is read in. Boxing is something is a step you do on before output. A conversion, at least preliminary, should be made from InputForm to one of the kinds of output Forms.

[mmatera]

Indeed, if you define MakeBoxes[InputForm[expr]], it is never evaluated in the WMA interpreter: however, consider these two examples, (with output evaluated in WMA)

In[1]:=MakeBoxes[InputForm["a"]]
Out[1]= InterpretationBox[
 StyleBox["\"a\"", ShowStringCharacters -> True, 
  NumberMarks -> True], InputForm["a"], Editable -> True, 
 AutoDelete -> True]

(in Mathics, Out[1]=StyleBox["a", ShowStringCharacters -> True] )

In[1]:=MakeBoxes[OutputForm["a"]
Out[1]= InterpretationBox[PaneBox["\"a\""], OutputForm["a"], 
 Editable -> False]

(in Mathics, Out[1]="\"a\"" )

In Mathics, to achieve this similar behaviour we have added the rule

MakeBoxes[InputForm[expr_]]->MakeBoxes[expr, InputForm]

but I think we can implement it in a different way...

[rocky]

if you define MakeBoxes[InputForm[expr]]

Why would I want to do that? There are all sorts of nonsensical and complicated things could be defined that I don't care to worry about.

Mathics doesn't have Intpretation or InterpretationBox, so of course you aren't going to expect that to appear in the output. And it doesn't look like the we have documentation for StyleBox or option ShowStringCharacters even though they do exist. (I guess we'd have to read the code to understand what they do.)

.. In Mathics, to achieve this similar behaviour we have added the rule

I don't see that Out[1]=InterpretationBox[PanelBox["\"a\""], OutputForm["a"], Editable -> False] looks like Out[1]="\"a\"".

but I think we can implement it in a different way...

I am concerned about trying to implement something that isn't well explained which may be a result it being a problem not well understood either.

So I guess before offering solutions, maybe we should take stock and describe specific problems.

[mmatera]

if you define MakeBoxes[InputForm[expr]]

Why would I want to do that? There are all sorts of nonsensical and complicated things could be defined that I don't care to worry about.

Because we have already done here->

mathics-core/mathics/builtin/makeboxes.py

Line 470 in a0f456b

"MakeBoxes[(form:FullForm|InputForm)[expr_], StandardForm|TraditionalForm|OutputForm]": "StyleBox[MakeBoxes[expr, form], ShowStringCharacters->True]",

If you remove that line, the code is broken....

Mathics doesn't have Intpretation or InterpretationBox, so of course you aren't going to expect that to appear in the output. And it doesn't look like the we have documentation for StyleBox or option ShowStringCharacters even though they do exist. (I guess we'd have to read the code to understand what they do.)

To clarify the point: the behavior we want to reproduce is that InputForm["a"] and OutputForm["a"] produces different textual output. In one case, the printed text is surrounded by double quotes (so, if you copy and paste it in the interpreter, you get again a string) and in the other case, no. How are the boxes we use to represent that do not really matter too much. WMA changes that a lot of times throughout its history.

.. In Mathics, to achieve this similar behaviour we have added the rule

I don't see that Out[1]=InterpretationBox[PanelBox["\"a\""], OutputForm["a"], Editable -> False] looks like Out[1]="\"a\"".

but I think we can implement it in a different way...

I am concerned about trying to implement something that isn't well explained which may be a result it being a problem not well understood either.

This is why I am trying to change the implementation to something that we can describe in a slightly simpler way. In the current state, we have (as in many other areas) a lot of sparsed code, which heavily interacts among its parts. For this reason, my proposal is:

• Check what we need to do in order to make that all the broken test cases get fixed.
• Build an infrastructure like mathics.core.formatters, to implement the MakeBoxes work.

So I guess before offering solutions, maybe we should take stock and describe specific problems.

So, to start with the specific problems, my attempt is to make changes in order to that all the commented tests enumerated here

mathics-core/mathics/builtin/makeboxes.py

Line 374 in a0f456b

# TODO: Convert operators to appropriate representations e.g. 'Plus' to '+'

pass, and without adding more obscure and heavily intertwined code.

[rocky]

Ok. Thanks for the explanation.

Given this,$BoxForms should not include InputForm in its list. Because in WMA it isn't included. Efforts to "fix up" $BoxForms to include InputForm are going in the wrong direction with respect to WMA compatibility.

[mmatera]

No, the plan is not to "fix up" $BoxForms but MakeBoxes. Maybe I have to improve the comment.