Possible future directions for Mathics: pattern matching and parsing/operators

[rljacobson]

@rocky inspired a few thoughts I have about Mathics and what the team might consider for improvements to pattern matching and parsing and operator support.

Pattern matching

My own project Loris isn't the right choice for pattern matching in Mathics. What is interesting about the algorithm in Loris is not that it is performant but that it can handle infinitary theories–but not as I've implemented it! However, I am working on something that might be a good fit, but it's a ways off from being functional, I'm afraid. It is a reimplementation of the fastest known expression matching algorithms and focuses on performance and on packaging the algorithms in a generic library.

I am also interested in implementing many-to-one matching, but I haven't really started down that road yet. (Many-to-one is especially important for computer algebra where multiple patterns need to be matched against the subject at once.) I'm aware of the recent literature on the subject, though.

For a pure Python pattern matching library, you probably want MatchPy—at the very least talk to its developers. MatchPy is the most capable expression matching library in Python as far as I know.

Parsing and operators

You might already know about the WLTools / Wolfram Language Spec information I have: https://wltools.github.io/LanguageSpec/. It is a brain dump of most of what I know about the syntax and semantics of Wolfram Language. The most important thing there is the table of operator data I have collected. (I wrote an article about finding the data.) It was with this data in mind that I wrote about my opinions on the design of Pratt parsers. Life events kept me from writing Part II of that article, but I have implemented the algorithm in Rust, and it would be easy to implement it in Python. The Rust source code is heavily commented and should be accessible to anyone familiar with a major programming language.

The quantity and accuracy of my operator data far exceeds that of Mathics' present implementation. If there is interest, I can work on a rough draft, a sort of proof-of-concept parser for Mathics that incorporates everything I've learned. On the other hand, if the team is already happy with Mathics' scanner and parser implementations and isn't interested, my feelings won't be hurt.

[rocky]

This should probably done as a discussion to get a wider audience and well because it is a discussion.

Although I added the "discussion" feature for this repository, I think this really goes in mathics-core: https://github.com/Mathics3/mathics-core/discussions

Put that there, and I'll add the thoughts I have about this there.

[rljacobson]

Added "Pattern matching" to the thread you began, Mathics3/mathics-core/issues/800. Made another thread for parsing and operators, Mathics3/mathics-core/issues/802.