Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start noting what's up and what goals have been met #799

Merged
merged 5 commits into from
Feb 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 117 additions & 3 deletions FUTURE.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,120 @@

.. contents::

The following 2023 road map that appears the 6.0.0 hasn't gone through enough discussion. This provisional.
Check the github repository for updates.


2023 Roadmap
============


When the release settles, "Forms, Boxing, And "Formatting" is the next
large refactor slated. Having this will allow us to supporting Jupyter or other front
ends. And it is something that is most visibly wrong in Mathics3 output.

See ``PAST.rst`` for how the 2023 Roadmap compares to the 2022 Roadmap.

Forms, Boxing and Formatting
----------------------------

This remains the biggest holdover item from 2022, and seems easily doable.
It hinders interaction with Jupyter or other front ends.

Right now "Form" (a high-level specification of how to format) and
"format" (a low level specification of how output is encoded) are sometimes muddied.

For example, TeXForm may be a "Form", but output encoded for AMS-LaTeX is done by a *formatter*.
So AMS-LaTeX rendering and other kinds of rendering should be split into its own rendering for formatter module.
Currently we have asymptote, and svg "format" modules.

Back to high-level again, Boxing is something that can be written in Mathics3, and doing this at
least initially ensures that we have design that fits more naturally
into the Wolfram Language philosophy.


Performance
-----------

While this is probably more of an overall concern, for now, big refactoring needed here, such as
going over pattern matching, will get done after Forms, Boxing and Formatting .

Forms, Boxing and Formatting will however contain one improvement that
should speed up our performance: separating M-Expression evaluation from
Box "evaluations).

We expect there will be other little opportunities here and there as we have seen in the past.


More Custom kinds of (compound) Expressions
+++++++++++++++++++++++++++++++++++++++++++

We scratched the surface here with ListExpression. Associations and Python/Sympy/numpy literals can be customized with an aim towards reducing conversions from and to M-expressions.
A number of compound expressions, especially those which involve literals are more efficiently represented in some other way. For example,
representing a Mathics3 Association as a Python ordered dictionary, a Mathics3 List as a Python list or tuple, or as a numpy array.


Further Code Reorganization in Core and Eval
--------------------------------------------

Core object like ``BaseElement`` and possibly ``Symbol``, (and
probably others) are too "fat": they have too many custom methods that
are not applicable for most of the subclasses support. It is likely
another pass will be made over this.

We have started moving "eval" code out of the "eval" methods and into its own module.

Mathics3 Module Enhancement
---------------------------

While we have put in quite a bit of effort to get these to be 6.0.0 compliant. There is still more work to do, and there are numerous bugs there.
Refactoring code to generate Graphs in ``pymathics.graph`` might happen. Porting the ``pymathics.graph`` code to use NetworkX 3.0 would be nice;
``pymathics.natlang`` could also use a look over in terms of the libraries we are using.

Python upgrades
---------------

After Mathics3 Version 6.0.0, Python 3.6 will be dropped and possibly 3.7. Changes are needed to support 3.11 so we will be focusing on 3.8 to 3.11.

We have gradually been using a more modern Python programming style
and idioms: more type annotation, use of ``isort`` (order Python
imports), ``black`` (code formatting), and ``flake8`` (Python lint
checking).


Deferred
--------

As mentioned before, pattern-matching revision is for later. `This
discussion
<https://github.com/Mathics3/mathics-core/discussions/800>`_ is a
placeholder for this discussion.

Overhauling the documentation to use something better supported and
more mainstream like sphinx is deferred. This would really be nice to
have, but it will require a bit of effort and detracts from all of the other work that is needed.

We will probably try this out in a limited basis in one of the Mathics3 modules.

Speaking of Mathics3 Modules, there are probably various scoping/context issues that Mathics3 modules make more apparent.
This will is deferred for now.

Way down the line, is converting to a more sequence-based interpreter which is needed for JIT'ing and better Compilation support.

Likewise, speeding up startup time via saving and loading an image is something that is more of a long-term goal.

Things in this section can change, depending on the help we can get.


Miscellaneous
-------------

No doubt there will be numerous bug fixes, and builtin-function additions especially now that we have a better framework to support this kind of growth.
Some of the smaller deferred issues refactorings may get addressed.

As always, where and how fast things grow here depends on help available.


2022 Roadmap
=============

Expand Down Expand Up @@ -57,11 +171,11 @@ The current home-grown documentation should be replaced with Sphynx and autodoc.
Compilation
-----------

Complation is a rather unsophisticated process by trying to speed up Python code using llvmlite. The gains here will always be small compared the kinds of gains a compiler can get. However in order to even be able to contemplate writing a compiler (let alone say a JIT compiler), the code base needs to be made to work more like a traditional interpreter. Some work will be needed just to be able or create a sequence of instructions to run.
Compilation is a rather unsophisticated process by trying to speed up Python code using llvmlite. The gains here will always be small compared the kinds of gains a compiler can get. However in order to even be able to contemplate writing a compiler (let alone say a JIT compiler), the code base needs to be made to work more like a traditional interpreter. Some work will be needed just to be able or create a sequence of instructions to run.

Right now the interpreter is strictly a tree interperter.
Right now the interpreter is strictly a tree interpreter.

Simpiler Things
Simpler Things
---------------

There have been a number of things that have been deferred:
Expand Down
55 changes: 55 additions & 0 deletions PAST.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
While ``FUTURE.rst`` gives our current roadmap, this file ``PAST.rst``
looks the other way for what we have accomplished when compared to what _was_ planned in ``FUTURE.rst``

While this is also listed in ``CHANGES.rst``, here we extract that to
make it easier to see the bigger picture without the details that are
in ``CHANGES.rst``.

Progress from 2022
==================

A fair bit of code refactoring has gone on so that we might be able to
scale the code, get it to be more performant, and more in line with
other interpreters.

The buitin Functions have been organized into grouping akind to what is found in WMA.
This is not just for documentation purposes, but it better modularizes the code and keep
the modules smaller while suggesting where functions below as we scale.

Image Routines have been gone over.

A number of Built-in functions that were implemented were not accessible for various reasons.

Mathics3 Modules are better integrated into the documentation.
Existing Mathics3 modules pymathics.graph and pymathics.natlang have
had a major overhaul, although more is needed. And will continue after th 6.0.0 release

We have gradually been rolling in more Python type annotations and
current Python practices such as using ``isort``, ``black`` and ``flake8``.


Boxing and Formatting
---------------------

While some work on formatting is done has been made and the change in API reflects a little of this.
However a lot more work needs to be done.

Excecution Performance
----------------------

This has improved a slight bit, but not because it has been a focus, but
rather because in going over the code organization, we are doing this
less dumb, e.g. using Symbols more where symbols are intended. Or
fixing bugs like resetting mpmath numeric precision on operations that
need to chnage it temporarily.

Simpler Things
--------------

A number of items here remain, but should not be thought as independent items, but instead part of
"Forms, Boxing and Formatting".

"Making StandardOutput of polynomials match WMA" is really are Forms, Boxing and Formatting issue;
"Working on Jupyter integrations" is also very dependant this.

So the next major refactor will be on Forms, Boxing and Formatting.