Reinstate pymathics doc #778

rocky · 2023-02-05T13:54:38Z

This allows Mathics3 Modules loaded via -l or --load-module on the docpipetest.py command like to get loaded
using eval_LoadModule the underlying eval method for LoadModule[].

In other words, from Mathics3 core you can test Mathics3 Modules or select individual sections Mathics3 Module tests. But that also means, we are collecting XML-like documentation and test data which can then be put into the LaTeX-generated PDF.

I believe the information will go into the PCL (Python Pickle file), but I haven't tested that yet.

And I haven't looked at the piece that puts creates documentation.tex (mathics/doc/latex/doc2latex.py)

I think that will be another (small) PR. This is large enough as it is.

For expediency in being able to get this accomplished over the weekend in a limited time, there are/were several separable commits here. I am sure you know the pattern - you want to get X done but bug Y gets in the way. So you fix Y, and then bug Z pops up.

Over the week I will try to extract these so that they can be reviewed/discussed independently.

Here are two that I can think of though.

I would like to change the name Pymathics or Pymathics module to the simpler "Mathics3 Module" which has always been loaded via LoadModule.

The messages when a Mathics3 Module can't be loaded have been changed.

DRY code. docpipeline.py `-l` `--pymathics` probably works now These changes cannot get merge before Mathics3/mathics-core#778 is merged.

rocky · 2023-02-06T01:54:51Z

mathics/doc/latex_doc.py

        self.title = "Overview"
-        files = listdir(self.doc_dir)


A programmer who writes such a long piece of code like this without a single comment and in an __init__ method, should be graduated out of programming.

mmatera · 2023-02-06T14:04:19Z

mathics/builtin/assignments/assignment.py

     = $Failed
    """

    name = "LoadModule"
    messages = {
-        "notfound": "Python module `1` does not exist.",
-        "notmathicslib": "Python module `1` is not a pymathics module.",
+        "notfound": """Problem importing Python module: `1`.""",


If LoadModule produces the error notfound, it is because the module can not be found, not because there was a generic error in importing it. So, why the new message is better?

There is a difference between something existing and something importing. In fact, another check could be made to see if the file or module exists and give that message. The file or module may exist, but there is invalid Python or it may import modules that are not importable or cause circular dependencies.

OK, but the error we handle in importing the python module is about the existence of the library.

"Exist" is a vague or misleading word. The library can exist in some sense but not work. When I see an error message or warning I generally want to know what was tried. What was tried is that we ran a Python kind of "import": and that did not work,.

In Python, what we are capturing is:

>>> import amadeupmodulename Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'amadeupmodulename'

So, maybe "Python module 1 was not found in the system"
could be a more accurate message, isn't it?

There is a difference here. You issued the import from Python and not "not found" was given in response to that. So it is clear that Python import is where the problem lies. In Mathics3 though, the user a priori doesn't know that LoadModule runs a Python import underneath. It could have loaded the module using open() and done some magic there.

So using the words Python and import suggest what kind of thing is failing. In fact it suggests trying to run a Python "import".

What about Pymathics module `1` was not found in the system? Otherwise, it was just a suggestion. Please just keep it as you prefer.

I have been thinking about this some more. I do listen to your suggestions...

I think we can get the best of both worlds by catching any Python exception and passing back the message in the exception. In those cases where Python reports "not found" the message has that (indicating this is from an import).

In fact, it was probably a mistake using except ImportError rather then except Exception.

mmatera · 2023-02-06T14:06:04Z

mathics/doc/common_doc.py

@@ -689,251 +1017,32 @@ def __str__(self):
        return self.test


+# FIXME: think about - do we need this? Or can we use DjangoMathicsDocumentation and
+# LatTeXMathicsDocumentation only?


Are we using this? Maybe we could replace it with a function that produces the right class of object.

Yes, we are using this. Merging the two is another mess that I don't want to deal with right now.

If you want to try to do it go head. I will note that there are still lots of opportunities for DRYing the code and improving it

The motivation here was simply that we had code in Django to handle Mathics3 Modules and I wanted that used on the LaTeX/PDF side. So moving this code to common code seemed like the way to do this which both accomplishes what I want to do while improving the code.

However in doing so there have been small breakages here and there precisely because there were two sets of code and an improvement or necessary change was made in one copy but not the other. And the one with the improvement did not make it to the common code.

Last weekend I had my fill of trying to find and fix these kinds of things. Doing more in order to DRY the code more is left for the future for me.

mmatera · 2023-02-06T14:07:28Z

Apart from my comments, and the need of merging first #781 to avoid the hanging, LGTM.

mmatera · 2023-02-06T14:27:47Z

There is another thing: for some reason, the pymathics module is not loaded when the module doctests are run

rocky · 2023-02-06T18:07:07Z

There is another thing: for some reason, the pymathics module is not loaded when the module doctests are run

Works for me:

pyston mathics/docpipeline.py -x -l pymathics.graph -s BinomialTree
Mathics3 Module pymathics.graph loaded
Testing section(s): BinomialTree
Testing section: Graphs - Vertices and Edges / BinomialTree
b'   1 ( 0): TEST BinomialTree[3]'

What do you get when you run pyston mathics/docpipeline.py -x -l pymathics.graph -s BinomialTree

This PR fixes another issue with precision. In particular, this issue was the reason of the hangings in doctest in #778.

Move code from django_doc and latex_doc .py to common_doc.py In the process we should handle pymathics better. Remove a number of unused functions and variables like ``is_private`` and pymathics_doc_loaded.

See if we can get by with out using the want_sorting parameter

Pull out flaky Unique[] doctests and start a more robust pytest for these. more work is needed.

Fix bug in InterpretedBox

Reinstate hard-coded XMLDoc where self.doc_fn() is not available Reinstante want_sorting option

--pymathics -> --load-module. Note that one splits module names with a comma. The name Pymathics is deprecated

This should make it a little more robust.

* DRY doc code Much has been moved to mathics.doc.mathics.common now * Simplify: Remove bogus default_pymathics_modules See if we can get by without want_sorting * Reduce custom doc code... DRY code. docpipeline.py `-l` `--pymathics` probably works now These changes cannot get merge before Mathics3/mathics-core#778 is merged. * Reinstante guide section summaries * Reinstate want_sorting option * Remove "remake" for now * want sorting parameter adjustment

rocky marked this pull request as draft February 5, 2023 13:54

rocky force-pushed the reinstate-pymathics-doc branch from 6289259 to 3e3aba8 Compare February 5, 2023 13:56

rocky mentioned this pull request Feb 5, 2023

Small bugs... #779

Merged

rocky force-pushed the reinstate-pymathics-doc branch from 3e3aba8 to e6f50d1 Compare February 5, 2023 20:26

rocky added a commit to Mathics3/mathics-django that referenced this pull request Feb 5, 2023

Reduce custom doc code...

2026f7f

DRY code. docpipeline.py `-l` `--pymathics` probably works now These changes cannot get merge before Mathics3/mathics-core#778 is merged.

rocky mentioned this pull request Feb 5, 2023

Use doc common more Mathics3/mathics-django#186

Merged

rocky force-pushed the reinstate-pymathics-doc branch 2 times, most recently from 46d7d20 to f86cfa9 Compare February 6, 2023 01:30

rocky requested review from mmatera and TiagoCavalcante February 6, 2023 01:40

rocky force-pushed the reinstate-pymathics-doc branch from f86cfa9 to f885591 Compare February 6, 2023 01:47

rocky commented Feb 6, 2023

View reviewed changes

rocky force-pushed the reinstate-pymathics-doc branch from 4ab3e39 to 710339f Compare February 6, 2023 02:17

mmatera mentioned this pull request Feb 6, 2023

fix mpmath precision handling #781

Merged

mmatera reviewed Feb 6, 2023

View reviewed changes

rocky marked this pull request as ready for review February 7, 2023 14:28

mmatera added a commit that referenced this pull request Feb 7, 2023

fix mpmath precision handling (#781)

12f6aa3

This PR fixes another issue with precision. In particular, this issue was the reason of the hangings in doctest in #778.

rocky added 9 commits February 7, 2023 12:39

DRY doc code ...

ed84c68

Move code from django_doc and latex_doc .py to common_doc.py In the process we should handle pymathics better. Remove a number of unused functions and variables like ``is_private`` and pymathics_doc_loaded.

DRY more moving code to Documentation..

e6cac70

See if we can get by with out using the want_sorting parameter

better LoadModule option processing in docpipeline

736fc8e

Pull out flaky Unique[] doctests and start a more robust pytest for these. more work is needed.

docpipeline works again

de6d471

Fix bug in InterpretedBox

Revise --want-sorting help

69cd0b9

Reinstate x.is_private()

9712447

Preserve summary text in guide sections

5fe604f

Reinstate hard-coded XMLDoc where self.doc_fn() is not available Reinstante want_sorting option

Tweak docpipeline.py option

3cbe6a0

--pymathics -> --load-module. Note that one splits module names with a comma. The name Pymathics is deprecated

More COMMENTS!

cdcc00f

Adjust LoadModule error reporting...

e62333f

This should make it a little more robust.

rocky force-pushed the reinstate-pymathics-doc branch from 3e1f91a to e62333f Compare February 7, 2023 17:39

rocky merged commit 9a7ed16 into master Feb 7, 2023

rocky deleted the reinstate-pymathics-doc branch February 7, 2023 19:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reinstate pymathics doc #778

Reinstate pymathics doc #778

rocky commented Feb 5, 2023 •

edited

Loading

rocky Feb 6, 2023 •

edited

Loading

mmatera Feb 6, 2023

rocky Feb 6, 2023 •

edited

Loading

mmatera Feb 6, 2023

rocky Feb 6, 2023

mmatera Feb 6, 2023

rocky Feb 6, 2023 •

edited

Loading

mmatera Feb 6, 2023

rocky Feb 6, 2023

mmatera Feb 6, 2023

rocky Feb 6, 2023

mmatera commented Feb 6, 2023

mmatera commented Feb 6, 2023

rocky commented Feb 6, 2023

Reinstate pymathics doc #778

Reinstate pymathics doc #778

Conversation

rocky commented Feb 5, 2023 • edited Loading

rocky Feb 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rocky Feb 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rocky Feb 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmatera commented Feb 6, 2023

mmatera commented Feb 6, 2023

rocky commented Feb 6, 2023

rocky commented Feb 5, 2023 •

edited

Loading

rocky Feb 6, 2023 •

edited

Loading

rocky Feb 6, 2023 •

edited

Loading

rocky Feb 6, 2023 •

edited

Loading