Features and optional tags for sage modules provided by separate distributions

[mkoeppe]

... so that we can start writing # optional - sage.symbolic and similar.

We use it in #32432 (sagemath-polyhedra) to skip doctests that depend on sage.graphs, sage.combinat, sage.rings.number_field etc.

CC: @kliem @jhpalmieri

Component: doctest framework

Author: Matthias Koeppe

Branch/Commit: 4558791

Reviewer: John Palmieri, Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/32614

[mkoeppe]

Branch: u/mkoeppe/features_and_optional_tags_for_sage_subset_distributions

[mkoeppe]

comment:2

Currently optional tags can contain neither . nor -:

sage:         sage: from sage.doctest.parsing import parse_optional_tags
....: 
sage: parse_optional_tags("sage: #optional -- my.p.kg")
{'my'}
sage: parse_optional_tags("sage: #optional -- sagemath-symbolics")
{'sagemath'}

---

New commits:

9096edb

sage.features.sagemath: New

[mkoeppe]

Commit: 9096edb

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2938e13	src/sage/doctest/parsing.py: Allow . in optional tags
747b458	sage.doctest.control, sage.features.sagemath: Provide/use optional tags

[sagetrac-git]

Changed commit from 9096edb to 747b458

[mkoeppe]

Author: Matthias Koeppe

[sagetrac-git]

Changed commit from 747b458 to 5e04a85

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

e3a541e	Fixup
5e04a85	src/sage/features/sagemath.py: Fixup

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

66cf3e1

src/sage/doctest/control.py: Fixup handling of sage_optional_tags

[sagetrac-git]

Changed commit from 5e04a85 to 66cf3e1

[sagetrac-git]

Changed commit from 66cf3e1 to 1ec0c48

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

1ec0c48

src/sage/features/sagemath.py: Add sage.rings.number_field

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0063749

src/sage/features/sagemath.py: Add features for modules that were optional extensions

[sagetrac-git]

Changed commit from 1ec0c48 to 0063749

[tscrim]

comment:11

Next round of stupid questions: How is this going to work? Are we doing to have to add many such markers to a lot of doctests? How will we as developers test which ones we need to add?

[tscrim]

comment:12

Replying to @tscrim:
Are we doing to have to add many such markers to a lot of doctests?

I guess this part will be alleviated by #30778.

[mkoeppe]

comment:13

Replying to @tscrim:

How will we as developers test which ones we need to add?

By running the test infrastructure of the subset distributions, such as the one added in #32432 (sagemath-polyhedra). This can be done locally, but we will also run this automatically on GH Actions

[jhpalmieri]

comment:14

Still no hyphens allowed in # optional - keyword? In any case, please also change the documentation in developer/coding_basics.rst to match the behavior (currently says "Any punctuation (periods, commas, hyphens, semicolons, ...) after the first word ends the list of packages."). I think we should have a clearly defined (and well-thought out, or is that too much to ask?) syntax for the keywords. This section of the documentation should ideally also provide examples in which tags are combined, as in # optional - abc, long time. And I guess that is why we chose to parse # optional - abc, xyz as abc rather than abc and xyz. It is a little odd that with the proposed changes, # optional - abc. long time will be treated very differently than # optional - abc, long time.

[mkoeppe]

comment:15

Replying to @jhpalmieri:

Still no hyphens allowed in # optional - keyword?

That's right -- this reflects the fact that our tags for optional packages use the spkg names, which use underscore, not dash.
Dashes appear in Python distribution package names (tools like setuptools normalize underscores to dashes).

[mkoeppe]

comment:16

Replying to @jhpalmieri:

It is a little odd that with the proposed changes, # optional - abc. long time will be treated very differently than # optional - abc, long time.

Using . like this has not been observed in the wild, as can be checked with git grep '#.*optional.*[.]'

[mkoeppe]

comment:17

I'll update the documentation

[sagetrac-git]

Changed commit from 0063749 to 14fd1e5

[mkoeppe]

Dependencies: #30887

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

ea548d7	sage.features.four_ti_2: New, use it in sage.interfaces.four_ti_2, sage.sandpiles
f826ded	build/pkgs/4ti2/spkg-configure.m4: Check for executable's with prefix 4ti2_ too
56016ce	use AC_LINK_IFELSE instead of obsolete AC_TRY_LINK
2b45b77	sage.feature.join_feature: New, factored out from LatteFeature; use it to implement FourTi2Feature
5c23cc9	DocTestReporter: Fix 'sage -t --optional=all'
1b8634d	sage.doctest.external: Add 4ti2
d9d4f99	Merge tag '9.4.beta6' into t/30887/public/30887
646e182	src/sage/features/four_ti_2.py: Move import of SAGE_ENV inside the `__init__` method, to remove confusion of sage.misc.dev_tools
180e31d	Merge #30887
10e8d63	sage.features.sagemath: Use JoinFeature when tag is different from the actually tested module

[sagetrac-git]

Changed commit from 27c53ac to 10e8d63

[sagetrac-git]

Changed commit from 10e8d63 to 654d09c

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

654d09c

sage.features.sagemath: Change sage_optional_tags to sage_features

[sagetrac-git]

Changed commit from 654d09c to f63a7d0

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f63a7d0

src/sage/features/: Move features depending on optional packages to separate files

[tscrim]

Reviewer: John Palmieri, Travis Scrimshaw

[tscrim]

comment:25

John, any other comments? I think I am ready to set a positive review.

[jhpalmieri]

comment:26

I am happy with it. Please go ahead with a positive review when you are ready.

[mkoeppe]

comment:28

Thanks for the review!

[sagetrac-git]

Changed commit from f63a7d0 to 4558791

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

4558791

Merge tag '9.5.beta3' into t/32614/features_and_optional_tags_for_sage_subset_distributions

[mkoeppe]

Changed dependencies from #30887 to none

[vbraun]

Changed branch from u/mkoeppe/features_and_optional_tags_for_sage_subset_distributions to 4558791