Computed "__all__" value results in nothing being exported.

[mrolle45]

Bug Report

I have two modules:

A.py:
__all__ = 'Foo Bar'.split()
Foo = 0
Bar = 1

B.py:
from A import *
Foo

To Reproduce
mypy .

Expected Behavior
No errors

Actual Behavior

B.py:2: error: Name "Foo" is not defined
Found 1 error in 1 file (checked 2 source files)

Note:
If you make the change:

A.py:
__all__ = ('Foo', 'Bar')
Foo = 0
Bar = 1

Then the output is:

Success: no issues found in 2 source files

In method semanal.SemanticAnalyzer.process__all__(self, s: AssignmentStmt), it only recognizes s.rvalue as being either a TupleExpr or a ListExpr.
In this particular case, it is a CallExpr node. The callee is a MemberExpr with a StrExpr as the object and split as the member name.

Other cases that mypy should recognize would include redefining __all__, as in __all__.append("Baz").

I realize that by design, mypy never executes any user code, so it would be inappropriate to compile and execute the rvalue expression (i.e. there could be something dangerous). However, I think that by analyzing the syntax tree, it can be determined that it has constant value, and what that value is.
I wonder when it is always safe to compile and execute an arbitrary expression. Can you construct some sort of sandbox which is isolated from the current running environment?

At any rate, I'd like to see a robust ability in mypy to recognize expressions which have a constant value.

Your Environment

• Mypy version used: 0.931
• Mypy command-line flags: none

[erictraut]

Maintainers of the various Python type checkers had a discussion about this topic on the typing-sig email list a couple of years ago. In particular, we discussed __all__ and which expression forms should be supported by static analysis tools. As you point out, static analysis tools (including mypy) do not execute code, so they cannot support arbitrary program logic. However, there are certain expression forms that are commonly used in Python libraries to manipulate __all__. Custom logic needs to be added for each supported expression form — and for every static analysis tool, so there's value in keeping the list relatively short.

I analyzed a bunch of libraries and, based on this analysis, recommended a list of forms that static analysis tools should support. This list is documented here.

I'll also paste the list here for completeness:

__all__ = ('a', b')
__all__ = ['a', b']
__all__ += ['a', b']
__all__ += submodule.__all__
__all__.extend(['a', b'])
__all__.extend(submodule.__all__)
__all__.append('a')
__all__.remove('a')

Pyright supports this list of expression forms (and no others).

I'll note that the list does not include split(), which is a pattern I haven't seen used in any library. I would recommend not using such an atypical expression form when it confers no advantage over the more typical forms.

[hauntsaninja]

+1 to everything Eric said.

I think mypy currently supports most things on Eric's list; I believe the forms mypy does not yet support are the ones involving submodule.__all__ and .remove. There is no plan to support .split.

[AustinScola]

Is there any plan to add support for the forms involving submodule.__all__? I think it is useful for re-exporting from multiple submodules without having to list everything out.

[ikonst]

@AustinScola Mind creating a separate issue for this? Otherwise this legitimate feature request might be lost in the shuffle.

And then we should either:

• close this issue as Not Planned
• add a notice (error?) for computed __all__ directing users to documentation (I think @erictraut's and @hauntsaninja's comments above are informative and valuable, and it's too bad this isn't documented in mypy's docs)

[hauntsaninja]

Okay, given that I've merged #15279 and there's #15300 for the submodule case, I'm going to close this out. Documentation PR welcome (it could be useful to create an "infrequently asked questions" page for this kind of thing)