typeops: extend make_simplified_union fast path to enums #9394
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
huguesb
force-pushed
the
pr-faster
branch
4 times, most recently
from
129b2b5 to
57ce8eb
Compare
In PR python#9192 a fast path was created to address the slowness reported in issue python#9169 wherein large Union or literal types would dramatically slow down typechecking. It is desirable to extend this fast path to cover Enum types, as these can also leverage the O(n) set-based fast path instead of the O(n**2) fallback. This is seen to bring down the typechecking of a single fairly simple chain of `if` statements operating on a large enum (~3k members) from ~40min to 12s in real-world code! Note that the timing is taken from a pure-python run of mypy, as opposed to a compiled version.
|
Ping, can I get a review? For typechecking our entire codebase, this single change brings the mypy runtime from 12min down to 5min! |
gvanrossum
reviewed
Sep 16, 2020
msullivan
requested changes
Oct 18, 2020
|
I believe the existing test suite covers this code very well. It definitely caught the mistake I originally made of skipping the slow path entirely for literals (which breaks down when strict optional is false as indicated in the comment). And since there are no intended behavior changes, it's unclear to me what new tests to add. |
|
ping on this—does hugues' comment above about the existing test suite not convince? |
|
I see it now has conflicts anyway :/ |
|
ping |
|
Yeah, OK. I'll merge if someone fixes the conflicts. |
|
(I just did it) |
huguesb
pushed a commit
to huguesb/mypy
that referenced
this pull request
Jan 21, 2022
As a followup to python#9394 address a few more O(n**2) behaviors caused by decomposing enums into unions of literals.
huguesb
pushed a commit
to huguesb/mypy
that referenced
this pull request
Jan 21, 2022
As a followup to python#9394 address a few more O(n**2) behaviors caused by decomposing enums into unions of literals.
huguesb
pushed a commit
to huguesb/mypy
that referenced
this pull request
Apr 17, 2022
As a followup to python#9394 address a few more O(n**2) behaviors caused by decomposing enums into unions of literals.
JelleZijlstra
pushed a commit
that referenced
this pull request
Apr 18, 2022
As a followup to #9394 address a few more O(n**2) behaviors caused by decomposing enums into unions of literals.
JukkaL
pushed a commit
that referenced
this pull request
Apr 20, 2022
As a followup to #9394 address a few more O(n**2) behaviors caused by decomposing enums into unions of literals.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In PR #9192 a fast path was created to address the slowness reported
in issue #9169 wherein large Union or literal types would dramatically
slow down typechecking.
It is desirable to extend this fast path to cover Enum types, as these
can also leverage the O(n) set-based fast path instead of the O(n**2)
fallback.
This is seen to bring down the typechecking of a single fairly simple
chain of
ifstatements operating on a large enum (~3k members) from~40min to 12s in real-world code! Note that the timing is taken from
a pure-python run of mypy, as opposed to a compiled version.