Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash on a project using lots of generics #14783

Closed
magwas opened this issue Feb 26, 2023 · 3 comments · Fixed by #14792
Closed

Crash on a project using lots of generics #14783

magwas opened this issue Feb 26, 2023 · 3 comments · Fixed by #14792
Labels
crash topic-overlap Overlapping equality check topic-paramspec PEP 612, ParamSpec, Concatenate

Comments

@magwas
Copy link

magwas commented Feb 26, 2023

Crash Report

source code of the project is at commit 2f2947e4401a3f8d69fa6c77e0108ad110f8af0f of
https://github.com/kode-konveyor/cdd-python

BTW, have you ever thought about adopting the principles of the type system of TypeScript?
They got it quite right. Typing generics in python is still quite a challenge even with 3.11.
See type: ignore comments in the project.

Traceback

$ mypy --show-traceback
src/shall/ShallEntity.py:13: error: Unused "type: ignore" comment
test/MeanWhileContract.py:13: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.2.0+dev.284142d6e29d4588107f5c68f95c43986a2e26e5
Traceback (most recent call last):
  File "/home/mag/.local/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/build.py", line 194, in build
    result = _build(
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/build.py", line 277, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/build.py", line 2923, in dispatch
    process_graph(graph, manager)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/build.py", line 3320, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/build.py", line 3421, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/build.py", line 2317, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 474, in check_first_pass
    self.accept(d)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/nodes.py", line 774, in accept
    return visitor.visit_func_def(self)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 959, in visit_func_def
    self._visit_func_def(defn)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 963, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 1035, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 1215, in check_func_def
    self.accept(item.body)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/nodes.py", line 1204, in accept
    return visitor.visit_block(self)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 2607, in visit_block
    self.accept(s)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 582, in accept
    stmt.accept(self)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/nodes.py", line 1400, in accept
    return visitor.visit_return_stmt(self)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 4127, in visit_return_stmt
    self.check_return_stmt(s)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checker.py", line 4161, in check_return_stmt
    self.expr_checker.accept(
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checkexpr.py", line 4881, in accept
    typ = node.accept(self)
          ^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/nodes.py", line 2053, in accept
    return visitor.visit_comparison_expr(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checkexpr.py", line 3045, in visit_comparison_expr
    if self.dangerous_comparison(left_type, right_type):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checkexpr.py", line 3171, in dangerous_comparison
    return self.dangerous_comparison(left.args[0], right.args[0])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/checkexpr.py", line 3180, in dangerous_comparison
    return not is_overlapping_types(left, right, ignore_promotions=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 404, in is_overlapping_types
    return are_tuples_overlapping(left, right, ignore_promotions=ignore_promotions)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 585, in are_tuples_overlapping
    return all(
           ^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 586, in <genexpr>
    is_overlapping_types(
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 505, in is_overlapping_types
    if all(
       ^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 506, in <genexpr>
    _is_overlapping_types(left_arg, right_arg)
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 279, in _is_overlapping_types
    return is_overlapping_types(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mag/.local/lib/python3.11/site-packages/mypy/meet.py", line 519, in is_overlapping_types
    assert type(left) != type(right), f"{type(left)} vs {type(right)}"
           ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: <class 'mypy.types.Parameters'> vs <class 'mypy.types.Parameters'>
test/MeanWhileContract.py:13: : note: use --pdb to drop into pdb

To Reproduce

(Write what you did to reproduce the crash. Full source code is
appreciated. We also very much appreciate it if you try to narrow the
source down to a small stand-alone example.)

Your Environment

  • Mypy version used: just built from git repo (1.2.0+dev.284142d6e29d4588107f5c68f95c43986a2e26e5)
  • Mypy command-line flags: --show-traceback
  • Mypy configuration options from mypy.ini (and other config files):
[mypy]
strict = True
explicit_package_bases = True
namespace_packages = True
#mypy_path = src:test
files = src,test
mypy_path = $MYPY_CONFIG_FILE_DIR/src:$MYPY_CONFIG_FILE_DIR/test
  • Python version used: 3.11.2-4 from debian sid
  • Operating system and version: Debian bullseye with some sid.
@magwas magwas added the crash label Feb 26, 2023
@AlexWaygood
Copy link
Member

AlexWaygood commented Feb 26, 2023

Thanks for the crash report! Here's a minimal repro:

from typing import Generic, ParamSpec

P = ParamSpec("P")

class Foo(Generic[P]): ...

def checker(foo1: Foo[[int]], foo2: Foo[[str]]) -> None:
    foo1 == foo2

Note that, interestingly, this only crashes if you run mypy with --strict-equality (which is enabled by default if you run mypy with --strict, as you're doing). That's because this crash is caused by an interaction between mypy's logic for the --strict-equality check and its logic for ParamSpecs.

https://mypy-play.net/?mypy=latest&python=3.11&flags=show-traceback%2Cstrict-equality&gist=e91391824b0fb34458a6775a660a29f9

@AlexWaygood AlexWaygood added topic-paramspec PEP 612, ParamSpec, Concatenate topic-overlap Overlapping equality check labels Feb 26, 2023
@AlexWaygood
Copy link
Member

The crash is reproducible as far back as 0.950, which is the first mypy version that properly supported ParamSpecs like this: https://mypy-play.net/?mypy=0.950&python=3.11&flags=show-traceback%2Cstrict-equality&gist=fec0952fc809f63c5d0946f317964f16

@AlexWaygood
Copy link
Member

Here's a snippet illustrating mypy's current semantics for some similar cases involving TypeVars/ParamSpecs and --strict-equality:

from typing import Callable, Generic, ParamSpec, TypeVar

T = TypeVar("T")
T1 = TypeVar("T1")
P = ParamSpec("P")
P1 = ParamSpec("P1")

class Foo(Generic[T]):
    x: T

class Bar(Generic[P]):
    x: Callable[P, str]

def checker(foo1: Foo[int], foo2: Foo[str]) -> bool:
    return foo1 == foo2  # error: Non-overlapping equality check (left operand type: "Foo[int]", right operand type: "Foo[str]")  [comparison-overlap]

def checker1(foo1: Foo[int], foo2: Foo[bool]) -> bool:
    return foo1 == foo2

def checker2(foo1: Foo[T], foo2: Foo[T1]) -> tuple[T, T1] | None:
    if foo1 == foo2:
        return foo1.x, foo2.x
    return None

def checker3(bar1: Bar[P], bar2: Bar[P1]) -> tuple[Callable[P, str], Callable[P1, str]] | None:
    if bar1 == bar2:
        return bar1.x, bar2.x
    return None

JukkaL pushed a commit that referenced this issue Feb 27, 2023
… `ParamSpec` (#14792)

Fixes #14783.

Running mypy on this snippet of code currently causes a crash if you
have the `--strict-equality` option enabled:

```python
from typing import Generic, ParamSpec

P = ParamSpec("P")

class Foo(Generic[P]): ...

def checker(foo1: Foo[[int]], foo2: Foo[[str]]) -> None:
    foo1 == foo2
```

This is because the overlapping-equality logic in `meet.py` currently
does not account for the fact that `left` and `right` might both be
instances of `mypy.types.Parameters`, leading to this assertion being
tripped:


https://github.com/python/mypy/blob/800e8ffdf17de9fc641fefff46389a940f147eef/mypy/meet.py#L519

This PR attempts to add the necessary logic to `meet.py` to handle
instances of `mypy.types.Parameters`.
koogoro pushed a commit that referenced this issue Mar 2, 2023
… `ParamSpec` (#14792)

Fixes #14783.

Running mypy on this snippet of code currently causes a crash if you
have the `--strict-equality` option enabled:

```python
from typing import Generic, ParamSpec

P = ParamSpec("P")

class Foo(Generic[P]): ...

def checker(foo1: Foo[[int]], foo2: Foo[[str]]) -> None:
    foo1 == foo2
```

This is because the overlapping-equality logic in `meet.py` currently
does not account for the fact that `left` and `right` might both be
instances of `mypy.types.Parameters`, leading to this assertion being
tripped:


https://github.com/python/mypy/blob/800e8ffdf17de9fc641fefff46389a940f147eef/mypy/meet.py#L519

This PR attempts to add the necessary logic to `meet.py` to handle
instances of `mypy.types.Parameters`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crash topic-overlap Overlapping equality check topic-paramspec PEP 612, ParamSpec, Concatenate
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants