Overloaded def including TypeVarTuple and variable length Tuple crash #16427

robjhornby · 2023-11-09T02:31:36Z

Crash Report

Crash when running mypy from latest master (a1864d4) on the code copied below (note - it does not crash on latest release 1.6.1)

Traceback

Traceback (most recent call last):
  File "/Users/rob/projects/typed-parsy/.venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/main.py", line 100, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/main.py", line 182, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/build.py", line 191, in build
    result = _build(
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/build.py", line 265, in _build
    graph = dispatch(sources, manager, stdout)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/build.py", line 2943, in dispatch
    process_graph(graph, manager)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/build.py", line 3341, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/build.py", line 3442, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/build.py", line 2311, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/checker.py", line 481, in check_first_pass
    self.accept(d)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/checker.py", line 591, in accept
    stmt.accept(self)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/nodes.py", line 584, in accept
    return visitor.visit_overloaded_func_def(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/checker.py", line 628, in visit_overloaded_func_def
    self._visit_overloaded_func_def(defn)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/checker.py", line 658, in _visit_overloaded_func_def
    self.check_overlapping_overloads(defn)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/checker.py", line 768, in check_overlapping_overloads
    if is_unsafe_overlapping_overload_signatures(sig1, sig2, type_vars):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/checker.py", line 7712, in is_unsafe_overlapping_overload_signatures
    ) or is_callable_compatible(
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/subtypes.py", line 1506, in is_callable_compatible
    unified = unify_generic_callable(
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/subtypes.py", line 1827, in unify_generic_callable
    cs = mypy.constraints.infer_constraints(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 314, in infer_constraints
    return _infer_constraints(template, actual, direction, skip_neg_op)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 407, in _infer_constraints
    return template.accept(ConstraintBuilderVisitor(actual, direction, skip_neg_op))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/types.py", line 1964, in accept
    return visitor.visit_callable_type(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 1071, in visit_callable_type
    infer_callable_arguments_constraints(template, cactual, self.direction)
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 1579, in infer_callable_arguments_constraints
    res.extend(infer_directed_arg_constraints(left_arg.typ, right_arg.typ, direction))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 1539, in infer_directed_arg_constraints
    return infer_constraints(left, right, neg_op(direction))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 311, in infer_constraints
    res = _infer_constraints(template, actual, direction, skip_neg_op)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 407, in _infer_constraints
    return template.accept(ConstraintBuilderVisitor(actual, direction, skip_neg_op))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/types.py", line 1429, in accept
    return visitor.visit_instance(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/rob/projects/typed-parsy/.venv/lib/python3.11/site-packages/mypy/constraints.py", line 958, in visit_instance
    and unpacked.type.fullname == "builtins.tuple"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError:

To Reproduce

mypy playground: https://mypy-play.net/?mypy=master&python=3.11&gist=d8a94004f71a87a6a8621da0aff744c6

This code causes the crash:

from typing import Any, Tuple, TypeVar, TypeVarTuple, Unpack, overload

T = TypeVar("T")
Ts = TypeVarTuple("Ts")


@overload
def add(self: Tuple[Unpack[Ts]], other: Tuple[T]) -> Tuple[Unpack[Ts], T]:
    ...
@overload
def add(self: Tuple[T, ...], other: Tuple[T, ...]) -> Tuple[T, ...]:
    ...
def add(self: Any, other: Any) -> Any:
    ...

Your Environment

Mypy version used: master (a1864d4)
Mypy command-line flags:
Mypy configuration options from mypy.ini (and other config files):
Python version used: 3.11.4
Operating system and version: macOS Ventura 13.5

The text was updated successfully, but these errors were encountered:

Fixes #16427 The test case added in the first commit crashes. The second commit addresses the crash - I don't know whether this fix is correct, it just happens to stop the crash but it leads to a code branch which just `continue`s out of a for loop iteration, so it might be bypassing something it shouldn't. I don't completely understand it. --------- Co-authored-by: Ivan Levkivskyi <[email protected]>

hauntsaninja · 2023-11-10T01:17:39Z

Thanks for the fix! Did you encounter this on open source code?

Fixes #16427 The test case added in the first commit crashes. The second commit addresses the crash - I don't know whether this fix is correct, it just happens to stop the crash but it leads to a code branch which just `continue`s out of a for loop iteration, so it might be bypassing something it shouldn't. I don't completely understand it. --------- Co-authored-by: Ivan Levkivskyi <[email protected]>

robjhornby · 2023-11-11T03:14:03Z

Yep, I encountered it in an experiment which isn't currently used by anyone, code here:

https://github.com/robjhornby/typed-parsy/blob/28d2d6be401a3f7eaa174467e4eada15d13482fe/parsy/__init__.py#L399-L460

I wanted annotations for an __add__ method of a Parser class to act as if the Parser wrapper wasn't there. Example: Parser[Tuple[str]] added to a Parser[Tuple[int, bool]] should lead to a type of Parser[Tuple[str, int, bool]], same as if I was directly adding tuples of type Tuple[str] and Tuple[int, bool]. It took me a lot of overloads to get something which still isn't completely general...

robjhornby added the crash label Nov 9, 2023

robjhornby mentioned this issue Nov 9, 2023

Handle TypeVarTupleType when checking overload constraints #16428

Merged

ilevkivskyi added the topic-pep-646 PEP 646 (TypeVarTuple, Unpack) label Nov 10, 2023

ilevkivskyi mentioned this issue Nov 10, 2023

Release 1.7 planning #16341

Closed

2 tasks

ilevkivskyi closed this as completed in #16428 Nov 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Overloaded def including TypeVarTuple and variable length Tuple crash #16427

Overloaded def including TypeVarTuple and variable length Tuple crash #16427

robjhornby commented Nov 9, 2023 •

edited

Loading

hauntsaninja commented Nov 10, 2023

robjhornby commented Nov 11, 2023

Overloaded def including TypeVarTuple and variable length Tuple crash #16427

Overloaded def including TypeVarTuple and variable length Tuple crash #16427

Comments

robjhornby commented Nov 9, 2023 • edited Loading

hauntsaninja commented Nov 10, 2023

robjhornby commented Nov 11, 2023

robjhornby commented Nov 9, 2023 •

edited

Loading