Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate TypeAlias, TypeVar, ParamSpec, TypeVarTuple, Generic if you are using >= 3.12 #244

Open
KotlinIsland opened this issue Apr 2, 2024 · 19 comments
Labels
type checking / linting issues relating to existing diagnostic rules or proposals for new diagnostic rules

Comments

@KotlinIsland
Copy link
Collaborator

No description provided.

@KotlinIsland KotlinIsland added the type checking / linting issues relating to existing diagnostic rules or proposals for new diagnostic rules label Apr 2, 2024
@DetachHead
Copy link
Owner

what if you want to explicitly specify the variance, you can't do that with the new generic syntax right?

@jorenham
Copy link
Contributor

jorenham commented Apr 3, 2024

@DetachHead I once made an attempt at a PEP that uses +T and -T for the explicit specification of co-/contra- variance. But it got rejected because it was "superseded" by PEP 695, under the false assumption that type variance can always be magically inferred 🤷🏻‍♂️.

... anyway, what about allowing TypeVar on py312+ ff. 1) it has a co(ntra)variant=True, and 2) it would have been invariant when infer_variance=True?

@KotlinIsland
Copy link
Collaborator Author

KotlinIsland commented Apr 3, 2024

@jorenham We think that In/Out would be better

Also see this series of discussions where I tried to argue the same thing:

It also seemed that Traut missed a lot of relevant info about other languages variance:

@jorenham
Copy link
Contributor

jorenham commented Apr 4, 2024

But then how would you be able to explicitly mark a type type parameter as being invariant, in cases where the type checker infers it to be co- or contravariant? This could become especially confusing with In[].

With prefix operators, the obvious solution is ~T. I believe mypy uses this this already in output messages.

I also think that +T and -T are more readable than Out[T] and In[T], e.g. consider

class Spam[-V, +R: Sequence[tuple[str, int]]]: ...

vs

class Spam[In[V], Out[R: Sequence[tuple[str, int]]]]: ...

In case of upper bounds, it isn't immediately obvious how to define it; should it be Out[T: Upper] or Out[T]: Upper? But with prefix ops, there is only one possible way: +T: Upper.

There's also the naming issue of In, namely (pun intended) that it's very close to in, even though it's unrelated.
There are also many fonts where a I is difficult to distinguish from a a lowercase l. So it could be detrimental to readability when In is "mentally parsed" as ln, which isn't a big mental jump for developers.

Just like I've seen TypeVar('T_co', covariant=True) used in many places where it shouldn't, *i.e. not as a type parameter, but as a type argument), I can see that this can become an issue again with In and Out. Whereas with the prefix ops, misuses will simply raise a TypeError out-of-the-box (TypeVar doesn't implement __pos__ or __neg__).

@KotlinIsland
Copy link
Collaborator Author

KotlinIsland commented Apr 4, 2024

But then how would you be able to explicitly mark a type type parameter as being invariant

That would be the default:

class A[T]: ...  # invariant

Or, if you were interested in preserving pep compliant behavior, an InOut form could be introduced, similar to typescript:

class A[InOut[T]]: ...

I also think that +T and -T are more readable than Out[T] and In[T], e.g. consider

Ideally, these would be soft keywords, which would relieve your concerns:

class A[in T: str]: ...

Unfortunately this would be a syntax change.

I prefer in/out to +/-, it is more natural and understand able (out parameters can only go out of methods, not into them).

There's also the naming issue of In,

image

Again, ideally it would be be in and just reuse the existing keyword in. My font is the default and I don't have any issues. sounds like a font issue to me, idk.

Additionally, Kotlin, TypeScript, Dart and others all use in/out to great success.

Just like I've seen TypeVar('T_co', covariant=True) used in many places where it shouldn't, *i.e. not as a type parameter, but as a type argument), I can see that this can become an issue again with In and Out. Whereas with the prefix ops, misuses will simply raise a TypeError out-of-the-box (TypeVar doesn't implement __pos__ or __neg__).

Under my implementation, any invalid usage would raise a type error, but what cases are you referring to? use site variance is valid:

a: list[int] = []
b: list[out object] = a  # no error

@jorenham
Copy link
Contributor

jorenham commented Apr 6, 2024

That would be the default

That would break backwards compatibility, since the (PEP 695) default is to automatically infer variance.

Or, if you were interested in preserving pep compliant behavior, an InOut form could be introduced, similar to typescript

It was a challenge to find something about it online, but somewhere on the third Google page, I managed to find a brief mention of in out in typescript 🧐 .
But to me, InOut seems somewhat arbitrary to me, it could just as well have been named OutIn 🤷🏻 . From the name alone, it would make more sense if it was used to imply bivariance (no idea why someone would want that, though).

Ideally, these would be soft keywords, which would relieve your concerns

Yes, that would make it a lot better. I'm not sure whether the PEG parser will like it though, since the in keyword already has two (bivariate) uses (i.e.e.g. a in {} and for b in []), and IDE- and other syntax-highlighting software devs will probably hate you for it 😅 (e.g.i.e. pylance still can't properly highlight py312+ code with PEP 695 generics).

I prefer in/out to +/-, it is more natural and understand able (out parameters can only go out of methods, not into them).

I can see how it can be considered to be "more pythonic". But in the same way, + means "to produce" and - to consume, which for me is at least as intuitive.
For long type declarations that require linebreaks to fit on a 4k display (which is pretty much the default when working with async stuff), out and in look a lot more awkward than + and -:

class SupportsHashableGetitemThatReturnsAStringSubtype[
    in KeyType: Hashable,
    out ValueType: str,
](Protocol):
    def __getitem__(self, key: KeyType, /) -> ValueType: ...

For some reason, off-by-once formatting/indentation things like these can keep me up at night, but that's probably just me 🤷🏻‍♂️.

Under my implementation, any invalid usage would raise a type error, but what cases are you referring to? use site variance is valid

In your example you attempt to make object itself covariant, w.r.t. a list instance? Even if I leave aside the fact that the type parameter T@list is invariant (unlike T@Sequence, which is covariant), I still don't understand what it is that you're trying to show here 🤷🏻.

Variance is a property of a type parameter of a specific type. So it should be used when declaring types or type-aliases.
It's not unlike functions with e.g. positional-only parameters: def f(a, /, b): ... is correct and f(a, /, b) isn't.

So theoretically speaking, T = TypeVar('T', covariant=True) is plain wrong, because it must be used as both type parameter and type argument. This is what motivated me to write that PEP in the first place.

@KotlinIsland
Copy link
Collaborator Author

KotlinIsland commented Apr 6, 2024

That would break backwards compatibility, since the (PEP 695) default is to automatically infer variance.

Don't care about breaking backwards comparability, but for the 484 inclined, there would be the option for InOut.

Basedmypy already breaks a lot of backwards compatibility, and I plan to break it a hell of a lot more.

But to me, InOut seems somewhat arbitrary to me, it could just as well have been named OutIn 🤷🏻 .

InOut makes the slightest more sense to me, in that inputs come before outputs. That's what I would go with in lieu of a better alternative.

It was a challenge to find something about it online

@DetachHead has an eslint plugin that enforces the correct variant annotations

From the name alone, it would make more sense if it was used to imply bivariance

I disagree, InOut sounds like a variable that can go in and or out, which fits the existing behavior of invariant.

no idea why someone would want that, though

Yeah, it's currently an exception when you try to do it with TypeVar. I see no benefit to bivariance what so ever.

I can see how it can be considered to be "more pythonic". But in the same way, + means "to produce" and - to consume, which for me is at least as intuitive.

I disagree. No matter how I look at +/-, they just seem like arbitrary symbols, instead of a 'skeuomorphic' English reference. And the trend of most languages adopting in/out speaks for itself.

For some reason, off-by-once formatting/indentation things like these can keep me up at night, but that's probably just me 🤷🏻‍♂️.

Good point, but the same could be said about an invariant vs Xvariant one:

class A[
    X,
    +Y,
]: ...

Maybe formatters could special case this scenario and produce:

class A[
    in  T1,
    out T2,
]: ...

In your example you attempt to make object itself covariant, w.r.t. a list instance?

The concept of use-site variance (which no python type checkers support) is very useful to modify an invariant type parameter to become co/contra variant:

a: list[int] = []
b: list[out object] = a  # no error

In this example, b has the type of list[out object] meaning that the type of list here has been transformed such that the definition would look like class list[out T]:.

class A[T]:
    def set(self, t: T): ...
    def get(self) -> T: ...
a: A[out object]
a.set(1)  # error: expected "Never", found "int"

The same could be applied to a usage with a type parameter passed as a type argument:

def f[T, R: T](r: R, t: T) -> R:
    a: list[R] = [r]
    b: list[out T] = a  # no error
    b[0] = t  # error: expected "Never", found "T@f"
    return b[0]

So theoretically speaking, T = TypeVar('T', covariant=True) is plain wrong, because it must be used as both type parameter and type argument. This is what motivated me to write that PEP in the first place.

Yes, I do agree with this notion. you are correct in that the TypeVar itself doesn't have variance as a type argument, only when the variance modifier is applied (X[out T]).

@jorenham
Copy link
Contributor

jorenham commented Apr 6, 2024

Don't care about breaking backwards comparability
Breaking backwards backwards compatibility in basedpyright only is fine of course, but I was under the impression that you were trying to push this as a core-python feature, especially after you mentioned the in and out keywords.

I disagree, InOut sounds like a variable that can go in and or out, which fits the existing behavior of invariant.

Biviarance is effectively being both co- and contravariant, so if InOut can go both in and out, it could just as well be used do describe exactly that.

Good point, but the same could be said about an invariant vs Xvariant one:

The alignment with prefix ops can be made pretty again if you're explicit about your invariance:

class Generator[
    -V,
    ~T,
    +R,
]: ...

The concept of use-site variance (which no python type checkers support) is very useful to modify an invariant type parameter to become co/contra variant

I didn't know about site-variance; it's pretty cool! Any plans on submitting a PEP for this?

@KotlinIsland
Copy link
Collaborator Author

KotlinIsland commented Apr 6, 2024

Breaking backwards backwards compatibility in basedpyright

@DetachHead has different feelings about backwards-compatibility that I do. So he would probably have different ideas here for basedpyright.

but I was under the impression that you were trying to push this as a core-python feature, especially after you mentioned the in and out keywords.

Yeah, I was pushing hard before 695 was finalized, but no-one listened 😢. Although, I still think that 695 should be updated to default to invariant. The backwards breakage would be worth it in my opinion.

Biviarance is effectively being both co- and contravariant, so if InOut can go both in and out, it could just as well be used do describe exactly that.

True, but we would just say that it falls as invariant and be done with it.

Any plans on submitting a PEP for this?

I tried working on the Intersection pep after I implemented intersections in basedmypy, but it just sounds like a bunch of dealing with Eric Traut, which I absolutely don't want to do.

@jorenham
Copy link
Contributor

jorenham commented Apr 6, 2024

I tried working on the Intersection pep after I implemented intersections in basedmypy, but it just sounds like a bunch of dealing with Eric Traut, which I absolutely don't want to do.

... so can I expect a basedpython fork / pre-compiler then?

@KotlinIsland
Copy link
Collaborator Author

... so can I expect a basedpython fork / pre-compiler then?

Yes.

But because type annotations aren't generally evaluated at runtime, we can get away with a lot of stuff like intersections/type-guards in basedmypy:

from __future__ import annotations
a: int & str  # no runtime error

def guard(x: object) -> x is int: ...  # no runtime error

Additionally, there aren't too many use cases for evaluating runtime annotations, so it can be handled case by case:

@jorenham
Copy link
Contributor

jorenham commented Apr 8, 2024

Additionally, there aren't too many use cases for evaluating runtime annotations

Well, there's dataclasses.dataclass, typing.NamedTuple, typing.TypedDict, typing.get_*(), pydantic, typer, beartype, typeguard, typical, pytypes, ...

@DetachHead
Copy link
Owner

when deprecating TypeAlias we should make sure you still get an error on type aliases that aren't explicitly annotated with TypeAlias:

Foo = int

@jorenham
Copy link
Contributor

jorenham commented Aug 20, 2024

when deprecating TypeAlias we should make sure you still get an error on type aliases that aren't explicitly annotated with TypeAlias:

Foo = int

If you use Foo at runtime as an alias for the builtins.int constructor, then it shouldn't be a type alias.

So an alias for a type != a type alias...

@DetachHead
Copy link
Owner

yeah but in my experience most of the time it's used as a type alias. perhaps the option to ban them can be a separate setting

@jorenham
Copy link
Contributor

Perhaps only do it for type-only stuff, like Foo = int | str.

Or maybe it's possible to detect wether Foo is used at a constructor at runtime?

@UltimateLobster
Copy link

I'd like to also add that you must use TypeVar if you want to use the new PEP-696 added in Python 3.13.

@jorenham
Copy link
Contributor

jorenham commented Sep 12, 2024

@UltimateLobster

There's no need for TypeVar (unless you prefer explicit variance) with PEP 696:

class Spam[T = object]: ...

@UltimateLobster
Copy link

@UltimateLobster

There's mo need for TypeVar (unless you prefer explicit variance) with PEP 696:

class Spam[T = object]: ...

Oops sorry, I meant you have to use it even when working with Python 3.12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type checking / linting issues relating to existing diagnostic rules or proposals for new diagnostic rules
Projects
None yet
Development

No branches or pull requests

4 participants