-
-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simple dependent types #3062
Comments
I think we can have I think regex will be problematic. How do you determine whether they overlap? String literals seems to suffice. (And I don't think it counts as dependent type since it has little to do with runtime). |
The mode string syntax for open() is more complicated than just I find If this doesn't count as a dependent type maybe you can explain your definition of dependent types? And what would you call this? |
I'm not sure going into definitions is helpful, but the difference is (I guess) the reason you call it "Simple" dependent types - there nothing like However, I might be wrong. I am also too easily dragged into issues of terminology. |
Possibilities for when you can't statically determine the overload variant
because your argument isn't literal enough:
* Union the alternatives you've got
* Use the return type of the implementation (if any) (simpler, I think)
* Use `Any` (simplest, I think)
…On Mon, Mar 27, 2017 at 2:37 PM, Guido van Rossum ***@***.***> wrote:
The mode string syntax for open() is more complicated than just 'r', 'w',
etc. Hence the regex.
I find LiteralType[True] pretty verbose; also it's more general than we
want to implement (we really don't want this to be a big project, just
enough to handle the cases @JukkaL <https://github.com/JukkaL> mentioned).
If this doesn't count as a dependent type maybe you can explain your
definition of dependent types? And what would you call this?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#3062 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABjs4Q_pmhJYnbeMNorZzX3OAIZpKeCzks5rqCw1gaJpZM4Mqfj6>
.
|
I suspect special cases will continue to appear and build, and one day we will have to do this big project - which I have the feeling is not big right now, but will be big then and not easily backwards-compatible since special-casing is too easy to over fit for specific applications. How complex is the mode for |
Back when there were a couple interns working on this feature for a week, we had a few different use cases that they wanted to support:
What else? Is |
Maybe also Any finite language is dispatch on a constant. |
As of 3.6, the |
But the main thing we really want is determining the correct return type when the argument is a compile-time literal. You can't overload on True or False because there's no way to spell "the subtype of bool that only contains True" (etc.). If it can't be determined what the value is at compile time it's okay to revert to the more general overload. Similar for open() and everything that takes a similar mode string (it's bred quite a following). |
I would prefer this syntax:
It's flexible, and can be implemented like this:
|
How? Running arbitrary user code is certainly out of question. What do you suggest? To me this sounds like a turing-complete metaprogramming language and full-blown dependent type system. I do like is the idea of avoiding the scoping problem (how to express |
@elazarg Ah yes, I did mean to run arbitrary user code. If the type checker is not allowed to do that for security or other reasons, my suggestion won't work. |
It can't really do that in general, since there's nothing requiring type checkers to run in the same Python version as they're checking for, or even in Python at all. |
Just found this: https://typesandkinds.wordpress.com/#what-are-singletons |
Regarding syntax, perhaps we can allow something like this: @overload
def f(x: False) -> bytes: ...
@overload
def f(x: True) -> str: ... It's not too pretty, but it certainly isn't verbose... Strings are problematic, but we can use raw strings: @overload
def open(file: str, mode: r'w') -> IO[str]: ...
@overload
def open(file: str, mode: r'wb') -> IO[bytes]: ... It is only intended for stub files, so people will guess the meaning even without dedicated name for the type. And it is consistent with (TBH I'm not really sure about this proposal) |
If we only care about overload dispatch, we may pass it as an argument to @overload(mode='w')
def open(file: str, mode: str) -> IO[str]: ...
@overload(mode='wb')
def open(file: str, mode: str) -> IO[bytes]: ... Thus avoiding the question of how to spell the name of the type. |
I don't think we need to perfectly determine overlaps? If someone does a
But wouldn't @JukkaL
it infers |
That's easy for locals, but what about attributes or class vars? Any time you call anything it may change one of those: class C:
def f(self) -> None:
self.foo = True
self.bar()
# Here we don't know if self.foo is still True
def bar(self): ... # A subclass may override |
I thought this whole discussion was limited to literals or simple identifiers of immutable basic types. [Edit: now I understand what you mean; what if an attribute gets this type, we still have to deal with it. Auto-upcasting on a function call or auto-upcasting immediately would seem to solve it, though.] I guess we can make it work for attributes and global variables, by automatically upcasting attributes and global variables on any function calls rather than just on the second assignment. That seems ok, since for example, there's no hope to predict the return type of
|
Sorry, I mixed this discussion up with the discussion about constants to
index a named tuple.
…On Mar 29, 2017 4:50 PM, "pkch" ***@***.***> wrote:
I thought this whole discussion was limited to literals or simple
identifiers of immutable basic types.
I guess we can make it work by automatically upcasting attributes and
global variables on any functions calls rather than on second assignment.
That seems ok, since for example, there's no hope to predict the return
type of pow when it's called like this:
class C:
def f(self) -> None:
self.foo = 5
self.bar()
pow(2, self.foo) # is it int or float?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3062 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ACwrMldlthhv1D8OEK2pAI2xBs_lIx5Vks5rqu5IgaJpZM4Mqfj6>
.
|
TypeScript 2.0 (Oct 2016) added precisely these simple (literal) dependent types. Their stated use cases are enum, dealing with legacy APIs, and as a tag for a Union; all somewhat relevant. Just thought I'd point this out since they seem to be solving a similar problem to what is being discussed here. |
Interesting. We couldn't borrow their syntax because in PEP 484 using a string literal where a type is expected just means a forward reference. Python already has string enums. For union tags PEP 484 uses isinstance() so I think we don't need that either. I still think that the only use case we don't yet support is overloading on value, with the most complex use cases being |
Constant/immutable types as mentioned in #1214 could be treated like literals in the typecheck flow, as long as they were assigned to a literal or another constant. Cases where reassignment occurs seem to be most challenging: # sane.py
MODE = 'rb' # Const[str]
with open(path, MODE) as f:
# since MODE never gets reassigned, f is binary
... # insane.py
MODE = 'wa'
MODE = MODE.replace('a', 'b')
with open(path, MODE) as f:
# human understands this is binary, but no simple typecheck
... Also, the flow of a bounded integer type is complex but potentially doable with well-documented functions and immutable variables. Mutability makes these bounds harder to reason about: # wild.py
from random import randint
# hypothetically assuming randint has input/output bounding rules
my_int = randint(0, 10) # Bound[int, 0, 10]
# potentially doable but likely more trouble than it's worth
int_two = randint(my_int, 2 * my_int) # Bound[int, 0, 20]
# rough
int_two /= 10 # Bound[float, 0.0, 2.0] |
@brainrape Is it possible to use this compile time type checking from Python? What could this look like? |
@saulshanabrook Here's an example that uses numpy from Idris: |
Hello, What's the final answer to this ? Is this still under discussion ? |
We have plans to work on this, but no concrete timeline yet. |
@JukkaL Is there some workaround I can use for now ? you mentioned a plugin. How can i use it in my code ? |
@mehdigmira The plugin API is not documented, so you are on your own if you want to go there. There are some examples at https://github.com/python/mypy/tree/master/mypy/plugins. |
@JukkaL Thank you for the response. |
@mehdigmira Yes. The plugin approach is really only practical for library functions with relatively stable interfaces, in my opinion. Otherwise it will be too difficult to maintain. |
Here's my current thinking on what's needed to support this in a minimally useful fashion. Our current idea for the syntax is like this:
Literal types are not inferred for variables unless the lvalue is final: y: Literal['foo'] = 'foo' # Without the annotation the type would be 'str'
x: Final = 'foo' # Type is Literal['foo']
z = 'foo' # Type of 'z' is 'str'
zz: Final = z # The type of 'zz' is 'str' here as well Note that the a: Final = ['a'] # Inferred type is still List[str] We'd likely promote a string literal to Literal types are compatible with the corresponding non-literal types, but not the other way around: x = 'foo'
y: Literal['foo'] = x # Error
z: str = y # Ok For overloads we may need a fallback in addition to literal types to avoid false positives with legacy interfaces: @overload
def f(x: Literal[True]) -> str: ...
@overload
def f(x: Literal[False]) -> bytes: ...
@overload
def f(x: bool) -> Any: ...
y: bool = ...
a = f(y) # Any inferred
b = f(True) # str inferred cc @ilevkivskyi |
I really don't like the enforced |
If we don't want to execute arbitrary Python code at the type level, how about specifying a value-to-type mapping with a dict? Something like this: def f(x: bool) -> FResult[x]: ...
# ^ the argument `x`
FResult = TypeMapping[bool]({
True: str,
False: bytes,
}) It's less verbose than Here's a bigger example where the type depends on two arguments: def open_file(path: str, access: str = 'r', encoding: str = 'u') -> SomeFile[access, encoding]: ...
SomeFile = TypeMapping[str, str]({
('r', 'u'): ReadableFile[str], # a read-only handle
('r', 'b'): ReadableFile[bytes],
('w', 'u'): WritableFile[str], # a write-only handle
('w', 'b'): WritableFile[bytes],
}) There's a bunch of ways [1] Using a function argument in a type annotation would constrain this extension to python 3.7+ where type annotations aren't evaluated. This could be remedied with something like [2] For an example, see the |
@srittau Something like the @lubieowoce Your proposal introduces quite a bit of new syntax for functionality that |
@JukkaL -- @Michael0x2a and me are already working on a draft-PEP about @lubieowoce -- Overloads (including type guards) is an important use case which will be explicitly mentioned in the draft PEP. @srittau -- I don't think literal types will be used so often that saving 6 keystrokes is important. Second, I think explicit is better than implicit here, some people may be confused by bare literals in annotations. Finally, as Jukka mentioned, it just doesn't work. Anyway, I would propose to pause the discussion here until the actual draft is posted here (hopefully very soon). |
@ilevkivskyi Would something like |
No. |
@ilevkivskyi @JukkaL thank you for your comments. I'm aware (and not too happy) that my proposal duplicates some functionality – I didn't mention it to keep the post short. I'll hold off with responding until you publish the draft PEP – perhaps it'll clarify some things that concerned me about the |
Small update -- here's the draft PEP mentioned above: https://github.com/Michael0x2a/peps/blob/literal-types/pep-9999.rst. I think the new plan is to try discussing it on our shiny new typing-sigs mailing list instead of here, though? We'll see how it goes. |
@Michael0x2a seeing as PEP 586 is accepted (congrats!) perhaps this can be closed, in favor of #5935? |
Yes, mypy now supports literal types |
[This is based on a discussion between me and @gvanrossum last week. I'm writing this down so that we won't forget about this.]
Several stdlib functions have signatures where the return type depends on the value of an argument. Examples:
open
returns anIO[str]
orIO[bytes]
depending on whether the mode argument has'b'
as a substring.subprocess.check_output
returns astr
orbytes
depending on the value of the booleanuniversal_newlines
argument.A simple way to support these would be to introduce a few additional types that work with
str
andbool
literals:False
andTrue
.For example, if we had
FalseType
andTrueType
, we could write a function whose return value depends on the value of a boolean argument:Not sure about this one:
We could also have a generic class where a generic type argument depends on the value of an argument to
__init__
. Not sure how we should represent these -- here is one potential approach:Alternatively, we could use
__new__
, but this would be problematic if there is no corresponding method defined at runtime.The text was updated successfully, but these errors were encountered: