Switch to explicit `typing.Optional` throughout #2096

michaeloliverx · 2022-02-26T12:16:36Z

Fixes #2092

from contextlib import AsyncExitStack
import httpx


async def main() -> None:
    async with AsyncExitStack() as stack:
        async_http_client = await stack.enter_async_context(httpx.AsyncClient())
        reveal_type(async_http_client)

        sync_http_client = stack.enter_context(httpx.Client())
        reveal_type(sync_http_client)

❯ mypy test-script.py --strict
test-script.py:8: note: Revealed type is "httpx._client.AsyncClient*"
test-script.py:11: note: Revealed type is "httpx._client.Client*"

florimondmanca · 2022-02-28T10:51:34Z

Thanks @michaeloliverx

This seems right to me.

But it also looks like this would belong to a wider "do we want to comply with --strict mypy" discussion, right? Right now we have no way to prevent a regression on this on CI, because we only run mypy (no strict mode).

Edit: see #1747

florimondmanca · 2022-02-28T10:57:44Z

Happy to read @tomchristie's viewpoint on this -- should we start a discussion on mypy --strict, go ahead with this change, something else?

tomchristie · 2022-02-28T11:09:11Z

Thanks @michaeloliverx, @florimondmanca.

Happy to read @tomchristie's viewpoint on this -- should we start a discussion on mypy --strict, go ahead with this change, something else?

Interesting, right? It's a little bit niggly that typing in Python ends up having this effect. We've got correct and consistent typing throughout, that helps us have confidence in changes within our codebase. But if our users choose to run mypy --strict, then we end up having to conform to that too.

I'd suggest that we value consistency above one-off fixes. So... we ought to consider a PR that switches to explicit typing.Optional everywhere, rather in just one place.

Right now we have no way to prevent a regression on this on CI, because we only run mypy (no strict mode).

This is a good point. A smart first pass at this would be a pull request that just turns on strict mode for our type checking. That'll fail, but it'd give us the complete set of changes that we'd need to make if we wanted to change this.

michaeloliverx · 2022-02-28T11:34:28Z

Currently --strict enables the following flags:

  --warn-unused-configs
  --disallow-any-generics
  --disallow-subclassing-any
  --disallow-untyped-calls
  --disallow-untyped-defs
  --disallow-incomplete-defs
  --check-untyped-defs
  --disallow-untyped-decorators
  --no-implicit-optional
  --warn-redundant-casts
  --warn-unused-ignores
  --warn-return-any
  --no-implicit-reexport
  --strict-equality

Enabling this may generate a lot of noise unrelated to using explicit typing.Optional expressions.

This is a good point. A smart first pass at this would be a pull request that just turns on strict mode for our type checking. That'll fail, but it'd give us the complete set of changes that we'd need to make if we wanted to change this.

We could enable --no-implicit-optional rather than --strict which would result in less changes and still achieve the desired effect of using explicit typing.Optional.

michaeloliverx · 2022-02-28T22:27:32Z

Okay I have now enabled no_implicit_optional in the mypy config file and fixed all the issues in httpx. I am still getting some errors:

httpx/_transports/default.py:173: error: Argument "proxy_auth" to "SOCKSProxy" has incompatible type "Optional[Tuple[bytes, bytes]]"; expected "Tuple[Union[bytes, str], Union[bytes, str]]"
httpx/_transports/default.py:176: error: Argument "max_keepalive_connections" to "SOCKSProxy" has incompatible type "Optional[int]"; expected "int"
httpx/_transports/default.py:177: error: Argument "keepalive_expiry" to "SOCKSProxy" has incompatible type "Optional[float]"; expected "float"
httpx/_transports/default.py:267: error: Argument "max_keepalive_connections" to "AsyncConnectionPool" has incompatible type "Optional[int]"; expected "int"
httpx/_transports/default.py:268: error: Argument "keepalive_expiry" to "AsyncConnectionPool" has incompatible type "Optional[float]"; expected "float"
httpx/_transports/default.py:271: error: Argument "uds" to "AsyncConnectionPool" has incompatible type "Optional[str]"; expected "str"
httpx/_transports/default.py:272: error: Argument "local_address" to "AsyncConnectionPool" has incompatible type "Optional[str]"; expected "str"
httpx/_transports/default.py:283: error: Argument "proxy_auth" to "AsyncHTTPProxy" has incompatible type "Optional[Tuple[bytes, bytes]]"; expected "Tuple[Union[bytes, str], Union[bytes, str]]"
httpx/_transports/default.py:287: error: Argument "max_keepalive_connections" to "AsyncHTTPProxy" has incompatible type "Optional[int]"; expected "int"
httpx/_transports/default.py:288: error: Argument "keepalive_expiry" to "AsyncHTTPProxy" has incompatible type "Optional[float]"; expected "float"
httpx/_transports/default.py:308: error: Argument "proxy_auth" to "AsyncSOCKSProxy" has incompatible type "Optional[Tuple[bytes, bytes]]"; expected "Tuple[Union[bytes, str], Union[bytes, str]]"
httpx/_transports/default.py:311: error: Argument "max_keepalive_connections" to "AsyncSOCKSProxy" has incompatible type "Optional[int]"; expected "int"
httpx/_transports/default.py:312: error: Argument "keepalive_expiry" to "AsyncSOCKSProxy" has incompatible type "Optional[float]"; expected "float"
httpx/_transports/default.py:332: error: Argument 1 to "__aexit__" of "AsyncConnectionPool" has incompatible type "Optional[Type[BaseException]]"; expected "Type[BaseException]"
httpx/_transports/default.py:332: error: Argument 2 to "__aexit__" of "AsyncConnectionPool" has incompatible type "Optional[BaseException]"; expected "BaseException"
httpx/_transports/default.py:332: error: Argument 3 to "__aexit__" of "AsyncConnectionPool" has incompatible type "Optional[TracebackType]"; expected "TracebackType"
Found 16 errors in 1 file (checked 60 source files)

I think these issues are actually with httpcore and the types passed to them are not compatible now.

adriangb · 2022-03-01T01:04:49Z

Those some like mostly internal issues, right? Can we just #type: ignore[arg-type] them so that we can move forward with the typing changes in the public API?

michaeloliverx · 2022-03-01T08:50:38Z

Those some like mostly internal issues, right? Can we just #type: ignore[arg-type] them so that we can move forward with the typing changes in the public API?

Yes we could do that for now.

michaeloliverx · 2022-03-01T08:59:48Z

Okay this is ready now, I have ignored internal type errors for now.

michaeloliverx · 2022-03-01T10:00:07Z

I opened a similar PR at encode/httpcore#513. If that is approved then we could remove the type ignore comments.

ref: encode/httpx#2096

michaeloliverx · 2022-04-01T08:52:37Z

Now that encode/httpcore#513 has been merged I think we should wait until httpcore cuts a release before merging this as then we can remove all of the type: ingore comments.

vfazio · 2022-05-17T19:03:33Z

Now that encode/httpcore#513 has been merged I think we should wait until httpcore cuts a release before merging this as then we can remove all of the type: ingore comments.

@michaeloliverx
0.15.0 was released a few hours ago.

vfazio · 2022-05-19T17:14:24Z

@michaeloliverx

running your branch and updating httpcore:

(venv) vfazio@vfazio2 /tmp/tmp.OcS7trgNiz $ git diff
diff --git a/setup.py b/setup.py
index 0ca769f..f6edfdd 100644
--- a/setup.py
+++ b/setup.py
@@ -60,7 +60,7 @@ setup(
         "charset_normalizer",
         "sniffio",
         "rfc3986[idna2008]>=1.3,<2",
-        "httpcore>=0.14.5,<0.15.0",
+        "httpcore>=0.15.0,<0.16.0",
     ],
     extras_require={
         "http2": "h2>=3,<5",

(venv) vfazio@vfazio2 /tmp/tmp.OcS7trgNiz $ git log -n1
commit 1dc160b8a325219047425ac4914e1d10106dbee9 (HEAD -> michaeloliverx/issue2092, origin/michaeloliverx/issue2092)
Merge: 7caf37b 9673a35
Author: Michael Oliver <[email protected]>
Date:   Wed May 18 09:49:45 2022 +0100

    Merge branch 'master' into michaeloliverx/issue2092

(venv) vfazio@vfazio2 /tmp/tmp.OcS7trgNiz :( $ mypy httpx tests
Success: no issues found in 60 source files

michaeloliverx · 2022-05-19T22:06:08Z

@vfazio thanks!

adriangb · 2022-05-20T04:46:12Z

This looks ready to me!

tomchristie · 2022-05-20T10:12:21Z

Sure thing, let's do it.

Fix issue with Mypy --strict and AsyncExitStack

98e754a

michaeloliverx added 2 commits February 28, 2022 22:20

Enable no_implicit_optional in Mypy

fbb09ca

Merge branch 'master' into michaeloliverx/issue2092

d3b1e30

michaeloliverx changed the title ~~Fix issue with Mypy --strict and AsyncExitStack~~ Switch to explicit typing.Optional throughout Feb 28, 2022

Ignore internal type errors

c22d923

mnixry added a commit to mnixry/nonebot-plugin-gocqhttp that referenced this pull request Mar 4, 2022

🏷️ Fix typing issue caused by hints of httpx

001c1c4

ref: encode/httpx#2096

michaeloliverx and others added 2 commits March 4, 2022 14:41

Merge branch 'master' into michaeloliverx/issue2092

eb6e56e

Merge branch 'master' into michaeloliverx/issue2092

25af71d

tomchristie mentioned this pull request Mar 7, 2022

Switch to explicit typing.Optional throughout #2092

Closed

Merge branch 'master' into michaeloliverx/issue2092

f35e95b

michaeloliverx added 2 commits April 11, 2022 09:53

Merge branch 'master' into michaeloliverx/issue2092

1b26a96

Merge branch 'master' into michaeloliverx/issue2092

f00f5d9

RA80533 mentioned this pull request Apr 24, 2022

Create a new release encode/httpcore#542

Closed

michaeloliverx added 2 commits May 6, 2022 10:52

Merge branch 'master' into michaeloliverx/issue2092

76efee9

Merge branch 'master' into michaeloliverx/issue2092

7caf37b

Merge branch 'master' into michaeloliverx/issue2092

1dc160b

michaeloliverx added 2 commits May 19, 2022 22:59

Bump httpcore

30665b1

Remove unneeded type: ignore comments

597dcfd

adriangb approved these changes May 20, 2022

View reviewed changes

tomchristie merged commit 14a1704 into encode:master May 20, 2022

michaeloliverx deleted the michaeloliverx/issue2092 branch May 20, 2022 10:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to explicit `typing.Optional` throughout #2096

Switch to explicit `typing.Optional` throughout #2096

michaeloliverx commented Feb 26, 2022

florimondmanca commented Feb 28, 2022 •

edited

Loading

florimondmanca commented Feb 28, 2022

tomchristie commented Feb 28, 2022

michaeloliverx commented Feb 28, 2022 •

edited

Loading

michaeloliverx commented Feb 28, 2022

adriangb commented Mar 1, 2022

michaeloliverx commented Mar 1, 2022

michaeloliverx commented Mar 1, 2022

michaeloliverx commented Mar 1, 2022

michaeloliverx commented Apr 1, 2022

vfazio commented May 17, 2022 •

edited

Loading

vfazio commented May 19, 2022

michaeloliverx commented May 19, 2022

adriangb commented May 20, 2022

tomchristie commented May 20, 2022

Switch to explicit typing.Optional throughout #2096

Switch to explicit typing.Optional throughout #2096

Conversation

michaeloliverx commented Feb 26, 2022

florimondmanca commented Feb 28, 2022 • edited Loading

florimondmanca commented Feb 28, 2022

tomchristie commented Feb 28, 2022

michaeloliverx commented Feb 28, 2022 • edited Loading

michaeloliverx commented Feb 28, 2022

adriangb commented Mar 1, 2022

michaeloliverx commented Mar 1, 2022

michaeloliverx commented Mar 1, 2022

michaeloliverx commented Mar 1, 2022

michaeloliverx commented Apr 1, 2022

vfazio commented May 17, 2022 • edited Loading

vfazio commented May 19, 2022

michaeloliverx commented May 19, 2022

adriangb commented May 20, 2022

tomchristie commented May 20, 2022

Switch to explicit `typing.Optional` throughout #2096

Switch to explicit `typing.Optional` throughout #2096

florimondmanca commented Feb 28, 2022 •

edited

Loading

michaeloliverx commented Feb 28, 2022 •

edited

Loading

vfazio commented May 17, 2022 •

edited

Loading