Python 3.10 compatibility — deprecated loop argument for asnycio.sleep/gather calls

[sbraz]

Describe the bug

Hi,
Tests currently fail with Python 3.10 beta 4 because the loop attribute was removed.

To Reproduce

Run tests with Python 3.10.

Expected behavior

Tests pass.

Logs/tracebacks

to_cancel = {<Task pending name='Task-1' coro=<_run_app() running at /var/tmp/portage/dev-python/aiohttp-3.7.4-r2/work/aiohttp-3.7.4-python3_10/lib/aiohttp/web.py:429> wait_for=<Future cancelled>>}
loop = <_UnixSelectorEventLoop running=False closed=False debug=False>                                                                                                                                             
                                                                                                                                                                                                                   
    def _cancel_tasks(                                                                                   
        to_cancel: Set["asyncio.Task[Any]"], loop: asyncio.AbstractEventLoop                             
    ) -> None:                                                                                                                                                                                                     
        if not to_cancel:                                                                                
            return                                                                                                                                                                                                 
                                                                                                         
        for task in to_cancel:                                                                           
            task.cancel()                                                                                
                                                                                                         
        loop.run_until_complete(                                                                         
>           asyncio.gather(*to_cancel, loop=loop, return_exceptions=True)                                
        )                                                                                                                                                                                                          
E       TypeError: gather() got an unexpected keyword argument 'loop'                                    

self = <test_locks.TestEventResultOrError object at 0x7f49d37cfd00>, loop = <_UnixSelectorEventLoop running=False closed=False debug=False>                                                                        
                                                                                                                                                                                                                   
    async def test_cancel_waiters(self, loop) -> None:
        ev = EventResultOrError(loop=loop)                                                                                                                                                                         
                                                                                                                                                                                                                   
        async def c():                                                                                   
            await ev.wait()                                                                                                                                                                                        
                                                                                                                                                                                                                   
        t1 = loop.create_task(c())                                                                       
        t2 = loop.create_task(c())                                                                                                                                                                                 
>       await asyncio.sleep(0, loop=loop)                                                                                                                                                                          
E       TypeError: sleep() got an unexpected keyword argument 'loop'                                                                                                                                               

Python Version

$ python --version
Python 3.10.0b4

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.7.4
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: [email protected]
License: Apache 2
Location: /usr/lib/python3.9/site-packages
Requires: attrs, chardet, multidict, async-timeout, yarl, typing-extensions
Required-by: Electrum, aiohttp-socks

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 5.1.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /usr/lib/python3.10/site-packages
Requires: 
Required-by: yarl, aiohttp

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.6.3
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /usr/lib/python3.9/site-packages
Requires: multidict, idna
Required-by: aiohttp

OS

Gentoo unstable amd64.

Related component

Server, Client

Additional context

This seems to fix the tests but I don't know asyncio well enough to be sure that this is the right fix. It also lacks fixes for examples/legacy/crawl.py which also uses the deprecated argument.

commit ec87d9f2b6541599dd7fc8aaebf0fdfbb812ade7
Author: Louis Sautier <[email protected]>
Date:   Tue Jul 20 23:37:27 2021 +0200

    Remove deprecated loop argument from asyncio.sleep/gather calls

diff --git a/aiohttp/web.py b/aiohttp/web.py
index 557e3c3b..52dfdf93 100644
--- a/aiohttp/web.py
+++ b/aiohttp/web.py
@@ -441,7 +441,7 @@ def _cancel_tasks(
         task.cancel()
 
     loop.run_until_complete(
-        asyncio.gather(*to_cancel, loop=loop, return_exceptions=True)
+        asyncio.gather(*to_cancel, return_exceptions=True)
     )
 
     for task in to_cancel:
diff --git a/tests/test_locks.py b/tests/test_locks.py
index 55fd2330..5f434eac 100644
--- a/tests/test_locks.py
+++ b/tests/test_locks.py
@@ -18,7 +18,7 @@ class TestEventResultOrError:
             return 1
 
         t = loop.create_task(c())
-        await asyncio.sleep(0, loop=loop)
+        await asyncio.sleep(0)
         e = Exception()
         ev.set(exc=e)
         assert (await t) == e
@@ -31,7 +31,7 @@ class TestEventResultOrError:
             return 1
 
         t = loop.create_task(c())
-        await asyncio.sleep(0, loop=loop)
+        await asyncio.sleep(0)
         ev.set()
         assert (await t) == 1
 
@@ -43,7 +43,7 @@ class TestEventResultOrError:
 
         t1 = loop.create_task(c())
         t2 = loop.create_task(c())
-        await asyncio.sleep(0, loop=loop)
+        await asyncio.sleep(0)
         ev.cancel()
         ev.set()
 
diff --git a/tests/test_proxy_functional.py b/tests/test_proxy_functional.py
index 68763cd4..65c773a1 100644
--- a/tests/test_proxy_functional.py
+++ b/tests/test_proxy_functional.py
@@ -238,7 +238,7 @@ async def test_proxy_http_multi_conn_limit(proxy_test_server, loop) -> None:
         resp = await sess.get(url, proxy=proxy.url)
 
         current_pid = pid
-        await asyncio.sleep(0.2, loop=loop)
+        await asyncio.sleep(0.2)
         assert current_pid == pid
 
         await resp.release()
@@ -443,7 +443,7 @@ async def xtest_proxy_https_multi_conn_limit(proxy_test_server, loop):
         resp = await sess.get(url, proxy=proxy.url)
 
         current_pid = pid
-        await asyncio.sleep(0.2, loop=loop)
+        await asyncio.sleep(0.2)
         assert current_pid == pid
 
         await resp.release()

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[Dreamsorcerer]

It also lacks fixes for examples/legacy/crawl.py which also uses the deprecated argument.

The legacy examples are completely broken at the moment. They will need to removed or completely refactored, so just ignore them for now.

First thing should probably be to figure out why these errors don't appear in CI. They should be causing DeprecationWarnings, and I don't see any obvious reason why they are not appearing.

[Hanaasagi]

Maybe I can solve this. If nobody is working for this, then please assign to me.

[Dreamsorcerer]

Sounds good to me. Ideally if you can get the deprecation warnings appearing in CI first, then fix the issues that come up (which should already be in the patch at the top).

[Dreamsorcerer]

OK, did a bit of debugging on why the deprecation warnings are not appearing. The asyncio.gather() does not create a warning unless the task list is empty, so it appears to be a Python bug:
https://github.com/python/cpython/pull/27568/files

Likewise for asyncio.sleep() when the first argument is 0 there is no warning.

[Dreamsorcerer]

Right, I think we can just remove the loop arguments now, with that mystery solved.