read_chunk wait forever when client aborts file uploading

[yanxurui]

🐞 Describe the bug

When I am uploading files to the server using ajax,the server's read_chunk does not raise any exception when client ajax aborts. That is, the server's handler will wait foreever.

💡 To Reproduce

Quickstart -> File Uploads
https://docs.aiohttp.org/en/stable/web_quickstart.html#file-uploads

💡 Expected behavior

An exception should be raised

📋 Logs/tracebacks

📋 Your version of the Python

$ python --version
Python 3.8.1

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.6.2
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: [email protected]
License: Apache 2
Location: /home/yxr/.pyenv/versions/3.8.1/lib/python3.8/site-packages
Requires: attrs, chardet, async-timeout, yarl, multidict
Required-by: aiohttp-session, aiohttp-security, aiohttp-jinja2, aiohttpdemo-chat, snapfile

$ python -m pip show multidict
Name: multidict
Version: 4.7.5
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /home/yxr/.pyenv/versions/3.8.1/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp

$ python -m pip show yarl
Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /home/yxr/.pyenv/versions/3.8.1/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp

📋 Additional context

server

[yanxurui]

Has this been fixed?
The master branch is doing right by throwing a ConnectionResetError exception.

[yanxurui]

you can see where it throws from the log below

ERROR:aiohttp.server:Error handling request
Traceback (most recent call last):
  File "/Users/yxr/Documents/programming/test/python/aiohttp/aiohttp/aiohttp/web_protocol.py", line 414, in _handle_request
    resp = await self._request_handler(request)
  File "/Users/yxr/Documents/programming/test/python/aiohttp/aiohttp/aiohttp/web_app.py", line 334, in _handle
    resp = await handler(request)
  File "app.py", line 32, in store_mp3_handler
    chunk = await field.read_chunk(1024*1024)  # 8192 bytes by default.
  File "/Users/yxr/Documents/programming/test/python/aiohttp/aiohttp/aiohttp/multipart.py", line 309, in read_chunk
    chunk = await self._read_chunk_from_stream(size)
  File "/Users/yxr/Documents/programming/test/python/aiohttp/aiohttp/aiohttp/multipart.py", line 363, in _read_chunk_from_stream
    chunk = await self._content.read(size)
  File "/Users/yxr/Documents/programming/test/python/aiohttp/aiohttp/aiohttp/streams.py", line 358, in read
    await self._wait('read')
  File "/Users/yxr/Documents/programming/test/python/aiohttp/aiohttp/aiohttp/streams.py", line 299, in _wait
    await waiter
ConnectionResetError: Connection lost
INFO:aiohttp.access:127.0.0.1 [16/May/2020:23:23:32 +0800] "POST /upload HTTP/1.1" 500 0 "http://localhost:8080/index.html" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1"

BTW, do you have any plan for the new version?

[WisdomPill]

Have you tried version 4.0.1a?

[yanxurui]

@WisdomPill No. Pip will install 3.6.2 by default and it's problematic. The master branch is 4.0.0a and it works.

[WisdomPill]

you can install it using pip like the following pip install aiohttp==4.0.0a1

regarding the new release follow here https://aio-libs.discourse.group/t/aiohttp-new-release/49 as suggested in #4516

[yanxurui]

Finnally I figured it out.
In 3.6.2, a CancelledError is thrown.

here is a simple demo handler

async def store_mp3_handler(request):

    reader = await request.multipart()

    # /!\ Don't forget to validate your inputs /!\

    # reader.next() will `yield` the fields of your form

    field = await reader.next()
    assert field.name == 'mp3_2'
    filename = field.filename
    # You cannot rely on Content-Length if transfer is chunked.
    size = 0
    try:
        with open(filename, 'wb') as f:
            print('begin')
            while True:
                # pdb.set_trace()
                chunk = await field.read_chunk(1024*1024)  # 8192 bytes by default.
                if not chunk:
                    break
                print(len(chunk))
                size += len(chunk)
                f.write(chunk)
                sleep(1)
            print('end')
    except:
        logging.exception('oops')

    return web.Response(text='{} sized of {} successfully stored'
                             ''.format(filename, size))

In order to simulate abort uploading, just close the browser while uploading is still in progress

[yanxurui]

logging.exception('oops') output

ERROR:root:oops
Traceback (most recent call last):
  File "app.py", line 32, in store_mp3_handler
    chunk = await field.read_chunk(1024*1024)  # 8192 bytes by default.
  File "/Users/yxr/.pyenv/versions/tmp/lib/python3.7/site-packages/aiohttp/multipart.py", line 303, in read_chunk
    chunk = await self._read_chunk_from_stream(size)
  File "/Users/yxr/.pyenv/versions/tmp/lib/python3.7/site-packages/aiohttp/multipart.py", line 332, in _read_chunk_from_stream
    chunk = await self._content.read(size)
  File "/Users/yxr/.pyenv/versions/tmp/lib/python3.7/site-packages/aiohttp/streams.py", line 368, in read
    await self._wait('read')
  File "/Users/yxr/.pyenv/versions/tmp/lib/python3.7/site-packages/aiohttp/streams.py", line 298, in _wait
    await waiter
concurrent.futures._base.CancelledError