Don't decompress responses for HEAD requests

[kxepal]

aiohttp causes error on receiving response against HEAD request with CONTENT-ENCODING header.

Traceback (most recent call last):
  File "/home/kxepal/projects/aiocouchdb/test.py", line 94, in <module>
    loop.run_until_complete(go('http://localhost:5984'))
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/base_events.py", line 208, in run_until_complete
    return future.result()
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/futures.py", line 243, in result
    raise self._exception
  File "/usr/local/opt/virtualenv/aiocouchdb/lib/python3.3/site-packages/asyncio/tasks.py", line 319, in _step
    result = coro.send(value)
  File "/home/kxepal/projects/aiocouchdb/test.py", line 41, in go
    att = yield from doc.att('bar.txt')
  File "/home/kxepal/projects/aiocouchdb/aiocouchdb/document.py", line 58, in attachment
    yield from resp.read()
  File "/home/kxepal/projects/aiocouchdb/aiocouchdb/client.py", line 91, in read
    data.extend((yield from self.content.read()))
  File "/home/kxepal/projects/aiohttp/aiohttp/streams.py", line 382, in read
    return (yield from super().read())
  File "/home/kxepal/projects/aiohttp/aiohttp/streams.py", line 393, in read
    return (yield from super().read())
  File "/home/kxepal/projects/aiohttp/aiohttp/streams.py", line 353, in read
    raise self._exception
  File "/home/kxepal/projects/aiohttp/aiohttp/parsers.py", line 197, in set_parser
    next(p)
  File "/home/kxepal/projects/aiohttp/aiohttp/protocol.py", line 307, in __call__
    out.feed_eof()
  File "/home/kxepal/projects/aiohttp/aiohttp/protocol.py", line 386, in feed_eof
    raise errors.IncompleteRead(0)
aiohttp.errors.IncompleteRead: IncompleteRead(0 bytes read)

This happens because DeflateBuffer is been used if response is compressed and it tries to decompress the received data on feed_eof. Obliviously, it fails because no data is returned back on HEAD request.

[fafhrd91]

Looks ok, but that seems fix for this specific problem. What if you get empty body for other responses. Should deflatebuffer support empty body in general instead?

[kxepal]

@fafhrd91 others aren't affected by this change since it depends on response_with_body flag which sets only for HEAD requests: https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/client.py#L615
Unless you'd set it manually to False for others, but even if you did you probably had some significant reasons for that and you have to understand the related behaviour.

[kxepal]

As in general, for others, this is an error case caused by decoding mechanisms must be applied :

The Content-Encoding entity-header field is used as a modifier to the media-type. When
present, its value indicates what additional content codings have been applied to the 
entity-body, and thus what decoding mechanisms must be applied in order to obtain the 
media-type referenced by the Content-Type header field.

-- http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.11

The error raises not by HTTP protocol routines, but by decodings one.

[kxepal]

@fafhrd91 I got your idea. There is also good old HTTP 204 No Content response which MUST NOT contains payload body, but there is nothing about response headers. And it also fails into the same trap. PR updated with the fix.

[fafhrd91]

I'm talking about bugs in server software. Right now aiohttp can't handle
zero body response with deflate encoding, I'd prefer to cover all responses
and not just head

On Monday, September 29, 2014, Alexander Shorin [email protected]
wrote:

@fafhrd91 https://github.com/fafhrd91 I got your idea. There is also
good old HTTP 204 No Content response which MUST NOT contains payload
body, but there is nothing about response headers. And it also fails into
the same trap. PR updated with the fix.

—
Reply to this email directly or view it on GitHub
#154 (comment).

[kxepal]

Fair enough, agreed. I'll open another PR with proper fix and better name with backreference.

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.