ClientResponseError on header values with non-printable characters #5355
Comments
|
Your example works when I try it. |
|
test.py: |
|
@Dreamsorcerer Maybe that server is not always generating bad responses. Here's a test case with a local server: import aiohttp
from aiohttp import web
import asyncio
from multiprocessing import Process
import time
from urllib.request import urlopen
PORT = 10001
async def handle_req(request):
return aiohttp.web.Response(text="OK", headers={'My-Header': 'foo\1bar'})
async def main():
async with aiohttp.ClientSession() as sess:
r = await sess.get(f"http://127.0.0.1:{PORT}/")
print(await r.text())
def run_server():
app = web.Application()
app.add_routes([web.get('/', handle_req)])
web.run_app(app, host="127.0.0.1", port=PORT)
p = Process(target=run_server)
p.start()
time.sleep(2)
print('urlopen:', urlopen(f"http://127.0.0.1:{PORT}/").headers.items())
asyncio.run(main())
p.join() |
|
Thanks. Seems the exception is raised from this cython code: I'm not at all familiar with that code, I can't even figure out where the code for cparser lives, but hopefully someone else will be able to pick it up from that. |
|
It's probably here: Or here: The fact that http-parser is deprecated doesn't help though (see #3561). |
|
@smola looks like you are trying to use non ASCII char in the header. |
|
@derlih I don't control the server, I'm scraping the web. There are servers in the wild that return non-printable characters in headers. Even if this is not standard, other HTTP implementations (e.g. CPython, curl, web browsers) handle this case gracefully. |
|
@smola |
|
I also have this problem, and @derlih answer worked. In my case i try to fetch information from Toyota Connected Services Endpoints and in What are the plans to fix this, if it can be fixed? |
|
Another example here: |
|
I am getting the same issue when I do the get request using below as a header. |
|
the parser is replaced with llhttp. |
🐞 Describe the bug
When a server replies with a header value including non-printable characters,
ClientResponseErroris thrown with the messageinvalid character in header. This is an issue in the http-parser library.💡 To Reproduce
💡 Expected behavior
Other HTTP clients work just fine with these responses (e.g. CPython, curl, web browsers).
📋 Logs/tracebacks
See above.
📋 Your version of the Python
📋 Your version of the aiohttp/yarl/multidict distributions
📋 Additional context
This may be a duplicate of #5269. It's hard to tell given the context given on that issue.
The text was updated successfully, but these errors were encountered: