TypeError: getaddrinfo() argument 1 must be string or None #4557
Comments
|
@likewhoa I can not reproduce the issue. Please give exact steps, or at least debug on your system what is sent to getaddrinfo as the first argument. Without that the issue can not be solved. |
|
@socketpair I am basically reproducing with the example client code at https://docs.aiohttp.org/en/stable/#getting-started |
|
You forgot to specify version of aiohttp. Also, it can be related to your computer configuration, any chance that you can run it on another machine? |
He specified it but posted under the wrong section (the command says
|
webknjaz
added
the
reproducer: missing
|
for multidict info this is it @hh-h i can't reproduce on another server |
|
@likewhoa to make this report useful, we need a stable reproducer, I don't think we can move the investigation forward w/o a way to reproduce it on any dev's env. You need to find out what the difference is between your two envs where you try to run that code. |
What are some steps that you would suggest. I verified that aiohttp/yarl/multidict versions match on both servers. The other difference is kernel with the server with issues having kernel 5.0.x while on server were I was no able to reproduce issue is running 5.4.x kernel. |
You could try to stick a debugger here and see what the args are and then trace it to the calls in your app or in aiohttp at least. |
|
@webknjaz both servers are showing this for args host=python.org,port=80,family=0,type=1,proto=0,flags=0
host=python.org,port=443,family=0,type=1,proto=0,flags=0
host=www.python.org,port=443,family=0,type=1,proto=0,flags=0but server in question has this extra one host=10,port=b'\x01\xbb\x00\x00\x00\x00*\x04NB\x00\x05\x00\x00',family=10,type=1,proto=6,flags=4both using en_US.utf8 locale. |
|
Well, this is weird. You need to go up the stack and grab the local var values from each frame up... |
I will do that, I did a |
|
Also, try to check DNS records of |
Looks like it's an IPv6 address: Also, calling that socket method gives me this: >>> socket.getaddrinfo('www.python.org', 80)
[(<AddressFamily.AF_INET: 2>,
<SocketKind.SOCK_STREAM: 1>,
6,
'',
('151.101.112.223', 80)),
(<AddressFamily.AF_INET: 2>,
<SocketKind.SOCK_DGRAM: 2>,
17,
'',
('151.101.112.223', 80)),
(<AddressFamily.AF_INET: 2>,
<SocketKind.SOCK_RAW: 3>,
0,
'',
('151.101.112.223', 80)),
(<AddressFamily.AF_INET6: 10>,
<SocketKind.SOCK_STREAM: 1>,
6,
'',
('2a04:4e42:1b::223', 80, 0, 0)),
(<AddressFamily.AF_INET6: 10>,
<SocketKind.SOCK_DGRAM: 2>,
17,
'',
('2a04:4e42:1b::223', 80, 0, 0)),
(<AddressFamily.AF_INET6: 10>,
<SocketKind.SOCK_RAW: 3>,
0,
'',
('2a04:4e42:1b::223', 80, 0, 0))] |
|
Looking at the traceback again, it appears that this broken value may be coming from https://github.com/aio-libs/aiohttp/blob/4a08f89/aiohttp/connector.py#L943-L956. Worth checking on the affected server with a debugger. |
|
@webknjaz yea it's not just with python.org, the reason you're seeing that is because I pointed to the official client example. A while ago, i had same issue and commenting out the ipv6 line in /etc/hosts fixed it but this is not the case this time, perhaps it's looking else? |
|
Do you use any other domains in your code? |
|
@weastur looks like https://github.com/aio-libs/aiohttp/blob/4a08f89/aiohttp/connector.py#L943 is what is returning when changing the domain, so asyncio.shield is what is putting in that port. with regards to my code, it uses many domains but i am reproducing this with a simple example aiohttp client code found upstream. |
|
What is used -- aiodns (pycares) or thread resolver ? I bet, first. |
aiodns not on the system. I am going to disable ipv6 all together on system though to rule that out |
|
@likewhoa is on Gentoo using Portage-managed Python packages. I tried to reproduce with the following package versions:
but everything worked as normal. Because aiohttp, multidict and yarl all have compiled C code, one possibility is glibc was upgraded or the Gentoo profile was changed after one or more of them had been installed. @likewhoa is that possible? If so, can you try again after re-emerging aiohttp, multidict, and yarl? |
|
@webknjaz @socketpair so disabling ipv6 fixes the issue but now makes you wonder why would ipv6 support enable be the issue. |
I didn't see there was any changes to the profile or packages relating to python, so I figured it was something else and my first suspect was ipv6 support and once i disabled that everything started working. We would need to figure out why this is the case though. |
|
Strange. On that Gentoo testbed where I couldn't repro: I'd expect IPv6 come into play during the |
|
@JustinTArthur strange indeed but since I don't have use for ipv6 it's best to be disabled until I actually need it on this server. On my workstation I also have ipv6 enabled and i can't reproduce. |
Here is my setup, the only difference is glibc and probably gcc as I use ~arch (unstable) but my workstation matches this and it doesn't reproduce the issue even with ipv6 enabled
and my profile is but keep in mind, i was using 17.0 when i ran into issue, i just updated to 17.1 just now. |
First two bytes are
The same here, but port is https://github.com/aio-libs/aiohttp/blob/v3.6.1/aiohttp/connector.py#L774 Try disabling dns cache. and re-check. conn = aiohttp.TCPConnector(use_dns_cache=False)
session = aiohttp.ClientSession(connector=conn) |
I will try this later today and get back to you with results. |
|
@likewhoa I'm very curious to ask how is it going ? :) |
sorry got distracted since I got the workaround going but I will try your suggestion this coming week. |
No go, here is the sample code I used to test import aiohttp
import asyncio
async def fetch(session, url):
async with session.get(url) as response:
return await response.text()
async def main():
conn = aiohttp.TCPConnector(use_dns_cache=False)
async with aiohttp.ClientSession(connector=conn) as session:
html = await fetch(session, 'http://python.org')
print(html)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main()Can you confirm the glibc version you tested with? I am using sys-libs/glibc-2.30-r3 and someone else suggested this could be the issue but I ran into this once with a previous glibc version so I am assuming that's not the real cause here. At this point I just disabled ipv6 with sysctl to work around it. |
|
Try to reduce down the code even more. For example, try to use just the connector. I mean import aiohttp
import asyncio
from yarl import URL
from aiohttp.client import ClientRequest, DEFAULT_TIMEOUT
async def main():
conn = aiohttp.TCPConnector(use_dns_cache=False)
rq = ClientRequest('GET', URL('http://python.org'))
c = await conn.connect(rq, [], DEFAULT_TIMEOUT)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main()) |
|
If it still fails, try copy-pasting the |
output |
|
I ran into a very similar issue today with aiohttp==3.6.3 and Python 3.7.9. For me, it appears to the difference between calling $ cat bug.py
import asyncio
import socket
from aiohttp.resolver import ThreadedResolver
async def main():
resolver = ThreadedResolver()
print("0:", await resolver.resolve('localhost', 8080, family=0))
print(socket.getaddrinfo('localhost', 8080, type=socket.SOCK_STREAM, family=0))
print("===")
print("AF_INET:", await resolver.resolve('localhost', 8080, family=socket.AF_INET))
print(socket.getaddrinfo('localhost', 8080, type=socket.SOCK_STREAM, family=socket.AF_INET))
asyncio.run(main())$ python bug.py
0: [{'hostname': 'localhost', 'host': '127.0.0.1', 'port': 8080, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'localhost', 'host': 10, 'port': b'\x1f\x90\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'family': <AddressFamily.AF_INET6: 10>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}]
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8080)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', (10, b'\x1f\x90\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'))]
===
AF_INET: [{'hostname': 'localhost', 'host': '127.0.0.1', 'port': 8080, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}, {'hostname': 'localhost', 'host': '127.0.0.1', 'port': 8080, 'family': <AddressFamily.AF_INET: 2>, 'proto': 6, 'flags': <AddressInfo.AI_NUMERICHOST: 4>}]
[(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8080)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8080))] |
same here, I don't compile ipv6 since I am not using it. |
|
I think this may be related to issues in aiodns which have been fixed. Can anyone try reproducing it again? |
Dreamsorcerer
added
the
needs-info
|
likely fixed by #8295 or a cpython change |
|
No response, and cannot reproduce. I'm going to assume this is fixed by #8295 |
🐞 Describe the bug
While trying to fetch urls I am getting the above error message and I recalled a while ago that it was caused by an ipv6 line in /etc/hosts so I checked to see if that had been uncommented out and that was not the case, so now I don't know what could be the issue. The server has no ipv6 support enabled or any ipv6 addresses.
💡 To Reproduce
You can reproduce by running the client example code at https://docs.aiohttp.org/en/stable/#getting-started
💡 Expected behavior
Fetch website html
📋 Logs/tracebacks
📋 Your version of the Python
📋 Your version of the aiohttp/yarl/multidict distributions
📋 Additional context
The text was updated successfully, but these errors were encountered: