Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do fire DNS tracing events #2858

Merged
merged 2 commits into from
Mar 20, 2018
+41 −5
Merged

Do fire DNS tracing events #2858

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d09d327
Fix #2841: do fire DNS tracing events
asvetlov Mar 20, 2018
7117c2d
Merge branch 'master' into fix-dns-tracing-signals
pfreixes Mar 20, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES/2841.bugfix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix firing DNS tracing events.
4 changes: 2 additions & 2 deletions aiohttp/connector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -728,12 +728,12 @@ async def _create_connection(self, req, traces=None):
if req.proxy:
_, proto = await self._create_proxy_connection(
req,
traces=None
traces=traces
)
else:
_, proto = await self._create_direct_connection(
req,
traces=None
traces=traces
)

return proto
Expand Down
41 changes: 38 additions & 3 deletions tests/test_web_functional.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import io
import json
import pathlib
import socket
import zlib
from unittest import mock

Expand Down Expand Up @@ -1638,10 +1639,14 @@ async def handler(request):
assert resp.status == 200


async def test_request_tracing(aiohttp_client):
async def test_request_tracing(aiohttp_server):

on_request_start = mock.Mock(side_effect=asyncio.coroutine(mock.Mock()))
on_request_end = mock.Mock(side_effect=asyncio.coroutine(mock.Mock()))
on_dns_resolvehost_start = mock.Mock(
side_effect=asyncio.coroutine(mock.Mock()))
on_dns_resolvehost_end = mock.Mock(
side_effect=asyncio.coroutine(mock.Mock()))
on_request_redirect = mock.Mock(side_effect=asyncio.coroutine(mock.Mock()))
on_connection_create_start = mock.Mock(
side_effect=asyncio.coroutine(mock.Mock()))
Expand All @@ -1663,20 +1668,50 @@ async def redirected(request):
on_connection_create_start)
trace_config.on_connection_create_end.append(
on_connection_create_end)
trace_config.on_dns_resolvehost_start.append(
on_dns_resolvehost_start)
trace_config.on_dns_resolvehost_end.append(
on_dns_resolvehost_end)

app = web.Application()
app.router.add_get('/redirector', redirector)
app.router.add_get('/redirected', redirected)
server = await aiohttp_server(app)

class FakeResolver:
_LOCAL_HOST = {0: '127.0.0.1',
socket.AF_INET: '127.0.0.1'}

def __init__(self, fakes):
"""fakes -- dns -> port dict"""
self._fakes = fakes
self._resolver = aiohttp.DefaultResolver()

async def resolve(self, host, port=0, family=socket.AF_INET):
fake_port = self._fakes.get(host)
if fake_port is not None:
return [{'hostname': host,
'host': self._LOCAL_HOST[family], 'port': fake_port,
'family': socket.AF_INET, 'proto': 0,
'flags': socket.AI_NUMERICHOST}]
else:
return await self._resolver.resolve(host, port, family)

client = await aiohttp_client(app, trace_configs=[trace_config])
resolver = FakeResolver({'example.com': server.port})
connector = aiohttp.TCPConnector(resolver=resolver)
client = aiohttp.ClientSession(connector=connector,
trace_configs=[trace_config])

await client.get('/redirector', data="foo")
await client.get('http://example.com/redirector', data="foo")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a f"localhost:{server.port}/redirector" does not fire a DNS event? If it does we wouldn't need it the FakeResolver

Copy link
Member Author

@asvetlov asvetlov Mar 20, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolver doesn't perform DNS lookup for IP addresses, only for domain names.
localhost should be sufficient but unfortunately there are too many boxes with declared but actually disabled IPv6 support.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I don't get the following sentence.

there are too many boxes with declared but actually disabled IPv6 support.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont want to block the PR just for a test that in any case can be at any moment refactorized.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll try to describe. Some broken environments (Travis-CI included IIRC) resolves localhost to both 127.0.0.1 and ::1 but IPv6 stack is disabled.
By this communication over IPv6 ::1 fails.
That's why aiohttp.test_utils and aiohttp test suite use IPv4 127.0.0.1 explicitly.
The situation could be improved I hope but working on it is definitely out if the issue scope.


assert on_request_start.called
assert on_request_end.called
assert on_dns_resolvehost_start.called
assert on_dns_resolvehost_end.called
assert on_request_redirect.called
assert on_connection_create_start.called
assert on_connection_create_end.called
await client.close()


async def test_return_http_exception_deprecated(aiohttp_client):
Expand Down