Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-40007: Make asyncio.transport.writelines on selector use sendmsg #19062

Closed
wants to merge 1 commit into from

Conversation

tzickel
Copy link
Contributor

@tzickel tzickel commented Mar 18, 2020

@tzickel tzickel changed the title bpo-XXX: Make asyncio.transport.writelines on selector use sendmsg bpo-40007: Make asyncio.transport.writelines on selector use sendmsg Mar 18, 2020
Copy link
Contributor

@aeros aeros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @tzickel.

Assuming the general idea is approved of by Yury and/or Andrew, this PR will likely need to include a unit test that ensures it behaves as expected; it would be best included in Lib/test/test_asyncio/test_selector_events.py. See the existing transport tests in there for examples, such as the ones that start with test_write*.

The PR could also use a Misc/NEWS entry to briefly explain the changes.

In the meantime, I have a few suggestions/general comments:

Lib/asyncio/selector_events.py Show resolved Hide resolved
Lib/asyncio/selector_events.py Outdated Show resolved Hide resolved
Lib/asyncio/selector_events.py Outdated Show resolved Hide resolved
Lib/asyncio/selector_events.py Show resolved Hide resolved
Lib/asyncio/selector_events.py Outdated Show resolved Hide resolved
@aeros
Copy link
Contributor

aeros commented Mar 19, 2020

Also, the current CI failures will have to be addressed of course. My comments above were just concerns that I noticed at a glance. I suspect that some of the failures may have been occurring as a result of the seld._buffer code typos that I mentioned, but I haven't looked over them yet.

@tzickel
Copy link
Contributor Author

tzickel commented Mar 19, 2020

@aeros Thanks for your comments, I've hopefully fixed the code / logic mistakes.

@aeros
Copy link
Contributor

aeros commented Mar 20, 2020

@tzickel

Thanks, but it's a small one-time overhead instead of a small overhead on each object initialization

I wasn't referring to repeating the getattr(socket.socket, "sendmsg", False) every time the object is created; that's not necessary. Here's an example of what I had in mind:

Current:

sendmsg = getattr(socket.socket, "sendmsg", False)

# [snip]
def writelines(self, lines):
    if not sendmsg:
        return self.write(b''.join(lines))
    # [snip]

Recommended:

# Used to determine if platform supports sendmsg to sockets
# [_NAME is our typical convention for internal globals, to differentiate it]
_SENDMSG = None

class _SelectorSocketTransport(_SelectorTransport):
    # [snip]

    def __init__(self, loop, sock, protocol, waiter=None,
                 extra=None, server=None):
       
       global _SENDMSG
       # [ensures the `getattr()` is only done once, but doesn't create extra
       # overhead when select_events is imported]
       if _SENDMSG is None:
           _SENDMSG = getattr(socket.socket, "sendmsg", False)
       
      self._sendmsg = _SENDMSG
      # [snip]

    def writelines(self, lines):
        if not self._sendmsg:
            return self.write(b''.join(lines))
        # [snip]

Does that make it more clear?

@aeros
Copy link
Contributor

aeros commented Mar 21, 2020

Also, as a side note, there's typically no need to force push over old commits for PR branches in the CPython repo since we squash prior to merging. It can occasionally help if it the history becomes muddled over time, but it can also make changes made to the PR harder to follow for review purposes.

See https://discuss.python.org/t/pep-601-forbid-return-break-continue-breaking-out-of-finally/2239/42. It's also mentioned at the end of https://devguide.python.org/pullrequest/#submitting.

@asvetlov
Copy link
Contributor

#31871 is the newer resurrection of the idea.

@asvetlov asvetlov closed this Mar 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants