Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DNSCache race-condition #2621

Merged
merged 1 commit into from
Dec 27, 2017
Merged

Fix DNSCache race-condition #2621

merged 1 commit into from
Dec 27, 2017

Conversation

socketpair
Copy link
Contributor

@socketpair socketpair commented Dec 24, 2017

Are there changes in behavior for the user?

Related issue number

#2620

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
    • The format is <Name> <Surname>.
    • Please keep alphabetical order, the file is sorted by names.
  • Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> for example (588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the pr
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

@socketpair socketpair force-pushed the dnscache_bug branch 2 times, most recently from fe80327 to 7d4dcf4 Compare December 25, 2017 08:03
@socketpair socketpair changed the title Fix DNSCache race-condition (#2620) Fix DNSCache race-condition Dec 25, 2017
@codecov-io
Copy link

codecov-io commented Dec 25, 2017

Codecov Report

Merging #2621 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2621      +/-   ##
==========================================
+ Coverage   97.92%   97.92%   +<.01%     
==========================================
  Files          38       38              
  Lines        7265     7268       +3     
  Branches     1259     1259              
==========================================
+ Hits         7114     7117       +3     
  Misses         47       47              
  Partials      104      104
Impacted Files Coverage Δ
aiohttp/connector.py 96.99% <100%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 079d8d4...2f277f9. Read the comment docs.

return islice(self._addrs_rr[host], len(self._addrs[host]))
addrs = self._addrs[host].copy()
shuffle(addrs)
return addrs
Copy link
Contributor

@pfreixes pfreixes Dec 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not keep the original round robin but making sure that we get rid of the corner case ? such as

>>> addrs = cycle([1, 2, 3, 4])
>>> list(islice(addrs, 4))
[1, 2, 3, 4]
>>> next(addrs)
1
>>> list(islice(addrs, 4))
[2, 3, 4, 1]

Following your idea of using an ad-hoc list per call to next_addrs it's just a matter of make a next after built the list to make sure that we always use a shifted list per call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because, anyway, cycle will start from actually random element. Why not to randomize explicitly ? Why we should stick on round-robin ?

In [7]: timeit list(islice(xxx, 4))
The slowest run took 15.44 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 447 ns per loop

In [8]: timeit shuffle(qwe)
The slowest run took 5.42 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 2.82 µs per loop

Well, it seems you are right....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I fixed.

@socketpair socketpair force-pushed the dnscache_bug branch 4 times, most recently from fca0258 to 668fc3c Compare December 26, 2017 12:04
@pfreixes
Copy link
Contributor

So, ready to be merged? flake8 is failing for some reason. Up to yo to merge it once it is fixed.

@asvetlov
Copy link
Member

Let me review soon.
P.S. Travis is red still :(

@asvetlov
Copy link
Member

Looks good

@asvetlov asvetlov merged commit d4a15fd into master Dec 27, 2017
@asvetlov asvetlov deleted the dnscache_bug branch December 27, 2017 09:03
asvetlov pushed a commit that referenced this pull request Dec 27, 2017
@asvetlov asvetlov mentioned this pull request Dec 27, 2017
@lock
Copy link

lock bot commented Oct 28, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new

@lock lock bot added the outdated label Oct 28, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 28, 2019
@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bot:chronographer:provided There is a change note present in this PR outdated
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants