Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfixes and improvements to rate limit check #701

Merged
merged 1 commit into from
Apr 23, 2021

Conversation

fredrikekre
Copy link
Member

@fredrikekre fredrikekre commented Apr 22, 2021

Bugfixes and improvements to rate limit check

  • use the correct IP adress of the client
  • use DateTime(0) for first time connections
  • don't warn when capping the rate limit allowance
  • fix rate_limit specification with non-1 denominator

@codecov-commenter
Copy link

codecov-commenter commented Apr 22, 2021

Codecov Report

Merging #701 (8f915e6) into master (2a03ca7) will decrease coverage by 2.97%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #701      +/-   ##
==========================================
- Coverage   77.99%   75.02%   -2.98%     
==========================================
  Files          36       36              
  Lines        2372     2346      -26     
==========================================
- Hits         1850     1760      -90     
- Misses        522      586      +64     
Impacted Files Coverage Δ
src/Servers.jl 79.48% <100.00%> (+12.60%) ⬆️
src/status_messages.jl 1.21% <0.00%> (-98.79%) ⬇️
src/debug.jl 50.00% <0.00%> (-16.67%) ⬇️
src/DebugRequest.jl 0.00% <0.00%> (-12.50%) ⬇️
src/parseutils.jl 66.66% <0.00%> (-6.67%) ⬇️
src/sniff.jl 84.96% <0.00%> (-1.90%) ⬇️
src/ConnectionPool.jl 80.43% <0.00%> (-0.19%) ⬇️
src/Messages.jl 87.76% <0.00%> (-0.18%) ⬇️
src/Parsers.jl 97.89% <0.00%> (-0.03%) ⬇️
src/HTTP.jl 100.00% <0.00%> (ø)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a03ca7...8f915e6. Read the comment docs.

ip = Sockets.getsockname(tcp)[1]
rate = Float64(rate_limit.num)
rl = get!(RATE_LIMITS[Threads.threadid()], ip, RateLimit(rate, Dates.now()))
ip = Sockets.getpeername(tcp)[1]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did this ever work before? It essentially acted like a global rate_limiter (same as max_connections) IIUC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this was never very well-tested

rate = Float64(rate_limit.num)
rl = get!(RATE_LIMITS[Threads.threadid()], ip, RateLimit(rate, Dates.now()))
ip = Sockets.getpeername(tcp)[1]
rl = get!(RATE_LIMITS[Threads.threadid()], ip, RateLimit(rate_limit, Dates.DateTime(0)))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use DateTime(0) here to not "punish" first time connections.

Comment on lines -58 to -61
if rl.allowance > rate
@warn "throttling $ip"
rl.allowance = rate
end
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved into the update! function with the warning removed. Why should HTTP warn here at all?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just thought it might be useful to know, server-side, if there are potentially malicious calls going on.

Copy link
Member Author

@fredrikekre fredrikekre Apr 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is not the case here. For example if rate limiting allowed 2 connections per second, and it was 10 seconds since my last connection this would warn because my allowance would be 20 connections?

@@ -51,14 +54,9 @@ soon, it is closed and discarded, otherwise, the timestamp for the
ip address is updated in the global cache.
"""
function check_rate_limit(tcp, rate_limit::Rational{Int})
ip = Sockets.getsockname(tcp)[1]
rate = Float64(rate_limit.num)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also fixes the a bug if you specify rate_limit = 3//2 or something (e.g. non-1 denominator).

 - use the correct IP adress of the client
 - use DateTime(0) for first time connections
 - don't warn when capping the rate limit allowance
 - fix rate_limit specification with non-1 denominator
Copy link
Member

@quinnj quinnj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@quinnj quinnj merged commit 6058022 into JuliaWeb:master Apr 23, 2021
@fredrikekre fredrikekre deleted the fe/rate-limit branch April 23, 2021 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants