Rate Limiting to prevent Trello errors?

[mockdeep]

I saw this issue, but had a slightly different suggestion to handle Trello rate limiting. It would be great if the client automatically accounted for Trello API rate limits and simply delayed requests as necessary to prevent errors. I use threading to speed up my scripts, but then I'll put in some sleeps to prevent hitting the rate limiting.

[runlevel5]

@mockdeep I don't think sleep is the solution because there is no way you could guarantee the delay helps hitting the API within the threshold, just imagine you have huge number of API calls here.

I think it should be handled case by case by the user app.

[mockdeep]

@joneslee85 sorry, I didn't mean to suggest that there should be an arbitrary sleep, more just that that's what worked for me in that case. Here I could see doing something a little more sophisticated to keep track of how many requests have been made within a timeframe. Something like:

module Trello
  class Client
    REQUEST_MUTEX = Mutex.new
    WINDOW_SIZE = 10000 # 10 seconds
    MAX_REQUESTS_PER_WINDOW = 100

    class << self
      attr_accessor :window_start, :window_request_count
    end
    ...
    def invoke_verb(name, uri, body = nil)
      REQUEST_MUTEX.synchronize do
        self.class.window_start ||= Time.now
        self.class.window_request_count ||= 0
        if self.class.window_request_count > MAX_REQUESTS_PER_WINDOW && Time.now - self.class.window_start < WINDOW_SIZE
          sleep(WINDOW_SIZE - (Time.now - self.class.window_size))
          self.class.window_start = Time.now
          self.class.window_request_count = 0
        end
        # probably more logic to keep track...
        self.class.window_request_count += 1
      end
      # other invoke stuff
    end
  end
end

[mockdeep]

I could see making this sort of functionality part of an external gem, though. There's an old throttling gem that does something similar. Maybe I'll try to build something, since I've run into this sort of problem a few times.

[runlevel5]

@mockdeep IMHO I want to keep this gem as simple and as close to the supported API that Trello offer. Feature like this tends to be varied case by case. I 👍 if it is handled by a separate gem. I am happy to feature the link to the gem in README should users look for one.

[mockdeep]

Sounds good. I'll work on something someday when I'm trying to put off doing something else.

[psy-q]

@mockdeep Have you had any success hacking deeper into this? I'm hitting the rate limit a lot as well and will solve it on the client side with a stupid and primitive thing. But since Trello's rate limits seem to be hardcoded (?), having a solution in a gem that fits everybody would be much better.

Edit: I can't solve this from the client side since I don't know how many HTTP requests the Trello library will produce internally when I call the various methods.

[mockdeep]

@psy-q I've generally resolved this with sleep and retry logic. Here's a simple Retryer class with exponential backoff you could toss in your code:

class Retryer
  def initialize
    @retry_count = 0
  end

  def call(retry_on:, retry_times: 5)
    yield
  rescue StandardError => ex
    raise ex if @retry_count >= retry_times || !accepted_error?(ex.class, retry_on)

    @retry_count += 1
    sleep 0.1 * 2**@retry_count
    retry
  end

  def accepted_error?(klass, retry_on)
    retry_on.is_a?(Enumerable) ? retry_on.include?(klass) : retry_on == klass
  end
end

Another thing I've done is to manually cache results from Trello to minimize requerying for the same data.

[psy-q]

Ah, brilliant, thank you!

[cblackburn-ajla]

Not sure when Trello started sending rate-limit stats back in the headers for responses but wouldn't this make it super easy to manage and stay within rate limits? https://developers.trello.com/docs/rate-limits . I'll work on a PR to use these headers for myself. Hopefully it will be useful to others.

Update: After testing the rate-limit headers, they apparently do not work. I can exceed my limit which raises:

Code/ruby/Gems/ruby-trello/lib/trello/client.rb:99:in `invoke_verb': {"error":"RATE_LIMIT_EXCEEDED","message":"Exceeded rate limit to /1/member"} (Trello::Error)

However the headers remain as follows:

 :x_rate_limit_api_key_interval_ms=>"10000",
 :x_rate_limit_api_key_max=>"300",
 :x_rate_limit_api_key_remaining=>"299",
 :x_rate_limit_api_token_interval_ms=>"10000",
 :x_rate_limit_api_token_max=>"100",
 :x_rate_limit_api_token_remaining=>"99",
 :x_rate_limit_member_interval_ms=>"10000",
 :x_rate_limit_member_max=>"200",
 :x_rate_limit_member_remaining=>"199",

So much for high hopes. I'll use the nested resource calls instead, but sadly I'll have to write my own calls for those unless I am just not finding that feature in ruby-trello.