-
Notifications
You must be signed in to change notification settings - Fork 1.1k
flux hitting Google API rate limits when working with GCR #1016
Comments
For the record, these are the settings in question (from https://github.com/weaveworks/flux/blob/master/site/daemon.md)
We could tune them down for everyone by giving them more conservative defaults. As a beer coaster calculation: to fetch image metadata from scratch needs |
... by which I mean, acceptable if you are on GCP and can't make it go faster :) I think it'd be better to tune it down just for GCP, and to do that, we'll have to alter generated config or something else.
We probably do get specific status codes when throttled, so this may be a possibility depending on how well (or if at all) the docker distribution lib exposes those. |
Doesn't that mean it would take that long for the likes of 'Deploy Status' in Weave Cloud to fully populate?
That would be neat. |
Yep, but whatcha gonna do. |
Out of interest, what actually happens when the rate limit is reached? Shouldn't we still get a success at roughly the rate limit? Or to put it another way, what does rate limiting at the flux end actually achieve? |
In the first instance, it's being a good API user. But the other side of that coin is that some registries will blacklist your IP if you are continually bouncing off their throttling. |
Giving the argument |
Google Cloud API seems to have a basic rate limit of 20 reqs/sec: https://cloud.google.com/compute/docs/api-rate-limits
If a user has more than 20 images in their GCR registry, then this rate limit can be triggered. Is there any way to back off, or have preconfigured rate limit values for different image registries so that users do not have to manually configure this setting?
The text was updated successfully, but these errors were encountered: