GHA jobs fail because of too many requests

[jakirkham]

Have started seeing GHA jobs fail because of too many request (for example:

#26 ERROR: toomanyrequests: Too Many Requests (HAP429).

------
 > pushing rapidsai/ci-conda:cuda11.8.0-rockylinux8-py3.10-amd64 with docker:
------

This is the same as issue ( rapidsai/miniforge-cuda#72 ), which already contains suggestions on how this could be resolved

[jameslamb]

I just observed the same thing on https://github.com/rapidsai/ci-imgs/actions/runs/11614334107/job/32342353447

43 of 270 jobs failed because they hit this rate limit. All succeeded on a rebuild. Just posting since this is an old issue, to note that this is still a problem we hit sometimes.

[jameslamb]

I just merged #216, which limits this repo to running 150 jobs at a time within a workflow run. Let's see if that helps.

[vyasr]

@jameslamb what data do we want to see before we close this issue?