The tasks are not cached for pull request CI

[eu9ene]

I would expect the following behavior: if all the data and code files that serve as a step input (including ci kinds and transforms) are unchanged, it should use the cached output. Sometimes you want to debug the last step of the pipeline on taskcluster and CI is a convenient way to do that. Then it shouldn’t rerun the whole thing every time. It adds to costs and wait time.

Also, the cache should be reused across all the PRs/runs. I don't see any risks with that as soon as we can trust the caching mechanism to take into account everything that affects the output.

[bhearsum]

I agree this is something that we should do. Even with the much speedier CI runs we have now, there's no reason to re-run things we've already done in another PR.

I don't think we can safely do this until we're using level 3 workers for real training runs again. At the moment, both real training runs and PRs use the "level 1" cache, which means that if we allow PRs to add things to the cache, it could potentially pollute things for real runs.

[eu9ene]

It's fixed now after this has landed.