jenkins/cloud: Stop binding /srv, use workspace

[cgwalters]

Just use $WORKSPACE for the images - it'll be naturally namespaced
by our job and lifecycled with it.

Note I tried really really hard to also drop --privileged but
the problem is the usage of bwrap drags us into unprivileged
nested containers.

One downside is we'll be rsyncing in the images on each job
invocation, but I'll work on a fix for that later.

[cgwalters]

See bwrap issue 284 for where I got on the unprivileged bits.

[cgwalters]

Rebased 🏄‍♂️

[jlebon]

One downside is we'll be rsyncing in the images on each job
invocation, but I'll work on a fix for that later.

Yeah, that will mean a massive amount of data to rsync on each run. May I suggest until we fix that, we drop the workspace cleanup here?

[cgwalters]

One approach we could take would be to do pruning (and s3 sync) as a separate task. So we drop the --delete, and this job only writes a new directory and swaps the latest symlink, which wouldn't require syncing in anything.

Does that sound OK?

[jlebon]

That makes sense, though... is it possible to do the symlink swap without having to rsync the parent directory? I guess we could revert to scp for that.

[openshift-bot]

@cgwalters: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.