[AWS/GCP] Upgrade image to support CUDA 12.1

[Michaelvll]

As pytorch has upgraded its default requirement for CUDA to be 12.1, we should upgrade the image to support this latest CUDA version.

[Michaelvll]

Just tried the latest projects/deeplearning-platform-release/global/images/common-gpu-v20231105-debian-11-py310 on GCP, but unfortunately, it does not have CUDA 12.1 installed, which still causes the issue with vllm installation #2786. We should figure out a workaround for this.

[Michaelvll]

This has been fixed by #2788 and skypilot-org/skypilot-catalog#49. Closing.

[romilbhardwaj]

Re-opening this - CUDA version mismatch across GCP and AWS makes it hard to make the same YAML work across clouds. We should update the GCP image (and k8s) to also use CUDA 12, for consistency with AWS.

[Michaelvll]

Re-opening this - CUDA version mismatch across GCP and AWS makes it hard to make the same YAML work across clouds. We should update the GCP image (and k8s) to also use CUDA 12, for consistency with AWS.

Just tested with GCP and it seems the CUDA is indeed 12.2. Is there a way to reproduce this?
sky launch -c test-gpu --gpus l4 nvidia-smi

[romilbhardwaj]

Ahh looks like my catalog was not being auto-updated. I fetched the latest one and now AWS and GCP are at parity.