[azure] support latest version for image id in azure cloud

[Yisaer]

As titled.

close #4435

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)

I manually tested with following task:

resources:
  cloud: azure
  region: japaneast
  instance_type: Standard_D2s_v3
  image_id: Canonical:ubuntu-24_04-lts:server:latest

# Working directory (optional) containing the project codebase.
# Its contents are synced to ~/sky_workdir/ on the cluster.
workdir: .

# Typical use: pip install -r requirements.txt
# Invoked under the workdir (i.e., can use its files).
setup: |
  echo "Running setup."

# Typical use: make use of resources, such as running training.
# Invoked under the workdir (i.e., can use its files).
run: |
  echo "Hello, SkyPilot!"
  conda env list

The cluster was launched and the job work well. I checked the image in the azure console, and it was ubuntu-24_04-lts as expected

• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

[cg505]

Thank you for the PR, @Yisaer! This will be a useful feature. I appreciate you digging into this problem because the Azure docs are kind of unclear about it. Left a comment to check on.

[cg505]

Did you confirm that the name field here is the image version number? I assume it is a string, so will max() work correctly? E.g. we need to make sure that 1.0.10 > 1.0.2.

[Yisaer]

Thanks for pointing out the max() problem, it should use packaging.version to parse the image version here.
Also I tried to find out the meaning of the name in the azure document, but it's not clear enough.
So when I tested my pr, I debug the returned versions during the runtime, and the returned name here was acutuall the image version number.

[cg505]

@Yisaer Just checking - we don't need this in make_deploy_resources_variables as well? Just for the image size check?

[Yisaer]

@Yisaer Just checking - we don't need this in make_deploy_resources_variables as well? Just for the image size check?

@cg505 Yes, it appears that compute_client.virtual_machine_images.get does not accept the latest image version, whereas compute_client.virtual_machines.begin_create_or_update does.

By querying the image size through a precise image version, while directly using the latest image version when creating a VM, there might be a difference in image information. However, the probability of this occurring is very small, and I think this is an acceptable solution.

[cg505]

Thank you, it looks good to me! Let's have @Michaelvll review as well.

[Michaelvll]

@Yisaer Just checking - we don't need this in make_deploy_resources_variables as well? Just for the image size check?

@cg505 Yes, it appears that compute_client.virtual_machine_images.get does not accept the latest image version, whereas compute_client.virtual_machines.begin_create_or_update does.

By querying the image size through a precise image version, while directly using the latest image version when creating a VM, there might be a difference in image information. However, the probability of this occurring is very small, and I think this is an acceptable solution.

Thanks for the explanation! We should mention this in the comment as well.