Spot controller instance

[guillermo-gabrielli-fer]

What instance is actually needed for the spot controller?, if the comment in the template is still valid the instance looks like it's overkill, e.g. it's almost the price of a spot GPU instance or an on-demand basic GPU instance for inference.

If only using a single cloud provider can the requirements be reduced by editing the template file?

[concretevitamin]

Indeed, reducing the fixed cost of the controller has been something on the roadmap. For your use case, specifying cpus: 4+ (cut cost by half) or cpus: 2+ in the template:

skypilot/sky/templates/spot-controller.yaml.j2

Lines 5 to 6 in 8d6f6a9

	resources:
	disk_size: 50

like this

resources:
  disk_size: 50
  cpus: 4+

should work.

Caveat: if you have N vCPUs in the controller, the maximum concurrent spot jobs is 2*N. This is due to the constant set here. For a few jobs, using a smaller controller makes sense; for a larger number of jobs, the controller cost can be amortized.

Let us know if the above works for you?

[guillermo-gabrielli-fer]

Thank you for your response.

How much memory is actually needed if only running let's say 1 or 2 tasks at one time? For the m6 instance ratio of 4GB/cvpu, it looks like an average of 2 GB/tasks, but I guess there is a base cost.

[concretevitamin]

Should be safe to assume up to 500MB-1GB/task. We haven't done a rigorous measurement or breakdown. Most of the memory is used by the Ray runtime.

Feel free to try (assuming the master branch)

resources:
  disk_size: 50
  cpus: 2+
  memory: 2+

for 1-2 tasks. This reduces the default ~$0.38 to ~$0.08.