[Feature] Set better autoscaling defaults

[DmitriGekhtman]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

The historical default behavior for the autoscaler is to set the worker idleTimeout to 5 upscalingSpeed to 1 (rate-limit scale-up).

Many (perhaps most) users find this behavior undesirable -- more responsive behavior from the Ray autoscaler is usually preferred.

For upscaling, maxReplicas is enough of an upper cap on autoscaling. Users tend to prefer upscaling as soon as they submit a Ray workload.
For downscaling, users prefer to release resources as soon as they are not needed, particularly since lower-level infrastructure (cluster autoscaler) may keep nodes around even if after the Ray pod is terminated.

The current options exposed by KubeRay are
upscalingMode: Default (rate-limited) and
upscalingMode: Aggressive (no rate limit).

I'd change these to
upscalingMode: Default (no rate limit)
upscalingMode: Conservative (rate limit, translates to upscaling_speed 1).

For compatibility we can keep Aggressive and make it equivalent to Default.

Use case

Responsive autoscaling. More suitable defaults for common use-cases.

Related issues

Follow up to #278

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[DmitriGekhtman]

Might actually leave this alone for now and instead cover the behavior in the docs.

[DmitriGekhtman]

Actually, might just go ahead and open a PR for this after all.