multithreaded ssh setup

[asaiacai]

Runs ssh initialization and setup asynchronously for faster startup. Closes #4156

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
• All smoke tests: pytest tests/test_smoke.py

[Michaelvll]

Thanks for adding this @asaiacai! This is a very important improvement. Some quick thoughts : )

[asaiacai]

these are some preliminary numbers but need to also test k8s and run smoke tests
before PR

(base) ubuntu@ip-172-31-16-12:~/skypilot$ time sky launch --cloud aws --num-nodes 20 --cpus 2+ -y -c scale --region us-east-1

real    1m13.987s
user    0m13.215s
sys     0m1.214s

(base) ubuntu@ip-172-31-16-12:~/skypilot$ time sky launch --cloud aws --num-nodes 40 --cpus 2+ -y -c scale --region us-east-1
real    1m39.759s
user    0m16.282s
sys     0m2.386s

# sky local kubernetes
(base) ubuntu@ip-172-31-16-12:~/skypilot$ time sky launch --cloud kubernetes --num-nodes 30 --cpus 2+ -y -c scale
real    1m58.966s
user    0m39.549s
sys     0m7.922s

after PR

(base) ubuntu@ip-172-31-16-12:~/skypilot$ time sky launch --cloud aws --num-nodes 20 --cpus 2+ -y -c scale --region us-east-1

real    0m58.341s
user    0m11.981s
sys     0m1.223s

(base) ubuntu@ip-172-31-16-12:~/skypilot$ time sky launch --cloud aws --num-nodes 40 --cpus 2+ -y -c scale --region us-east-1

real    1m27.969s
user    0m17.599s
sys     0m2.208s

# sky local kubernetes
(base) ubuntu@ip-172-31-16-12:~/skypilot$ time sky launch --cloud kubernetes --num-nodes 30 --cpus 2+ -y -c scale
real    1m28.728s
user    0m40.810s
sys     0m7.921s

[asaiacai]

reran smoke, but getting pretty limited capacity in AWS which is messing with the smoke tests if I can get some help running smoke on this as well.

[Michaelvll]

LGTM! Thanks for making this work @asaiacai!