Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Qwen 2.5 k8s #3960

Merged
merged 4 commits into from
Sep 18, 2024
Merged

Qwen 2.5 k8s #3960

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a1f9786
Update qwen example for 2.5 release
Michaelvll Sep 18, 2024
0f488cb
Add support for qwen 2.5 example
Michaelvll Sep 18, 2024
fe1a0fd
add kubernetes
Michaelvll Sep 18, 2024
27ca278
Merge branch 'master' of github.com:skypilot-org/skypilot into qwen-2…
Michaelvll Sep 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions llm/qwen/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Serving Qwen2 on Your Own Cloud
# Serving Qwen2 on Your Own Kubernetes or Cloud

[Qwen2](https://github.com/QwenLM/Qwen2) is one of the top open LLMs.
As of Jun 2024, Qwen1.5-110B-Chat is ranked higher than GPT-4-0613 on the [LMSYS Chatbot Arena Leaderboard](https://chat.lmsys.org/?leaderboard).
Expand All @@ -18,10 +18,10 @@ As of Jun 2024, Qwen1.5-110B-Chat is ranked higher than GPT-4-0613 on the [LMSYS

## Why use SkyPilot to deploy over commercial hosted solutions?

* Get the best GPU availability by utilizing multiple resources pools across multiple regions and clouds.
* Pay absolute minimum — SkyPilot picks the cheapest resources across regions and clouds. No managed solution markups.
* Get the best GPU availability by utilizing multiple resources pools across Kubernetes clusters and multiple regions/clouds.
* Pay absolute minimum — SkyPilot picks the cheapest resources across Kubernetes clusters and regions/clouds. No managed solution markups.
* Scale up to multiple replicas across different locations and accelerators, all served with a single endpoint
* Everything stays in your cloud account (your VMs & buckets)
* Everything stays in your Kubernetes or cloud account (your VMs & buckets)
* Completely private - no one else sees your chat history


Expand Down
Loading