😀 3 concurrents stream prompts running on a 3060 12gb !!! #3560
celsowm
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
on vllm, 2 or more I always got corrupted tokens, very sad:
bug_vllm.mp4
But I got 3 fast and fine on SGLang ! Thanks all team:
3_concurrent_sglang.mp4
Beta Was this translation helpful? Give feedback.
All reactions