Skip to content

Commit

Permalink
[Core] Use uvloop with zmq-decoupled front-end (vllm-project#7570)
Browse files Browse the repository at this point in the history
Signed-off-by: Alvant <[email protected]>
  • Loading branch information
njhill authored and Alvant committed Oct 26, 2024
1 parent 873b662 commit 872bf80
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion vllm/entrypoints/openai/rpc/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from typing import Any, Coroutine

import cloudpickle
import uvloop
import zmq
import zmq.asyncio
from typing_extensions import Never
Expand Down Expand Up @@ -217,4 +218,4 @@ def signal_handler() -> None:
def run_rpc_server(async_engine_args: AsyncEngineArgs,
usage_context: UsageContext, rpc_path: str):
server = AsyncEngineRPCServer(async_engine_args, usage_context, rpc_path)
asyncio.run(run_server(server))
uvloop.run(run_server(server))

0 comments on commit 872bf80

Please sign in to comment.