Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load Balancing #34

Open
wzh1994 opened this issue Jun 24, 2024 · 0 comments
Open

Load Balancing #34

wzh1994 opened this issue Jun 24, 2024 · 0 comments
Labels
v0.3
Milestone
LazyLLM v0.3

Comments

@wzh1994
Copy link
Contributor

wzh1994 commented Jun 24, 2024

During inference, when the available number of GPUs exceeds the demand, the tasks are distributed across multiple computing nodes. This is achieved through a load balancing mechanism to manage the task execution.
@wzh1994 wzh1994 added the V0.2 label Jun 24, 2024
@wzh1994 wzh1994 added this to the LazyLLM v0.2 milestone Jun 24, 2024
@wzh1994 wzh1994 modified the milestones: LazyLLM v0.2, LazyLLM v0.3 Aug 14, 2024
@wzh1994 wzh1994 added v0.3 and removed V0.2 labels Aug 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
v0.3
Projects
None yet
Milestone
LazyLLM v0.3
Development

No branches or pull requests
1 participant
@wzh1994