Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Support GPU servers for model serving framework #576

Closed
ylwu-amzn opened this issue Nov 28, 2022 · 1 comment
Closed

[FEATURE] Support GPU servers for model serving framework #576

ylwu-amzn opened this issue Nov 28, 2022 · 1 comment
Labels
enhancement New feature or request roadmap v2.5.0 'Issues and PRs related to version v2.5.0'

Comments

@ylwu-amzn
Copy link
Collaborator

ylwu-amzn commented Nov 28, 2022 •
edited
Loading

We released model serving framework in 2.4 as experimental feature (doc link). In 2.4, we have limited support running pytorch model on GPU ML nodes.

In 2.5, we plan to support popular GPU instances:

  1. NVIDIA GPU
  2. AWS Inferentia Instance
@ylwu-amzn ylwu-amzn added enhancement New feature or request untriaged and removed untriaged labels Nov 28, 2022
@sean-zheng-amazon sean-zheng-amazon added the v2.5.0 'Issues and PRs related to version v2.5.0' label Jan 5, 2023
@ylwu-amzn ylwu-amzn changed the title [FEATURE] GPU support for model serving framework [FEATURE] Support popular GPU servers for model serving framework Jan 5, 2023
@ylwu-amzn ylwu-amzn changed the title [FEATURE] Support popular GPU servers for model serving framework [FEATURE] Support GPU servers for model serving framework Jan 5, 2023
@b4sjoo
Copy link
Collaborator

b4sjoo commented Jan 11, 2023

We have added a doc to guide user how to prepare GPU ML node to run model serving framework in regards of this issue #677, so we think we can close it.

@b4sjoo b4sjoo closed this as completed Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request roadmap v2.5.0 'Issues and PRs related to version v2.5.0'
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ylwu-amzn @sean-zheng-amazon @b4sjoo