Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] write nvidia tool wrappers to allow old YARN versions to work with MIG #4305

Closed
tgravescs opened this issue Dec 6, 2021 · 0 comments · Fixed by NVIDIA/spark-rapids-examples#87
Assignees
Labels
feature request New feature or request P0 Must have for release

Comments

@tgravescs
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Since many customers are slow to upgrade YARN versions and older versions of YARN (pre YARN 3.3.0) do not have a pluggable device framework, write some wrappers for nvidia-smi and nvidia-container-cli that return a format such that existing YARN versions work with MIG devices.

For instance nvidia-smi would return output such that the MIG devices look like normal GPUs, then corresonding the nvidia-container-cli which is used to launch docker would need to translate that device back into a MIG device and pass along the properly formatted id (ie 0:3 instead of just 3).

@tgravescs tgravescs added feature request New feature or request ? - Needs Triage Need team to review and classify labels Dec 6, 2021
@tgravescs tgravescs self-assigned this Dec 6, 2021
@Salonijain27 Salonijain27 removed the ? - Needs Triage Need team to review and classify label Dec 7, 2021
@sameerz sameerz added this to the Dec 13 - Jan 7 milestone Dec 10, 2021
@sameerz sameerz assigned gerashegalov and unassigned tgravescs Dec 10, 2021
@sameerz sameerz added the P0 Must have for release label Dec 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request P0 Must have for release
Projects
None yet
4 participants