New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

推理报错CUDA:oom & 添加参数--use-bminf --memory-limit 10执行后，报错缺少参数 #99

Open

Szt-1 opened this issue Jul 20, 2023 · 0 comments

Szt-1 commented Jul 20, 2023

模型:2b的压缩模型
内存16G
显卡：v100 16G
部署后，运行python text_generation.py报错 CUDA：OOM
增加参数--use-bminf --memory-limit 10后执行，报错缺少参数
对bminf的版本是否有要求？具体要求什么版本？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment