Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solved the bug of inference failure when target_path is set but there is no model in it #226

Merged
merged 7 commits into from
Sep 13, 2024

Conversation

JingofXin
Copy link
Contributor

For models that are only deployed, lazy deployment is used on a single machine, and there is a merge folder in the current path, merge will be used first. This will cause errors in model loading.

【Bug】
Reproduction conditions:

  • Single machine
  • There is a merge folder in the current path
  • Only deploy models
    image

【Solve】
image

@wzh1994
Copy link
Contributor

wzh1994 commented Sep 5, 2024

如线下沟通,这里的逻辑统一改成:

  1. 当设置了base-model和target-path时,优先部署base-model
  2. 用户可以指定用target-path中的特定微调版本部署
  3. 当调用update进行微调时,微调结束立即设置当前版本为部署版本,然后进行部署和评测

此外应注意每个模型可以起1-多个微调任务,多个任务相互不影响

lazyllm/module/module.py Outdated Show resolved Hide resolved
lazyllm/module/module.py Outdated Show resolved Hide resolved
lazyllm/module/module.py Outdated Show resolved Hide resolved
lazyllm/module/module.py Outdated Show resolved Hide resolved
lazyllm/module/module.py Show resolved Hide resolved
lazyllm/components/finetune/alpacalora.py Show resolved Hide resolved
lazyllm/module/module.py Show resolved Hide resolved
lazyllm/components/finetune/alpacalora.py Show resolved Hide resolved
@wzh1994 wzh1994 changed the title Bug: local model just deploy but use merge path to load model Solved the bug of inference failure when target_path is set but there is no model in it Sep 13, 2024
@wzh1994 wzh1994 merged commit f1ff0ec into LazyAGI:main Sep 13, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants