Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

webui supports nexfort backend and refactor compile related code #966

Merged
merged 27 commits into from
Jun 24, 2024

Conversation

marigoold
Copy link
Collaborator

@marigoold marigoold commented Jun 19, 2024

增加了对 nexfort 后端的支持,但是目前还不支持量化和 controlnet。设置中增加了对后端的选择。目前 3090 sd1.5 512*512 速度是 34it/s,oneflow 后端是 45 it/s。

用 nexfort 后端编译未经 hack 的原生 webui,有以下几个问题,本 PR 针对这几个问题也进行了修复

  1. UNet 的 forward 里面输入是 fp32 的,需要重写一个 forward 把输入都 cast 成 fp16,用重写的 forward 替换原来的 forward
  2. 代码中到处都有 autocast,会导致 nexfort 编译失败,错误原因基本都是 fp16 和 fp32 混合计算导致的,这里在 forward 外层包了一层 autocast(enabled=False) 来关掉 autocast
  3. webui 提供了多种 attention 的 forward 供用户选择,默认的 forward 里面有较多的非计算逻辑(比如判断显存之类),用这个 forward 编译会挂掉。所以自己照抄实现了一个 attention,调用 spda,速度从 23it/s 增加到 34 it/s

@marigoold marigoold changed the title webui supports nexfort backend webui supports nexfort backend and refactor compile related code Jun 21, 2024
Base automatically changed from dev_wy_webui_controlnet to main June 22, 2024 01:19
@marigoold marigoold enabled auto-merge (squash) June 23, 2024 10:48
@marigoold marigoold merged commit 15ce18f into main Jun 24, 2024
7 checks passed
@marigoold marigoold deleted the dev_wy_support_webui_nexfort branch June 24, 2024 01:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants