Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StableDiffusion Pipeline 升级 #377

Closed
JunnYu opened this issue Jan 9, 2024 · 0 comments
Closed

StableDiffusion Pipeline 升级 #377

JunnYu opened this issue Jan 9, 2024 · 0 comments
Labels
HappyOpenSource Pro 快乐开源issue与PR,更具挑战的任务

Comments

@JunnYu
Copy link
Member

JunnYu commented Jan 9, 2024

StableDiffusion Pipeline 升级

任务描述

任务背景

  • PaddleMIX ppdiffusers升级 StableDiffusion Pipeline。

完成步骤

要求:
paddle:paddle版本2.6.0
ppdiffusers:https://github.com/PaddlePaddle/PaddleMIX/tree/upgrade_ppdiffusers0240/ppdiffusers
paddlenlp: pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/wheels/paddlenlp-2.6.2-py3-none-any.whl

开始复现之前可对比已经转换后的2个pipeline文件,学习一下api的映射关系:
* paddle:https://github.com/PaddlePaddle/PaddleMIX/blob/upgrade_ppdiffusers0240/ppdiffusers/ppdiffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py
* torch:https://github.com/huggingface/diffusers/blob/v0.24.0/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py

注意点:
* 所有涉及到 device 的设置,均可以删除,因为paddle内部会设置全局的device。
* 所有 clip、bert、t5、xlm_roberta 均从 ppdiffusers.transformers 导入,而不是使用paddlenlp.transformers。
* enable_freeu 、disable_freeu、enable_vae_slicing、disable_vae_slicing、enable_vae_tiling、disable_vae_tiling均已经放到DiffusionPipeline里面了,因此无需在具体的pipeline里面添加该内容。
* 关于权重转换:
  * 现在基本上所有模型权重都与pytorch一样,只需要将特殊的层进行转置,比如:nn.Linear 层需要转置。
  * clip、bert、t5、xlm_roberta类型的模型以及ppdiffusers/models目录下的等模型可使用 xxxx.from_pretrained(本地torch文件目录, from_diffusers=True) 进行权重转换。
  1. pipeline_stable_diffusion_img2img参考代码单测代码
  • 添加 pipeline_stable_diffusion_img2img.py
  • 添加对应的单测,使单测通过
  • 相同配置下,出图结果对齐 diffusers 0.24.0
  1. pipeline_stable_diffusion_inpaint参考代码单测代码
  • 添加 pipeline_stable_diffusion_inpaint.py
  • 添加对应的单测,使单测通过
  • 相同配置下,出图结果对齐 diffusers 0.24.0
  1. pipeline_stable_diffusion_inpaint_legacy参考代码单测代码
  • 添加 pipeline_stable_diffusion_inpaint_legacy.py
  • 添加对应的单测,使单测通过
  • 相同配置下,出图结果对齐 diffusers 0.24.0
  1. pipeline_stable_diffusion单测代码
  • 添加pipeline_stable_diffusion.py 的单测,使单测通过
  • 相同配置下,出图结果对齐 diffusers 0.24.0

提交内容:

  1. 提交到pipeline目录单测目录
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
HappyOpenSource Pro 快乐开源issue与PR,更具挑战的任务
Projects
None yet
Development

No branches or pull requests

1 participant