[Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU #3105

yingjie-han · 2023-04-14T08:18:44Z

This diffusion pipeline aims to speed up the inference of Stable Diffusion on Intel Xeon CPUs on Linux. It can get 1.5 times performance acceleration with BFloat16 on fourth generation of Intel Xeon CPUs, code-named Sapphire Rapids.
It is recommended to run on Pytorch/IPEX v2.0 to get the best performance boost.
-For Pytorch/IPEX v2.0, it benefits from MHA optimization with Flash Attention and TorchScript mode optimization in IPEX.
-For Pytorch/IPEX v1.13, it benefits from TorchScript mode optimization in IPEX.
Following tables show the test result on Intel® Xeon® Platinum 8480 Processor (56cores):

HuggingFaceDocBuilderDev · 2023-04-14T08:23:56Z

The documentation is not available anymore as the PR was closed or merged.

yingjie-han · 2023-04-18T08:12:23Z

Hi,patrickvonplaten, could you help to review it? Thanks!

github-actions · 2023-05-14T15:02:49Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

yingjie-han · 2023-05-16T07:27:14Z

Hi, @williamberman @patrickvonplaten is there any problem of this PR? Could you help me to review it ? This PR has been submitted for some time, but there has been no review. I don't know what I can do to move it on.

yingjie-han · 2023-05-16T07:48:05Z

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

bumping

pcuenca

It's hard for me to understand the benefits of this optimization given that the test results are performed on a Xeon Platinum CPU. I'd suggest to explain the compatibility and limitations of this method:

Does this work on any Intel CPU? Or is it intended for server hardware? Does it work on Windows?
What versions of PyTorch does it require?
How does performance compare against PyTorch 2 (with and without torch.compile())?

examples/community/README.md

examples/community/stable_diffusion_ipex.py

patrickvonplaten

@pcuenca feel free to merge if things look good to you

williamberman

Pedro's suggestions make sense, happy to merge after they've been addressed :) Sorry for the delay here @yingjie-han

Co-authored-by: Pedro Cuenca <[email protected]>

yingjie-han · 2023-05-19T07:00:51Z

It's hard for me to understand the benefits of this optimization given that the test results are performed on a Xeon Platinum CPU. I'd suggest to explain the compatibility and limitations of this method:

Does this work on any Intel CPU? Or is it intended for server hardware? Does it work on Windows?

What versions of PyTorch does it require?

How does performance compare against PyTorch 2 (with and without torch.compile())?

pcuenca Thanks very much for your review and suggestions. They make sense.
Here are the answers to your questions：
• Does this work on any Intel CPU? Or is it intended for server hardware? Does it work on Windows?
-- This pipeline is intended for Intel Xeon CPUs on Linux, not only Platinum Xeon. For FP32, it is supposed to benefit for every generation of Xeons (e.g., Skylake / Cascade Lake / IceLake), while BF16 works for CooperLake and Sapphire Rapids. Since it relies on Intel Extension for PyTorch, this pipeline only supports server CPUs and it does not work on Windows.
• What versions of PyTorch does it require?
It is recommended to use Pytorch/IPEX2.0 to get the best performance boost of flash attention optimization. It can also work on Pytorch/IPEX 1.13.
• How does performance compare against PyTorch 2 (with and without torch.compile())?
PyTorch 2.0 compile() is not support BF16 data type. For FP32, this ipex optimized pipeline get better performance than Pytorch2.0 compile(). I updated the test data in the above table, please check it.

pcuenca

Thanks a lot for iterating here! 🙌

I think the purpose of the pipeline is clearer now. I just suggested a couple of minor text modifications and then we are ready to merge.

Awesome contribution @yingjie-han, thanks a lot for your patience!

examples/community/README.md

Co-authored-by: Pedro Cuenca <[email protected]>

yingjie-han · 2023-05-23T07:45:08Z

pcuenca Thanks a lot for your review and suggestions. The modifications are committed. It's ready to merge.

pcuenca · 2023-05-23T08:55:09Z

Fixed the conflicts, merging now. Thanks again, @yingjie-han!

… on CPU (huggingface#3105) * add stable_diffusion_ipex community pipeline * Update readme.md * reformat * reformat * Update examples/community/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/community/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/community/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Update examples/community/README.md Co-authored-by: Pedro Cuenca <[email protected]> * Apply suggestions from code review Co-authored-by: Pedro Cuenca <[email protected]> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Pedro Cuenca <[email protected]> * style --------- Co-authored-by: Pedro Cuenca <[email protected]>

yingjie-han added 2 commits April 14, 2023 23:43

add stable_diffusion_ipex community pipeline

6032a08

Update readme.md

ce14a43

yingjie-han marked this pull request as draft April 18, 2023 08:23

yingjie-han marked this pull request as ready for review April 18, 2023 08:24

yingjie-han added 2 commits April 18, 2023 23:20

reformat

2eab9c4

reformat

3c63137

yingjie-han mentioned this pull request Apr 24, 2023

[Community Pipelines] #841

Open

6 tasks

github-actions bot added the stale Issues that haven't received updates label May 14, 2023

yingjie-han changed the title ~~Add community pipeline to accelerate inference of stable diffusion by IPEX on Intel CPUs~~ [Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU May 16, 2023

yingjie-han closed this May 16, 2023

yingjie-han reopened this May 16, 2023

pcuenca removed the stale Issues that haven't received updates label May 16, 2023

pcuenca reviewed May 16, 2023

View reviewed changes

patrickvonplaten reviewed May 16, 2023

View reviewed changes

williamberman approved these changes May 17, 2023

View reviewed changes

yingjie-han and others added 5 commits May 18, 2023 09:32

Update examples/community/README.md

d047fed

Co-authored-by: Pedro Cuenca <[email protected]>

Update examples/community/README.md

4059ae0

Co-authored-by: Pedro Cuenca <[email protected]>

Update examples/community/README.md

d36aec2

Co-authored-by: Pedro Cuenca <[email protected]>

Update examples/community/README.md

5fa265d

Co-authored-by: Pedro Cuenca <[email protected]>

Apply suggestions from code review

bd410a7

Co-authored-by: Pedro Cuenca <[email protected]>

yingjie-han added 3 commits May 19, 2023 15:33

Update README.md

ddbe4e1

Modified according to review comment

a0238d5

Update README.md

ed81aa2

pcuenca approved these changes May 23, 2023

View reviewed changes

Apply suggestions from code review

e7d54a1

Co-authored-by: Pedro Cuenca <[email protected]>

pcuenca added 2 commits May 23, 2023 09:55

Merge remote-tracking branch 'upstream/main' into ipex_pipeline

f88cece

style

5091e6f

pcuenca merged commit edc6505 into huggingface:main May 23, 2023

yingjie-han deleted the ipex_pipeline branch June 6, 2023 07:32

yingjie-han mentioned this pull request Jun 12, 2023

Using IPEX community pipeline in stable-diffusion-inference-intel.md huggingface/blog#1210

Open

ustcuna mentioned this pull request Jan 23, 2024

[Community Pipelines]Accelerate inference of stable diffusion xl (SDXL) by IPEX on CPU #6683

Merged

ustcuna mentioned this pull request Jun 20, 2024

[Community Pipelines] Accelerate inference of AnimateDiff by IPEX on CPU #8643

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU #3105

[Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU #3105

yingjie-han commented Apr 14, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Apr 14, 2023 •

edited

Loading

yingjie-han commented Apr 18, 2023 •

edited

Loading

github-actions bot commented May 14, 2023

yingjie-han commented May 16, 2023

yingjie-han commented May 16, 2023

pcuenca left a comment

patrickvonplaten left a comment

williamberman left a comment

yingjie-han commented May 19, 2023

pcuenca left a comment

yingjie-han commented May 23, 2023

pcuenca commented May 23, 2023

[Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU #3105

[Community Pipelines]Accelerate inference of stable diffusion by IPEX on CPU #3105

Conversation

yingjie-han commented Apr 14, 2023 • edited Loading

HuggingFaceDocBuilderDev commented Apr 14, 2023 • edited Loading

yingjie-han commented Apr 18, 2023 • edited Loading

github-actions bot commented May 14, 2023

yingjie-han commented May 16, 2023

yingjie-han commented May 16, 2023

pcuenca left a comment

Choose a reason for hiding this comment

patrickvonplaten left a comment

Choose a reason for hiding this comment

williamberman left a comment

Choose a reason for hiding this comment

yingjie-han commented May 19, 2023

pcuenca left a comment

Choose a reason for hiding this comment

yingjie-han commented May 23, 2023

pcuenca commented May 23, 2023

yingjie-han commented Apr 14, 2023 •

edited

Loading

HuggingFaceDocBuilderDev commented Apr 14, 2023 •

edited

Loading

yingjie-han commented Apr 18, 2023 •

edited

Loading