Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add CFG End Optimization #617

Merged
merged 11 commits into from
Apr 12, 2023
Merged

Add CFG End Optimization #617

merged 11 commits into from
Apr 12, 2023

Conversation

carson-katri
Copy link
Owner

@carson-katri carson-katri commented Mar 28, 2023 •
edited
Loading

This adds a new optimization that limits the number of steps classifier-free guidance is performed on.

Using a cfg_end value of 0.5 reduced inference time by around 22% in my tests.

As seen here: https://twitter.com/Birchlabs/status/1640033271512702977

1.0 (disabled) 0.5

ocean (cfg_end 1)

ocean (cfg_end 0 5)

Base automatically changed from optimizations to main April 7, 2023 17:05
NullSenseStudio
NullSenseStudio requested changes Apr 7, 2023
View reviewed changes
Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs some changes for batching.

Also should use the prompt embeds instead of negative embeds so that the produced image will be the same if either CFG scale is 1 or CFG end is 0.

@carson-katri carson-katri merged commit 83dc466 into main Apr 12, 2023
@carson-katri carson-katri deleted the cfg_end branch April 12, 2023 12:05
phi-line pushed a commit to phi-line/dream-textures that referenced this pull request Apr 25, 2023
@carson-katri @NullSenseStudio @phi-line
Add CFG End Optimization (carson-katri#617)
e1e998c 
* Add execution time below generate

* Remove placeholder

* Use cfg_end in prompt_to_image

* Use fewer decimals

* Use first embeddings instead of zero

* Extend to other pipelines

* Add min/max to cfg_end

* Update panel UI

* Simplify implementation

* Update for batching

Co-authored-by: NullSenseStudio <[email protected]>

* Update for ControlNet + inpaint

---------

Co-authored-by: NullSenseStudio <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NullSenseStudio NullSenseStudio NullSenseStudio requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@carson-katri @NullSenseStudio