Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ControlNet #585

Merged
merged 32 commits into from
Apr 2, 2023
Merged

Add ControlNet #585

merged 32 commits into from
Apr 2, 2023

Conversation

carson-katri
Copy link
Owner

This adds support for ControlNet.

A ControlNet model can be specified in the "Source Image" panel when using the "Modify" action.

Screenshot 2023-03-02 at 5 41 02 PM

ControlNet models can be downloaded via the add-on preferences, and will show up as a "ControlNet" type.

Screenshot 2023-03-02 at 5 41 54 PM

Note The diffusers cache directory changed, and it did not properly migrate my models. I had to move some folders around manually after updating to main.

@carson-katri
Copy link
Owner Author

I think this has a good minimum set of nodes to be useful, especially for more complex tasks like multi-ControlNet. Other nodes for upscaling, more ControlNet annotations, etc. could be added in separate PRs so this doesn't get ridiculously large.

I'll document this in a separate PR as well, along with some example scene files.
Screenshot_2023-03-16_at_11 00 22_PM
vulcan
space
Screenshot_2023-03-14_at_12 37 55_AM
darksuperman

@carson-katri carson-katri linked an issue Mar 25, 2023 that may be closed by this pull request
@carson-katri carson-katri added this to the v0.2.0 milestone Mar 25, 2023
Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran into quite a few issues though I think most are only caused by platform differences.

Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Download progress appears a little glitchy due to it downloading all files at the same time instead of sequentially, but that could be cleaned up in another PR.

And I missed one fix in the depth render.

Otherwise, LGTM

@carson-katri carson-katri merged commit 4062851 into main Apr 2, 2023
@carson-katri carson-katri deleted the controlnet branch April 2, 2023 14:42
@bsmith1310
Copy link

Wow! Thanks @carson-katri, and @NullSenseStudio!

This is exactly what I was trying to find and I see this was merged just hours ago. I'm going to try to set this up on a fresh environment, can't wait to give this a try!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ControlNet support for texture projection
3 participants