Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "Dream Textures" render pass to Cycles #229

Merged
merged 23 commits into from
Oct 6, 2022
Merged

Conversation

carson-katri
Copy link
Owner

This monkey patches a new render pass into cycles that can run stable diffusion each time a frame is rendered. It is especially useful for animations where you want to perform style transfer with stable diffusion.

@carson-katri carson-katri added the enhancement New feature or request label Oct 2, 2022
@carson-katri carson-katri marked this pull request as draft October 2, 2022 19:34
@carson-katri
Copy link
Owner Author

Memory consumption seems to be ok now. I'm not using 50+GB of swap after 50 frames anymore 😅

@carson-katri carson-katri changed the base branch from main to ui October 5, 2022 01:20
Base automatically changed from ui to main October 5, 2022 01:49
@carson-katri carson-katri marked this pull request as ready for review October 5, 2022 03:41
@carson-katri carson-katri added this to the v0.0.7 milestone Oct 5, 2022
@carson-katri
Copy link
Owner Author

carson-katri commented Oct 5, 2022

Running cycles and dream textures can use a lot of VRAM, so if needed cycles can be run on CPU instead. When running through dream textures a lot of path tracing samples typically aren’t needed anyways.

Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a couple of issues.

Copy link
Collaborator

@NullSenseStudio NullSenseStudio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks to be all good. Just one small suggestion.

Co-authored-by: NullSenseStudio <[email protected]>
@carson-katri carson-katri merged commit c459f2d into main Oct 6, 2022
@carson-katri carson-katri deleted the render-properties branch October 6, 2022 00:02
JasonHoku pushed a commit to JasonHoku/dream-textures that referenced this pull request Dec 20, 2022
* Add base RenderPropertiesPanel

* Working dream textures render pass

* Revised implementation (with memory leak)

* Fix step count

* Refactor into subpanels

* Remove Troubleshooting section

* Use file picker to copy weights instead of manual installation

* Replace Real-ESRGAN setup with ImportHelper

* Add soft_min/soft_max to simulate exclusive clamped values

* Add history management options and improved interface

* Add prompt import/export

* Fix seed export

* Update submodules to fix inpainting

* Updated to work with latest UI and internal changes

* Remove unused handler

* Remove completed TODO

* Allow render pass properties to be animated

* Update to latest submodules

* Fix step count target when init image is specified

* Add multiple of 64 check to size_x/size_y

* Add closest hint to error

Co-authored-by: NullSenseStudio <[email protected]>

Co-authored-by: NullSenseStudio <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants