[Community Pipeline] MagicMix #1839
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
|
Cool! But why did you put it in the community pipeline instead of the internal pipeline? |
There was a problem hiding this comment.
Very cool @daspartho ! The PR looks good, just left some nits
examples/community/README.md
Outdated
| pipe = DiffusionPipeline.from_pretrained( | ||
| "CompVis/stable-diffusion-v1-4", | ||
| custom_pipeline="magic_mix", | ||
| scheduler = DDIMScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", clip_sample=False, set_alpha_to_one=False), |
There was a problem hiding this comment.
The scheduler can be loaded using
DDIMScheduler.from_pretrained("CompVis/stable-diffusion-v1-4", subfolder="scheduler")| pipe = DiffusionPipeline.from_pretrained( | ||
| "CompVis/stable-diffusion-v1-4", | ||
| custom_pipeline="magic_mix", |
There was a problem hiding this comment.
Also, maybe load the pipeline in fp16, by passing the torch_dtype argument, to make inference faster.
examples/community/magic_mix.py
Outdated
| prompt: str, | ||
| kmin: float = 0.3, | ||
| kmax: float = 0.6, | ||
| v: float = 0.5, |
There was a problem hiding this comment.
could we use a more descriptive name for this argument? One letter variables aren't informative
|
@patil-suraj made some changes :) |
The wyt @patil-suraj |
Community pipeline based on my implementation of the MagicMix: Semantic Mixing with Diffusion Models paper.
This Diffusion Pipeline allows for the semantic mixing of an image and a text prompt to create a new concept while preserving the spatial layout and geometry of the subject in the image.
Here are some examples I reproduced from the paper using my implementation-
Input Image:
Prompt: "Bed"
Output Image:
Input Image:
Prompt: "Family"
Output Image:
Input Image:
Prompt: "ice-cream"
Output Image:
Input Image:
Prompt: "Cake"
Output Image:
@patrickvonplaten could you please take a look at it, looking forward to any comments!
Thanks :)
Reference: #841