SDXL support

[scarbain]

It appears the new SDXL model will be released on friday (according to recents tweets and messages from Emad and JoePenna).
Are you planning on supporting it ? :)

[scarbain]

Any update on this ?
We might be able to train some ControlNets (we're specifically interested in the normal bae model) to help provide compute if necessary.

[bjornlarssen]

Also interested. I got access to SDXL, but I can only use it in Gradio with approx. two options. I can't even reuse the same seed, because the interface won't tell me what it is, not to mention, welp, anything else than "Generate." Would LOVE to use it with WebUI.

[dayunbao]

This article might be of interest, where it says this:

Meanwhile, his Stability AI colleague Alex Goodwin confided on Reddit that the team had been keen to implement a model that could run on A1111—a fan-favorite GUI among Stable Diffusion users—before the launch. That plan, it appears, will now have to be hastened. “We were hoping to, y'know, have time to implement things before launch,” Goodwin wrote, “but [I] guess it's gonna have to be rushed now.”

So in its current state, XL currently won't run in Automatic1111's web server, but the folks at Stability AI want to fix that. Hopefully there will be few changes required to get it working with ControlNet.

[catboxanon]

This should really be directed towards ControlNet itself and not this extension, as no ControlNet model for SDXL currently exists in the first place.
lllyasviel/ControlNet#468

[scarbain]

I agree but the author lllyasviel is way more active on this repo

[Alchete]

BTW-- mysterguitarman just confirmed that Stability's own ControlNet will unfortunately not be in this week's SD-XL release...

Not ControlNet on launch 😔
Our internal ControlNet checkpoints only work for the older checkpoints – not the one we're releasing.

https://www.reddit.com/r/StableDiffusion/comments/157ybqf/comment/jt9rm1r/?context=3

[dayunbao]

Is that referring to Stability AI's own implementation of a ControlNet? Not lllyasviel's ControlNet? If so, I wonder if lllyasviel will continue maintaining theirs.

[ostap667inbox]

Yeah, they announced something resembling ControlNet, it looks like there will be a merged model for different adapters. But then the enthusiasm died down a bit and there were no more promises about dates. They only wrote something about the fact that a lot of things would have to be redesigned.
In any case, I personally would very much like to see new models working with SDXL just for ControlNet. I wouldn't want to break the workflow into pieces.

[omaralvarez]

Same, who knows when they will release. Having ControlNet working would be great.

[al-swaiti]

any news ,

[underlines]

https://huggingface.co/SargeZT/controlnet-v1e-sdxl-depth

https://huggingface.co/SargeZT/sdxl-controlnet-seg

[aiXander]

This looks amazing! Can anybody post some boilerplate code of how to load + use SDXLv1.0 + Controlnet? I cant find it anywhere in the docs and all the subclassing makes it tricky to figure this out from the code

[dayunbao]

I've assumed that the architecture of SD XL changed so much that ControlNet itself (not the pre-processors and models) needed to be updated to work with it. Is that not the case? Is the only problem that the pre-processors and models need to be retrained?

[omaralvarez]

I think that your assumption is correct. For optimal performance it is probably necessary to modify ControlNet, but using it as is is probably better than nothing.

[aiXander]

The above models (linked by @underlines) look like they were trained for SDXL, no?
Example should go here:
https://github.com/abraham-ai/diffusers/blob/sdxl/src/diffusers/pipelines/controlnet/pipeline_controlnet_sd_xl.py#L58

[wandrzej]

The 'official' Stability models are all here
https://huggingface.co/stabilityai/control-lora

Would be great to know if ControlNet extension is getting an update soon, or it's Comfy and other UIs for now

[oliverban]

So does this work right away or needs to be implemented in a1111?

[brennanerbz]

+1

[Saittham]

+10000

[linsmalldragon]

+111111

[FurkanGozukara]

stability ai released trained controlnet models

any plan to make this extension compatible with them?

[IPv6]

Looks like SDXL ControlLoRAs are not composable... so user can not have several controlnet units active at the same time. Some docs suggest to prepare masks for each control net to ensure they don`t overlap (which is unusable approach imho)
Is that so?

[huchenlei]

See #1936

[lllyasviel]

SDXL support for SAI/HuggingFace/diffuser/community's control models is under constrcution.

#1952

[Blumfeld123]

Thanx

[suonnon]

Hi, does Controlnet support SDXL yet?

[sashaok123]

https://github.com/Mikubill/sd-webui-controlnet/tree/sdxl

[jasoncow007]

how is it going?

[AugmentedRealityCat]

@jasoncow007 In case you might have missed it, it was officially released earlier today.

[jasoncow007]

I tried yesterday, seems not as good as in comfyui

[MoonRide303]

@jasoncow007 Please be more specific - how did you do the comparison? Input image(s), parameters, workflow configuration in Comfy, etc. Did you apply steps from "Achieving Same Outputs with StabilityAI Official Results" section (discussion 2039) before comparing the results?