[DevTask] Port TencentARC/PhotoMaker

[huchenlei]

https://github.com/TencentARC/PhotoMaker released code. The mechanism is similar to ip-adapter with varitations on how input are preprocessed.

[2blackbar]

Yeah but it accepts multiple images, which is something that IP adapters in auto11 controlnet should also accept and issue for that already exists but looks like nobody cares

IMO this needs to finally be addressed cause we can only feed one image in controlnet currently and that limits ipadapters useage cause they can accept multiple images at once in batch,

In comfy thanks to feeding multiple images with batch we can create instant lora, i just give it my folder path and it takes all iamges from that path in comfy to send to ipadapter.
Something like this would be very nice in auto11
So... how about we give it a path to images, or just choose multiple images in auto11 controlnet section for preprocessors that support multiple images

[huchenlei]

Multiple IP-Adapters issue has been fixed. See #2472 @2blackbar. For now you just need to enable enough ControlNet units to handle multiple inputs.

Taking the whole directory probably need you work with API a bit.

[2blackbar]

Yes but that just 1 pic per controlnet which is not enough to do what comfyui does, the fix is appreciated tho cause i tried one pic per cnet and it wasnt working before , thanks.

[lhucklen]

Any word on support?

[hben35096]

https://github.com/InstantID/InstantID
This one has a similar function, and the results are amazing

[sdbds]

After testing,
InstantID>ipadapter face IDv2>PhotoMaker
https://github.com/InstantID/InstantID

InstantID specially train a controlnetxl for control space position in here
https://huggingface.co/InstantX/InstantID

[hben35096]

经过测试，InstantID>ipadapter face IDv2>PhotoMaker

So, would you consider adding InstantID to ControlNet? :D

[haofanwang]

It would be very appreciated if you can add InstantID.

[anilken]

We would be grateful for support InstantID

[lhucklen]

I fourth this! I would like to try out both
@huchenlei

[Paper99]

@huchenlei. Thanks for your work.
https://twitter.com/PurzBeats/status/1751781133241421867

[EriIaz]

Is PhotoMaker implementation still being developed?
I've got a tad better results with it and SDXL in Comfy as opposed to FaceID, assuming multiple references, and I can't run InstantID in A1111's UI at adequate speed because the current implementation only takes effect with a secondary ControlNet, which is insanely VRAM hungry. I am pretty sure InstantID can be optimized in this regard, making ControlNet optional and possibly, converting it into ControlLORA. But I assume it's a challenge, any low hanging fruit would be already implemented otherwise. A PhotoMaker implementation would provide a decent SDXL-compatible alternative for people with 12GB and even 10GB GPUs.

[huchenlei]

Photomaker supported in https://github.com/lllyasviel/stable-diffusion-webui-forge

[kigy1]

Photomaker supported in https://github.com/lllyasviel/stable-diffusion-webui-forge

thanks for your work, can you make it work in automatic1111 ?

[mykeehu]

Any news for Auto1111?
AssertionError: ip-adapter-photomaker-v1-sdxl not found in ipadapter presets. Please try manually pick preprocessor.