Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] New Model Type: torch-kolors #11

Open
iwr-redmond opened this issue Jan 23, 2025 · 2 comments
Open

[Feature Request] New Model Type: torch-kolors #11

iwr-redmond opened this issue Jan 23, 2025 · 2 comments

Comments

@iwr-redmond
Copy link

iwr-redmond commented Jan 23, 2025

Kwai Kolors is a modified version of SDXL that adds a ChatGLM-based encoder and is customized for Asian users.

There is a basic pipeline in diffusers (KolorsPipeline), and additional diffusers-based code for Kolors-specific inpainting, ControlNets, and IP Adaptors lives here.

@Teriks
Copy link
Owner

Teriks commented Jan 23, 2025

Hmm, I will take a look

I think diffusers only has this pipeline packaged up with txt2img and img2img support

I would probably need to vendor (copy) their repo into my source tree with their license since there is no pypi packaging

Noticing a trend with lack of packaging in these AI projects :)

There are a lot of them like this

@iwr-redmond
Copy link
Author

iwr-redmond commented Jan 23, 2025

I would probably need to vendor (copy) their repo into my source tree with their license since there is no pypi packaging

Agreed. It looks like the license is compatible with vendored inclusion.

EDIT: There is a third-party packaged version, kolors-sdxl (repo), that might be better than a poke with a sharp stick.

Teriks added a commit that referenced this issue Jan 29, 2025
heavily modify original kolors code to add pipelines
currently not supported by diffusers, inpaint, and
(img2img + controlnet)

synthesize currently non-existant (txt2img + controlnet) and
(inpaint + controlnet) pipelines for completeness

add support for cpu offloading (vram optimizations)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants