-
Notifications
You must be signed in to change notification settings - Fork 7.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handling of HDR color spaces #6505
Comments
ModelSpec is the technically correct way to do this. There is nothing stopping you from adding the metadata elsewhere in the checkpoint though. |
Thank you, I have created an issue there: Stability-AI/ModelSpec#4 |
The easiest would most likely be to add a "HDR" key to the VAE so that ComfyUI could detect it as an HDR VAE and handle it correctly, either in the state dict or the safetensors metadata. Have you released this model and VAE anywhere? I do want to natively support different color spaces. |
@comfyanonymous I was thinking CICP would be a great and easy option, it is four values you can just copy as-is into PNG/AVIF/HEIF/JPEG XL and then you have support for all the most common color spaces and transfer curves including HDR. I plan on uploading it to Civitai, but I wanted to at least bring up the question on how to define the meta-data before making a proper release. I can share a link here if it can aid development. |
CICP sounds good. It's your choice if you put it in a tensor in the state dict or in the safetensors metadata. |
Since there have been no discussion on the ModelSpec issue, I have gone ahead and released the model with the CICP metadata: |
Feature Idea
I have created a model which can produce images in Rec. 2100 PQ to natively create HDR images. However for this to work well, two things are missing:
I think it would make a lot of sense to include some meta-data in the models. My primary motivation for creating this issue is to know how I could tag my models such that it could be potentially supported in the future, even if there is no interest to support HDR for now.
I see that there are several difficulties with supporting HDR:
And then there is the question on what part of the model should decide what the color space is. Both the UNET, Lora, and VAE can affect the final image. My approach ended up by fine-tuning a VAE to remap the SDR latents to Rec. 2100 PQ and then using a Lora to push it towards creating images outside the normal SDR range. (Only acting on the UNET made the results very finicky whenever the prompt affected colors and/or brightness.)
But with all that said, I personally think HDR support would be very exciting. Here are some examples:
https://civitai.com/images/52054640 (PNG 8-bit)
https://civitai.com/images/52208460 (PNG 8-bit)
https://civitai.com/images/51872956 (UltraHDR tweaked in Adobe Camera Raw)
Existing Solutions
No response
Other
No response
The text was updated successfully, but these errors were encountered: