question on guidance_scale

[Clement-Lelievre]

on SD* models (and possibly on Flux too, I couldn't find the place that does that in FLUX transformer's forward method), say SD1.5 for the sake of this example, the guidance is enforced by asking the UNET to make two noise predictions, not one: one without prompt conditioning and one with prompt conditioning, and then the actual prediction is a weighted sum using the guidance_scale as weight

This is visible here in diffusers.

I'm wondering if the following scenario is possible:

• the unconditioned prediction corresponds, by chance, to an image of a tree (or any other concept)

• the prompt is "a tree" and the conditioned prediction corresponds to an image of a tree too (the same concept)

• guidance_scale > 1

in other words, by chance, both predictions match the same concept.

in this case what is the meaning of noise_pred = noise_pred_uncond + self.guidance_scale * (noise_pred_text - noise_pred_uncond) ?

@sayakpaul @asomoza