Fix scheduler inference steps error with power of 3

[natolambert]

Discussing in #444.
More verification to do on the unclear number of iterations vs num_inference_steps

Solution inspired by this discussion in original SD repo, thanks @ryanirl.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Cool that looks good to me! Thanks a lot for the fix. @natolambert could you also add one test that shows that it can now be a power of three?

[natolambert]

Okay @patrickvonplaten, so this test should cover the test needed for now.
Note that the DDIMScheduler did not error out with powers of 3, but rather it had a number of iterations that did not match the expected behavior.

If we update DDIMScheduler to have the same alpha logic as PNDMScheduler, we should add the power of 3 test!

NOTE: I added a small fix where the DDIMSchedulerTest::test_inference_steps was not passing the time_step to check_over_forward like the other scheduler tests.

[pcuenca]

We need to verify this in the Flax implementation too; it probably happens there as well: https://github.com/patil-suraj/stable-diffusion-jax/blob/main/stable_diffusion_jax/scheduling_pndm.py#L103-L105

/cc @kashif

[patrickvonplaten]

PR looks great to me!

[natolambert]

I'll add a comment on the test explaining why it is there, then merge (otherwise it may be a little confusing to a reader). Great!

[anton-l]

LGTM, but there might be a case where we need to .round() for mathematical correctness, instead of just truncating the decimal with .astype(int). So watch out for those issues :)

[natolambert]

@anton-l that's a good point. I'm going to check with round, I like that better (it's also less opaque).