DirectML Execution

[NullSenseStudio]

Allows using non-NVIDIA GPU's on Windows. Currently only implemented for prompt_to_image. Doesn't support half precision so most users will likely need to make use of attention slicing and/or sequential CPU offload.

This is made as an alternative to ONNX, which has less models available, doesn't support memory optimizations like attention slicing or sequential CPU offload, and runs 3-4 times slower (on my machine at least).

I have encountered one odd bug where generating would start to only produce full white images, which requires releasing the generator to make it go away for a time.

[NullSenseStudio]

Now working for other actions. Depth_to_image doesn't follow the depth map at all, but can still generate rather abstract images. Seamless axes are broken and will cause the final image to be mostly a single color.

[carson-katri]

I added a workflow_dispatch trigger so you can manually create builds for AMD users to test.

https://github.com/carson-katri/dream-textures/actions/runs/3867749838

[carson-katri]

Started another run: https://github.com/carson-katri/dream-textures/actions/runs/3868659385

[NullSenseStudio]

I'm going to attempt to make half precision be mostly possible. There's not as much compatibility as there is with float32 but there might be some meaningful memory saving if I cast between float16 and float32 where necessary. After I'd like to see if there's anyone in the discord wanting to test it.

[faxlab3d]

Hello, can you please give some sort of info on how to test this on my AMD 5700xt? Do I need to build this branch or something? Thank you very much

[NullSenseStudio]

You can open the link that Carson provided then scroll down into Artifacts and download dream_textures-windows-directml. I believe the zip file you get contains another zip file so you'll have to extract that then install the extracted zip into Blender.

[NullSenseStudio]

Half precision was a great success! It doesn't give the same image with the same seed and prompt as full precision like CUDA can with half precision enabled or not, but I'm not going to worry about that.

I also believe I've fixed the white image bug, as long as there isn't anything else that causes it.

New run: https://github.com/carson-katri/dream-textures/actions/runs/3888885287

[NullSenseStudio]

New run with fixed model download: https://github.com/carson-katri/dream-textures/actions/runs/3894477733

[NullSenseStudio]

Very pleased with the new 0.1.13.1.dev230119 version. Eliminates most of the patches and gives further performance improvements. Used to need around 36 seconds in the denoising loop (25 steps) and now that's down around 22 seconds. Near 39% time savings on a GTX 1070, hopefully there'll be similar gains on AMD cards.

I've modified model handling so that the frontend model id only uses the model's name rather than the full path, this should prevent issues caused from having special characters in account names. I haven't been able to replicate the bug so I'm not entirely sure it'll work.

I've also modified model revision selection to have better preference towards the main and fp16 revisions when the preferred one isn't found. I was having issues with it selecting the onnx revision that I use to compare against.

[NullSenseStudio]

Let me know if you have any issues with the changes or that it somehow causes bugs on macOS. I don't think more user testing is needed before release, but maybe it should be tested due to the version change.

[carson-katri]

Is this ready for merge?

[NullSenseStudio]

Yes, unless if you want to do more user testing before release.

[carson-katri]

I think more testing can wait for the next set of pre-release builds.

[Mhowser]

Thanks so much for doing this! Will full precision eventually be possible in the future for AMD GPUs?