Add oneAPI device selector for xpu and some other changes. #6112
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I expect that there might be some opinions about 1.) and 2.) so I am open for anyone arguing for some detail changes or another way to implement if needs be. List of changes here include:
1.) Add in a
--oneapi-device-selector
that does something similar to--cuda-device
but for Intel oneAPI devices. This doesn't need to necessarily be limited to GPUs but I expect for the time being that it will effectively only do that. Documentation on how to use this can be found at https://github.com/intel/llvm/blob/sycl/sycl/doc/EnvironmentVariables.md#oneapi_device_selector2.) Per https://github.com/pytorch/pytorch/blob/v2.5.0/docs/source/notes/numerical_accuracy.rst#reduced-precision-reduction-for-fp16-and-bf16-in-scaled-dot-product-attention-sdpa of which pytorch/pytorch#135778 brought this up to my attention, the default behavior using SDPA was changed in Pytorch 2.5 to upcast by default to avoid numerical errors. Since the old behavior has been working fine with ComfyUI, set
torch.backends.cuda.allow_fp16_bf16_reduction_math_sdp
to default to true for ComfyUI if Pytorch 2.5 or up is detected.3.) Documentation changes for IPEX and noting one can install the mainline builds of Pytorch to get ComfyUI working on it with the caveat that most optimizations aren't there yet. It's if anything still a beta release.Defer to #6069 for documentation changes.