Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request to add NPU buttons for SAM and SD 1.5 #49

Open
indygit opened this issue Oct 31, 2024 · 4 comments
Open

Request to add NPU buttons for SAM and SD 1.5 #49

indygit opened this issue Oct 31, 2024 · 4 comments

Comments

@indygit
Copy link

indygit commented Oct 31, 2024

Hello, we now have NPUs that can run SAM and SD 1.5. Can you please add NPU buttons for them like the one for Whisper?

While testing SD 1.5, we found Safety Checker doesn't work on the NPU - would it be possible to add safetychecker=false to NPU's URL? I know you may not feel comfortable to disable Safety Checker, and if that is the case, would you still feel okay to add the NPU button for SD 1.5? We can tell the customer that they can manually add safetychecker=false to the URL. NPU driver will be fixed in the near future.

SAM works well. So please add the NPU button for it.

Best regards, Indy.

@indygit indygit changed the title Request to add NPU button for SAM and SD 1.5 Request to add NPU buttons for SAM and SD 1.5 Oct 31, 2024
@indygit
Copy link
Author

indygit commented Nov 8, 2024

SD 1.5 Safety Checker issue is fixed, I claim there is a bug in ARM64 CPU EP that when it runs Clip operator for constant folding it always outputs zero.

We fixed the issue by folding off 2 Clip operators in Safety Checker model, please huggingface.com for the new model.

Please add the NPU buttons, we are ready.

@huningxin
Copy link

@indygit , thanks!

SD 1.5 Safety Checker issue is fixed, I claim there is a bug in ARM64 CPU EP that when it runs Clip operator for constant folding it always outputs zero.

I understand WebNN doesn't use native CPU EP, could you elaborate the fix?

Please add the NPU buttons, we are ready.

I understand the two demos may not work on every NPU device or driver. How could we manage the difference? @ibelem @fdwr

@fdwr
Copy link
Collaborator

fdwr commented Nov 9, 2024

I understand WebNN doesn't use native CPU EP

Indy meant ORT's CPU EP used during the constant folding phase of model loading (microsoft/onnxruntime#21957).

How could we manage the difference

I think displaying some messaging to the user of supported known hardware configurations would help. May be time to resurrect this one: #32

@indygit
Copy link
Author

indygit commented Nov 9, 2024

Thanks Dwayne and Ningxin.

How could we manage the difference
I somehow remember seeing that Belem mentioned that we can allow the NPU buttons on ARM64 but not X64/X86.
I assume we can access the CPU architecture info in JavaScript.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants