Current use of execution providers is suboptimal

[DavidGOrtega]

Currently the library only operates with the WASM backend even in nodejs. However the better approach should be use node bindings in node and add the webgl EP to automatically fallback to wasm if webgl is not supported by the model.

Currenly in code:

let session = await InferenceSession.create(buffer, {
  // executionProviders: ["webgl"]
  executionProviders: ["wasm"]
});

Hack to at least improve in the web

let session;

try {
    session = await InferenceSession.create(buffer, {
        executionProviders: ['webgl', 'wasm']
    });
} catch (err) {
    session = await InferenceSession.create(buffer, {
        executionProviders: ['wasm']
    });
}

I can prepare a PR to improve the execution in node and web 😃

[DavidGOrtega]

related to #22 and #20

[xenova]

Hi! Yes, please do make a PR :) any contributions are most welcome.

I just had a question about webgl support for some of the models. When initially testing with the webgl backend, I kept getting errors saying that certain operations weren't supported. It might have been because the models are quantised (8-bit), but either way, I wasn't able to get it working reliably enough.

Do you have any suggestions?

[DavidGOrtega]

Hi! Yes, please do make a PR :) any contributions are most welcome.

Good! Let's do it!

Do you have any suggestions?

m open sourcing a project called aiui that transforms a model into an app, My first support was ONNX and it also happened to me. Not every layer is supported in webgl. It will depend on the model, but at least for the supported ones the speed will be very good.

Let me check what your script is doing.