[js/webgpu] Support uniforms for conv, conv transpose, conv grouped #18753

axinging · 2023-12-08T00:24:18Z

No description provided.

axinging · 2023-12-12T07:52:14Z

guschmue · 2023-12-13T17:04:30Z

/azp run ONNX Runtime Web CI Pipeline

guschmue · 2023-12-13T17:04:37Z

/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline

azure-pipelines · 2023-12-13T17:04:42Z

Azure Pipelines successfully started running 1 pipeline(s).

guschmue · 2023-12-13T17:04:44Z

/azp run Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,Windows x64 QNN CI Pipeline

azure-pipelines · 2023-12-13T17:04:49Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

azure-pipelines · 2023-12-13T17:04:59Z

Azure Pipelines successfully started running 1 pipeline(s).

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv2d_mm_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv_backprop_mm_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv_backprop_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/conv-grouped.ts

axinging

@qjia7, PTAL

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv2d_mm_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv_backprop_mm_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv_backprop_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/conv-grouped.ts

js/web/lib/wasm/jsep/webgpu/ops/matmul.ts

guschmue · 2023-12-14T21:06:18Z

/azp run ONNX Runtime Web CI Pipeline

guschmue · 2023-12-14T21:06:25Z

/azp run Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline

azure-pipelines · 2023-12-14T21:06:30Z

Azure Pipelines successfully started running 1 pipeline(s).

guschmue · 2023-12-14T21:06:31Z

/azp run Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,Windows x64 QNN CI Pipeline

azure-pipelines · 2023-12-14T21:06:37Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

azure-pipelines · 2023-12-14T21:06:45Z

Azure Pipelines successfully started running 1 pipeline(s).

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv2d_mm_webgpu.ts

js/web/lib/wasm/jsep/webgpu/ops/fuse-utils.ts

js/web/lib/wasm/jsep/webgpu/ops/matmul.ts

azure-pipelines · 2024-01-16T17:35:10Z

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

guschmue · 2024-01-16T17:35:14Z

/azp run Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,Windows x64 QNN CI Pipeline

azure-pipelines · 2024-01-16T17:35:28Z

Azure Pipelines successfully started running 1 pipeline(s).

guschmue · 2024-01-17T00:02:11Z

need to run
npm run format

axinging · 2024-01-17T00:15:19Z

need to run npm run format

Thanks @guschmue. The format issue is caused by file js/web/lib/wasm/jsep/webgpu/ops/layer-norm.ts, updated, PTAL

diff --git a/js/web/lib/wasm/jsep/webgpu/ops/layer-norm.ts b/js/web/lib/wasm/jsep/webgpu/ops/layer-norm.ts
index 3c9f6ce71b..97202e9ab7 100644
--- a/js/web/lib/wasm/jsep/webgpu/ops/layer-norm.ts
+++ b/js/web/lib/wasm/jsep/webgpu/ops/layer-norm.ts
@@ -94,7 +94,8 @@ const createLayerNormProgramInfo =
     }
     let mean = ${sumVector('meanVector', components)} / uniforms.norm_size;
     let invStdDev =
-        inverseSqrt(${sumVector('meanSquareVector', components)} / uniforms.norm_size - mean * mean + uniforms.epsilon);
+        inverseSqrt(${
+            sumVector('meanSquareVector', components)} / uniforms.norm_size - mean * mean + uniforms.epsilon);

     for (var j: u32 = 0; j < uniforms.norm_size_vectorized; j++) {
       let f32input = ${castToF32(dataType, components, 'x[j + offset]')};

fs-eire · 2024-01-17T07:18:40Z

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

fs-eire · 2024-01-17T07:18:42Z

/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-python-checks-ci-pipeline,onnxruntime-binary-size-checks-ci-pipeline,Android CI Pipeline

fs-eire · 2024-01-17T07:18:43Z

/azp run iOS CI Pipeline,ONNX Runtime React Native CI Pipeline

azure-pipelines · 2024-01-17T07:18:55Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2024-01-17T07:18:55Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2024-01-17T07:18:56Z

Azure Pipelines successfully started running 1 pipeline(s).

fs-eire · 2024-01-18T01:51:42Z

/azp run Windows ARM64 QNN CI Pipeline,Windows x64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,ONNX Runtime Web CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline

fs-eire · 2024-01-18T01:51:43Z

/azp run Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline,orttraining-amd-gpu-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed,onnxruntime-python-checks-ci-pipeline,onnxruntime-binary-size-checks-ci-pipeline,Android CI Pipeline

fs-eire · 2024-01-18T01:51:45Z

/azp run iOS CI Pipeline,ONNX Runtime React Native CI Pipeline

azure-pipelines · 2024-01-18T01:51:56Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2024-01-18T01:51:57Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2024-01-18T01:51:57Z

Azure Pipelines successfully started running 1 pipeline(s).

satyajandhyala · 2024-01-24T20:56:36Z

Maybe we could merge this PR and revisit this code to address @qjia7 has raised and clean-up.

axinging · 2024-01-25T04:12:50Z

Maybe we could merge this PR and revisit this code to address @qjia7 has raised and clean-up.

Sure.

…18753)

### Description This PR is a preview of cherry-picks for ort-web to `rel-1.17.3` based on `rel-1.17.2`. <details> <summary>Changes of ort-web to cherry-pick</summary> The following commits are from main branch. `o` stands for pick, and `x` stands for skip. ``` o 2e0a388 [js/webgpu] Add HardSigmoid support (#19215) o d226e40 [js/webgpu] set query type in onRunStart (#19202) o 61610ff [js/webgpu] Add FusedConv clip test case (#18900) o a33b5bd [JS/WebGPU] Added Uniforms to SkipLayerNorm. (#18788) o 591f90c [js/webgpu] Fix issue of timestamp query (#19258) o 7252c6e [WebNN EP] Support WebNN async API with Asyncify (#19145) o 5b06505 [js/webgpu] Fix Tanh explosion (#19201) o 656ca66 [js/webgpu] Support uniforms for conv, conv transpose, conv grouped (#18753) o a3f0e24 [js/webgpu] Support f16 uniform (#19098) o 9e69606 fix f16 for attention, enable slice and flatten for more types (#19262) o 624b4e2 [js/webgpu] Remove enableShapesUniforms (#19279) o 90883a3 [js/webgpu] Add hardSigmoid activation for fusedConv (#19233) o 85cef0a [js/webgpu] Support capture and replay for jsep (#18989) o d73131c [js/webgpu] Use DataType as uniform cpu type (#19281) o dd1f6cc [js/webgpu] resolve codescan alert (#19343) o 3a2ab19 [js/webgpu] Refactor createTensorShapeVariables (#18883) o efc17e7 [js/webgpu] Fix the undefined push error (#19366) x 50806a7 [js/web] support external data in npm test (#19377) o ccbe264 [js/webgpu] Add LeakyRelu activation for fusedConv (#19369) o 5ff27ef [js/webgpu] support customop FastGelu (#19392) x 03be65e [js/web] fix types exports in package.json (#19458) o 06269a3 [js/webgpu] allow uint8 tensors for webgpu (#19545) o dfeda90 [JS/WebGPU] Add MatMulNBits (#19446) o 1b48054 [js/webgpu] Create Split indices helpers by rank, not by shape (#19554) o 3fe2c13 [js] small fix to workaround formatter (#19400) x 70567a4 [js/web] use ApiTensor insteadof onnxjs Tensor in TensorResultValidator (#19358) o 6e04e36 [js/common] upgrade tsc in common from 4.9.5 to 5.2.2 (#19317) o 58f4921 [js] changes to allow Float16Array if any polyfill is available (#19305) o 57d6819 [js/web] Fix fused-conv is not included in npm test (#19581) o ebd220b Misspelling in README.md (#19433) o 38c3432 Bump ip from 1.1.8 to 1.1.9 in /js/react_native (#19582) o fe82fcc [js/webgpu] Fix Conv2DTransposeMatMul f16 compilation failure (#19596) o 76a2a48 Bump ip from 1.1.8 to 1.1.9 in /js/react_native/e2e (#19583) o 29b1106 [node] Switch to setImmediate to avoid starving the Node.js event loop (#19610) o ae3d73c [JS/WebGPU] Fix Split and Where to handle corner cases. (#19613) o aec2389 [js/webgpu] allows a ProgramInfo's RunData to use zero sized output (#19614) o bb43a0f [js/webgpu] minor fixes to make tinyllama work (#19564) o 0edb035 [js/web] fix suite test list for zero sized tensor (#19638) o 3cb81cd [js/common] move 'env.wasm.trace' to 'env.trace' (#19617) o e30618d [js/webgpu] use Headless for webgpu test by default (#19702) o f06164e [js/web] transfer input buffer back to caller thread (#19677) x a788514 [js/web] dump debug logs for karma for diagnose purpose (#19785) o 24b72d2 [JS/WebGPU] Preserve zero size input tensor dims. (#19737) o 4538d31 [js/webgpu] expose a few properties in WebGPU API (#19857) o 53de2d8 [js/webgpu] Enable GroupedConvVectorize path (#19791) o ed250b8 [JS/WebGPU] Optimize MatMulNBits (#19852) x e771a76 [js/test] align web test runner flags with ort.env (#19790) o 79e50ae [js/web] rewrite backend resolve to allow multiple EPs (#19735) o acb0df2 Fix #19931 broken Get Started link of "ONNX Runtime JavaScript API" page (#19932) o b29849a [js/common] fix typedoc warnings (#19933) o afdab62 Bump follow-redirects from 1.15.4 to 1.15.6 in /js/web (#19949) o 28ad6c3 Bump follow-redirects from 1.15.4 to 1.15.6 in /js/node (#19951) o 7e0d424 accumulate in fp32 for Reduce* (#19868) o 4c6a6a3 [js/webgpu] Fix NAN caused by un-initialized buffer in instance-norm (#19387) o 01c7aaf [js/webgpu] allow setting env.webgpu.adapter (#19940) o c45cff6 [js/webgpu] fix maxpool / fp16 (#19981) ``` </details> <details> <summary>Cherry-pick commandlines</summary> ```sh git cherry-pick 2e0a388 git cherry-pick d226e40 git cherry-pick 61610ff git cherry-pick a33b5bd git cherry-pick 591f90c git cherry-pick 7252c6e git cherry-pick 5b06505 git cherry-pick 656ca66 git cherry-pick a3f0e24 git cherry-pick 9e69606 git cherry-pick 624b4e2 git cherry-pick 90883a3 git cherry-pick 85cef0a #<<<<< Note: conflicts git cherry-pick d73131c git cherry-pick dd1f6cc git cherry-pick 3a2ab19 git cherry-pick efc17e7 git cherry-pick ccbe264 git cherry-pick 5ff27ef git cherry-pick 06269a3 git cherry-pick dfeda90 git cherry-pick 1b48054 git cherry-pick 3fe2c13 git cherry-pick 6e04e36 git cherry-pick 58f4921 git cherry-pick 57d6819 git cherry-pick ebd220b git cherry-pick 38c3432 git cherry-pick fe82fcc git cherry-pick 76a2a48 git cherry-pick 29b1106 git cherry-pick ae3d73c git cherry-pick aec2389 git cherry-pick bb43a0f git cherry-pick 0edb035 git cherry-pick 3cb81cd git cherry-pick e30618d git cherry-pick f06164e git cherry-pick 24b72d2 git cherry-pick 4538d31 git cherry-pick 53de2d8 git cherry-pick ed250b8 git cherry-pick 79e50ae git cherry-pick acb0df2 git cherry-pick b29849a git cherry-pick afdab62 git cherry-pick 28ad6c3 git cherry-pick 7e0d424 git cherry-pick 4c6a6a3 git cherry-pick 01c7aaf git cherry-pick c45cff6 ``` </details> <details> <summary>Cherry-pick conflicts</summary> - 85cef0a #18989 this change is for enabling graph capture feature for JSEP, and it is done after ROCM EP enabled graph capture feature. However, the ROCM EP graph capture feature is not cherry-picked in rel-1.17.2. </details> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Jiajia Qin <[email protected]> Co-authored-by: Xu Xing <[email protected]> Co-authored-by: satyajandhyala <[email protected]> Co-authored-by: Yang Gu <[email protected]> Co-authored-by: Wanming Lin <[email protected]> Co-authored-by: Jiajie Hu <[email protected]> Co-authored-by: Guenther Schmuelling <[email protected]> Co-authored-by: Matttttt <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Segev Finer <[email protected]> Co-authored-by: Belem Zhang <[email protected]>

…icrosoft#18753)

axinging force-pushed the conv_convtrans_uniform branch from 7e77cc2 to 5cf669b Compare December 12, 2023 07:24

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped

17989b8

axinging force-pushed the conv_convtrans_uniform branch from 5cf669b to 17989b8 Compare December 12, 2023 07:51

axinging marked this pull request as ready for review December 12, 2023 07:52

guschmue added the ep:WebGPU ort-web webgpu provider label Dec 13, 2023

qjia7 reviewed Dec 14, 2023

View reviewed changes

axinging added 4 commits December 14, 2023 16:04

Fix comments and remove clipMax clipMin todo

ae9f9f6

Merge branch 'main' into conv_convtrans_uniform

bb8ddba

Format

dc0d998

Nit

8bb80cc

axinging commented Dec 14, 2023

View reviewed changes

qjia7 reviewed Dec 14, 2023

View reviewed changes

js/web/lib/wasm/jsep/webgpu/ops/matmul.ts Show resolved Hide resolved

axinging added 3 commits December 15, 2023 09:00

Fix MatmulNaive

bb47410

Merge branch 'main' into conv_convtrans_uniform

988bf4d

Impl updateUniformsFromActivation

a848409

qjia7 reviewed Dec 15, 2023

View reviewed changes

js/web/lib/wasm/jsep/webgpu/ops/3rd-party/conv2d_mm_webgpu.ts Outdated Show resolved Hide resolved

js/web/lib/wasm/jsep/webgpu/ops/fuse-utils.ts Outdated Show resolved Hide resolved

js/web/lib/wasm/jsep/webgpu/ops/matmul.ts Outdated Show resolved Hide resolved

Fix comments

603bbd7

axinging added 2 commits January 17, 2024 08:11

Format layer-norm

cd936f5

Merge branch 'main' into conv_convtrans_uniform

6ddeb03

axinging added 2 commits January 18, 2024 09:43

Workaround windows linux cr lf format issue when colum reaches 120

d3be951

Revert layer-norm

d7f3603

satyajandhyala self-assigned this Jan 19, 2024

satyajandhyala approved these changes Jan 24, 2024

View reviewed changes

guschmue approved these changes Jan 25, 2024

View reviewed changes

guschmue merged commit 656ca66 into microsoft:main Jan 25, 2024
44 checks passed

fs-eire pushed a commit that referenced this pull request Mar 15, 2024

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped (#…

f02accb

…18753)

siweic0 pushed a commit to siweic0/onnxruntime-web that referenced this pull request May 9, 2024

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped (m…

c9b6a9c

…icrosoft#18753)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped #18753

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped #18753

axinging commented Dec 8, 2023

axinging commented Dec 12, 2023

guschmue commented Dec 13, 2023

guschmue commented Dec 13, 2023

azure-pipelines bot commented Dec 13, 2023

guschmue commented Dec 13, 2023

azure-pipelines bot commented Dec 13, 2023

azure-pipelines bot commented Dec 13, 2023

axinging left a comment

guschmue commented Dec 14, 2023

guschmue commented Dec 14, 2023

azure-pipelines bot commented Dec 14, 2023

guschmue commented Dec 14, 2023

azure-pipelines bot commented Dec 14, 2023

azure-pipelines bot commented Dec 14, 2023

azure-pipelines bot commented Jan 16, 2024

guschmue commented Jan 16, 2024

azure-pipelines bot commented Jan 16, 2024

guschmue commented Jan 17, 2024

axinging commented Jan 17, 2024

fs-eire commented Jan 17, 2024

fs-eire commented Jan 17, 2024

fs-eire commented Jan 17, 2024

azure-pipelines bot commented Jan 17, 2024

azure-pipelines bot commented Jan 17, 2024

azure-pipelines bot commented Jan 17, 2024

fs-eire commented Jan 18, 2024

fs-eire commented Jan 18, 2024

fs-eire commented Jan 18, 2024

azure-pipelines bot commented Jan 18, 2024

azure-pipelines bot commented Jan 18, 2024

azure-pipelines bot commented Jan 18, 2024

satyajandhyala commented Jan 24, 2024

axinging commented Jan 25, 2024

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped #18753

[js/webgpu] Support uniforms for conv, conv transpose, conv grouped #18753

Conversation

axinging commented Dec 8, 2023

axinging commented Dec 12, 2023

guschmue commented Dec 13, 2023

guschmue commented Dec 13, 2023

azure-pipelines bot commented Dec 13, 2023

guschmue commented Dec 13, 2023

azure-pipelines bot commented Dec 13, 2023

azure-pipelines bot commented Dec 13, 2023

axinging left a comment

Choose a reason for hiding this comment

guschmue commented Dec 14, 2023

guschmue commented Dec 14, 2023

azure-pipelines bot commented Dec 14, 2023

guschmue commented Dec 14, 2023

azure-pipelines bot commented Dec 14, 2023

azure-pipelines bot commented Dec 14, 2023

azure-pipelines bot commented Jan 16, 2024

guschmue commented Jan 16, 2024

azure-pipelines bot commented Jan 16, 2024

guschmue commented Jan 17, 2024

axinging commented Jan 17, 2024

fs-eire commented Jan 17, 2024

fs-eire commented Jan 17, 2024

fs-eire commented Jan 17, 2024

azure-pipelines bot commented Jan 17, 2024

azure-pipelines bot commented Jan 17, 2024

azure-pipelines bot commented Jan 17, 2024

fs-eire commented Jan 18, 2024

fs-eire commented Jan 18, 2024

fs-eire commented Jan 18, 2024

azure-pipelines bot commented Jan 18, 2024

azure-pipelines bot commented Jan 18, 2024

azure-pipelines bot commented Jan 18, 2024

satyajandhyala commented Jan 24, 2024

axinging commented Jan 25, 2024