Skip to content

Commit

Permalink
Document new external ONNX model and linalg operator test suites. (#1…
Browse files Browse the repository at this point in the history
  • Loading branch information
ScottTodd authored and Elias Joseph committed Oct 31, 2024
1 parent 26c616b commit 474a967
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ if crashes, bugs, numerical issues, etc. can be reproduced at that scale.
Some existing test suites can be found at these locations:

* <https://github.com/iree-org/iree/tree/main/tests/e2e>
* <https://github.com/nod-ai/SHARK-TestSuite/tree/main/iree_tests/onnx/node/generated>
* <https://github.com/iree-org/iree-test-suites>
* <https://github.com/nod-ai/SHARK-TestSuite/tree/main/e2eshark/onnx/operators>
* <https://github.com/nod-ai/SHARK-TestSuite/tree/main/e2eshark/pytorch/operators>
* <https://github.com/openxla/stablehlo/tree/main/stablehlo/tests/interpret>
21 changes: 21 additions & 0 deletions docs/website/docs/developers/general/testing-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,18 @@ repository also contains tests for many machine learning models. Some of these
tests are planned to be migrated into
[iree-org/iree-test-suites](https://github.com/iree-org/iree-test-suites).

### linalg operator tests

Tests for operators in the MLIR linalg dialect like `matmul`, and `convolution`
are being migrated from folders like
[`tests/e2e/matmul/`](https://github.com/iree-org/iree/tree/main/tests/e2e/matmul)
in the
[iree-org/iree](https://github.com/iree-org/iree) repository to
[`linalg_ops/`](https://github.com/iree-org/iree-test-suites/tree/main/linalg_ops)
in the
[iree-org/iree-test-suites](https://github.com/iree-org/iree-test-suites)
repository.

### ONNX operator tests

Tests for individual ONNX operators are included at
Expand Down Expand Up @@ -483,3 +495,12 @@ The workflow job that failed should then upload a new config file as an
committed:

![image](https://github.com/user-attachments/assets/b5dbdcb4-4c0a-4ff2-adc6-9021614179b2)

### ONNX model tests

Tests for ONNX models are included at
[`onnx_models/`](https://github.com/iree-org/iree-test-suites/tree/main/onnx_models)
in the
[iree-org/iree-test-suites](https://github.com/iree-org/iree-test-suites)
repository. These tests use models from the upstream
[onnx/models](https://github.com/onnx/models) repository.
3 changes: 2 additions & 1 deletion docs/website/docs/guides/ml-frameworks/onnx.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,8 @@ graph LR
| Code samples | |
| -- | -- |
Generated op tests | [iree-test-suites `onnx_ops`](https://github.com/iree-org/iree-test-suites/tree/main/onnx_ops)
Curated op and model tests | [SHARK-TestSuite `e2eshark/onnx`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/e2eshark/onnx)
Public model tests | [iree-test-suites `onnx_models`](https://github.com/iree-org/iree-test-suites/tree/main/onnx_models)
Curated op and model tests | SHARK-TestSuite [`e2eshark/onnx`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/e2eshark/onnx) and [`alt_e2eshark/onnx_tests`](https://github.com/nod-ai/SHARK-TestSuite/tree/main/alt_e2eshark/onnx_tests)
Importer tests | [torch-mlir `test/python/onnx_importer`](https://github.com/llvm/torch-mlir/tree/main/test/python/onnx_importer)
## :octicons-question-16: Troubleshooting
Expand Down

0 comments on commit 474a967

Please sign in to comment.