ORT would be crashed while loading the specific INT4 model

[peterer0625]

Describe the issue

ORT would be crashed while loading the specific INT4 model.

We can observe the issue on DML EP and CPU EP.

Here are the crash dumps - https://www.dropbox.com/scl/fi/h3wvh3vkap83gmvuugebs/onnxruntime-T5-crash-dump.7z?rlkey=kq7tu3i87eplnjo9z232zro10&st=aq0i4hvi&dl=0

The issue is gone if we set session_options.graph_optimization_level to onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL.

To reproduce

1. Export SD3 t5 model to onnx model.
2. Do INT4 quantization for MATMUL by this way Quantize ONNX models | onnxruntime.
3. Inference the quantized model.

Urgency

No response

Platform

Windows

OS Version

26100

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

Quantization should be with the newest commit. Inference can be run with ORT-DML 1.19.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU, DirectML

Execution Provider Library Version

No response

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[klin2024]

We still need your support for this issue.