Type Error when converting facebook/mms-tts-eng to ONNX

[Bowen7]

System Info

macOS 13.5.1, Apple M1 Max
Python 3.11.8

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I know there is already a Xenova/mms-tts-eng on HF, but I want to test the process of converting.

Use the conversion script and then receive the following error:

File "/Users/zhangwenxiang/Documents/code/transformers.js/scripts/convert.py", line 426, in main
main_export(**export_kwargs)
File "/Users/zhangwenxiang/.pyenv/versions/3.11.8/lib/python3.11/site-packages/optimum/exporters/onnx/__main__.py", line 540, in main_export
_, onnx_outputs = export_models(
                  ^^^^^^^^^^^^^^
File "/Users/zhangwenxiang/.pyenv/versions/3.11.8/lib/python3.11/site-packages/optimum/exporters/onnx/convert.py", line 753, in export_models
export(
File "/Users/zhangwenxiang/.pyenv/versions/3.11.8/lib/python3.11/site-packages/optimum/exporters/onnx/convert.py", line 884, in export
config.fix_dynamic_axes(output, device=device, input_shapes=input_shapes, dtype=dtype)
File "/Users/zhangwenxiang/.pyenv/versions/3.11.8/lib/python3.11/site-packages/optimum/exporters/onnx/base.py", line 305, in fix_dynamic_axes
session = InferenceSession(model_path.as_posix(), providers=providers, sess_options=session_options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/zhangwenxiang/.pyenv/versions/3.11.8/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 383, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/Users/zhangwenxiang/.pyenv/versions/3.11.8/lib/python3.11/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 424, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from models/facebook/mms-tts-eng/model.onnx failed:Type Error: Type parameter (T) of Optype (Concat) bound to different types (tensor(int64) and tensor(float) in node (/text_encoder/encoder/layers.0/attention/Concat_23).

Reproduction

I installed Optimum from git+https://github.com/huggingface/optimum.git@add-onnx-vits

Then

python -m scripts.convert --quantize --model_id facebook/mms-tts-eng --task text-to-waveform --skip_validation

[Bowen7]

I missed the huggingface/transformers#28141.

I successfully exported ONNX with this commit.