How to create a new onnx TTS model like mms-tts-eng

[elloza]

Question

First of all, congratulations on such a great library!

I would like to ask for your guidance and assistance in creating a new onnx model similar to the following one:

https://huggingface.co/Xenova/mms-tts-eng/tree/main

…but for the Malagasy language:

https://huggingface.co/facebook/mms-tts-mlg

Could you provide me with some advice on how to create that model?

Thank you so much.

[elloza]

Hello again,

I managed to convert the model using the script provided in the documentation and this thread:
#626

I have finally published it here:
https://huggingface.co/elloza/mms-tts-mlg-onnx

However, I am unable to get the model to work in a basic browser example (nor any of the available models like Xenova/mms-tts-eng), as they return a 404 error when trying to fetch a preprocessor_config.json.

https://huggingface.co/Xenova/mms-tts-fra/resolve/main/preprocessor_config.json 404 (Not Found)

If you could guide me in any way or provide a basic example that uses this model in the browser, I would greatly appreciate it.

Best regards,

[neonwatty]

I'm not sure why you're your browser is pulling

https://huggingface.co/Xenova/mms-tts-fra/resolve/main/preprocessor_config.json

As the url

https://huggingface.co/Xenova/mms-tts-fra/resolve/main/

does not exist.

I converted the model you point to above and didn't have an issue running the baseline test below - does this work for you?

import { pipeline } from "@xenova/transformers";
import wavefile from "wavefile";
import fs from "fs";

// Create a text-to-speech pipeline
const synthesizer = await pipeline(
  "text-to-speech",
  "neonwatty/mms-tts-mlg-onnx",
  {
    quantized: false,
  }
);

// Generate speech
const output = await synthesizer("Salama");

const wav = new wavefile.WaveFile();
wav.fromScratch(1, output.sampling_rate, "32f", output.audio);
fs.writeFileSync("out.wav", wav.toBuffer());

[elloza]

The URL is not correct and I had left it there from other tests.

I have tested with that code and it seems to work without problems now.

Thank you very much for the help!

Regards,

[neonwatty]

That's great!

[elloza]

If anyone is looking for the final full code of the PoC I was making I'll leave it here and the demo:

https://elloza.com/malagasy-tts/
https://github.com/elloza/malagasy-tts

Currently one of my students is working on this project as a browser extension (I'll share the link when he finishes)

Thanks so much for the help again!