We need more ready-to-use ONNX and/or TorchScript versions of populars pre-trained models

[JeeDevUser]

Hello everyone,

I think it would be great if we had some prepared and read-to use ONNX or TorchScript versions of popular models, like for example facebook/mbart-large-50-many-to-many-mmt and similar

I think that this is something that would greatly contribute to the popularity of this tool. The big obstacle facing Java developers is precisely how to get those versions of popular models. I would do something like this myself, but I find a lot of big problems that I can't overcome:

https://discuss.huggingface.co/t/how-to-export-facebook-mbart-large-50-many-to-many-mmt-to-torchscript-format/131717

https://discuss.huggingface.co/t/how-to-export-facebook-mbart-large-50-many-to-many-mmt-to-onnx-format/131852/1

I'm on the verge of giving up on using this Java tool. What do you think?

[frankfliu]

@JeeDevUser

Currently, we have not implemented general purpose Translator for text generation tasks. TextGeneration tasks are quite complicated, it not a single forward pass. We have to implement auto-regression search for text generation tasks. We only have a few examples and doesn't really work for all text generation models.