[Model request] Helsinki-NLP/opus-mt-ru-en (marian) #63

eviltik · 2023-03-31T09:18:28Z

Sorry for this noob question, can somebody give me a kind of guideline to be able to convert and use

https://huggingface.co/Helsinki-NLP/opus-mt-ru-en/tree/main

thank you

eviltik · 2023-03-31T09:35:46Z

lol it's in the readme okeyyyyyyyyy

xenova · 2023-03-31T10:22:48Z

Haha no worries :)

However, the model type you are wanting to run is of type "marian", which currently isn't supported. Fortunately, it is very similar to some existing models, so, it should be quite simple to add. I'll reopen the issue, and then close it once I add support for it.

eviltik · 2023-03-31T10:41:50Z

Ha ! thank you !

I've tried

python3 scripts/convert.py --model_id Helsinki-NLP/opus-mt-ru-en --from_hub --quantize --task seq2seq-lm-with-past

But i don't really know what i am doing.

It failed with an error :

ValueError: Two model protos need to have the same outputs. But one has 26 outputs while the other has 14 outputs.

xenova · 2023-03-31T10:45:58Z

Make sure you're on the dev branch of https://github.com/huggingface/optimum - this was fixed in the past few days:

But even if you get the conversion working, since the model isn't supported yet, it will throw an error. No worries though! I'll get it supported :)

xenova · 2023-03-31T16:11:48Z

While I look into this, can you provide some sample code for running the model (and its expected output)?

eviltik · 2023-03-31T20:08:43Z

const { pipeline, env } = require('./src/transformers.js')

env.localURL = './models';
env.remoteModels = false;

async function go() {
  let pipe = await pipeline('translation', 'onnx/quantized/Helsinki-NLP/opus-mt-ru-en');
  let summary = await pipe("Впервые в истории США экс-президент становится фигурантом уголовного дела. Большое жюри Манхэттена в четверг вечером проголосовало за предъявление Дональду Трампу обвинения по делу о предполагаемых выплатах порнозвезде Сторми Дэниелс. Решение жюри стало неожиданностью как для экс-президента, так и для его команды, считавшей, что прокурор откажется от этого дела из-за его крайней запутанности. Теперь бывший президент должен явиться в суд для дачи показаний.");
  console.log(summary);
}

go();

Something like that

expected output is an english sentence :)

xenova · 2023-03-31T20:21:27Z

Thanks 👍 I think I've got a basic version working. I'll do some testing tomorrow, and hopefully have a release out either tomorrow or the day afterwards 🔥

eviltik · 2023-03-31T21:12:57Z

Wow, thank you man !

eviltik · 2023-04-02T17:27:26Z

Hi !

I just finish some tests with your implementation.

Technically, sound's good, but the quality of the translation is really far away from the one i have at hugging face with the same model, i.e https://t.ly/g7f_

I've

updated my local transformers.js repo
go to scripts/ then pip install -r requirements.txt

Then

franck@debian:~/code/transformers.js$ python3 -m scripts.convert --model_id Helsinki-NLP/opus-mt-ru-en --from_hub --quantize --task seq2seq-lm-with-past

Using framework PyTorch: 2.0.0+cu117
Overriding 1 configuration item(s)
	- use_cache -> False
/home/franck/.local/lib/python3.9/site-packages/transformers/models/marian/modeling_marian.py:240: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
/home/franck/.local/lib/python3.9/site-packages/transformers/models/marian/modeling_marian.py:247: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attention_mask.size() != (bsz, 1, tgt_len, src_len):
/home/franck/.local/lib/python3.9/site-packages/transformers/models/marian/modeling_marian.py:279: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
============= Diagnostic Run torch.onnx.export version 2.0.0+cu117 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Using framework PyTorch: 2.0.0+cu117
Overriding 1 configuration item(s)
	- use_cache -> True
/home/franck/.local/lib/python3.9/site-packages/transformers/models/marian/modeling_marian.py:862: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if input_shape[-1] > 1:
/home/franck/.local/lib/python3.9/site-packages/transformers/models/marian/modeling_marian.py:83: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  mask = torch.full((tgt_len, tgt_len), torch.tensor(torch.finfo(dtype).min))
============= Diagnostic Run torch.onnx.export version 2.0.0+cu117 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Using framework PyTorch: 2.0.0+cu117
Overriding 1 configuration item(s)
	- use_cache -> True
Asked a sequence length of 16, but a sequence length of 1 will be used with use_past == True for `decoder_input_ids`.
/home/franck/.local/lib/python3.9/site-packages/transformers/models/marian/modeling_marian.py:199: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if (
============= Diagnostic Run torch.onnx.export version 2.0.0+cu117 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Quantizing:   0%
Ignore MatMul due to non constant B: /[/layers.0/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/layers.0/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/layers.1/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/layers.1/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/layers.2/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/layers.2/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/layers.3/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/layers.3/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/layers.4/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/layers.4/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/layers.5/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/layers.5/self_attn/MatMul_1]
Quantizing:  33%|
gnore MatMul due to non constant B: /[/model/decoder/layers.0/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/encoder_attn/MatMul_1]
Quantizing:  67%|
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.0/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.1/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.2/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.3/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.4/encoder_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/self_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/self_attn/MatMul_1]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/encoder_attn/MatMul]
Ignore MatMul due to non constant B: /[/model/decoder/layers.5/encoder_attn/MatMul_1]
Quantizing: 100%

Merging decoders
The two models proto have different outputs (25 and 13 outputs). Constant outputs will be added to unify the two models outputs.
Addind a constant output for present.0.encoder.key of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.0.encoder.value of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.1.encoder.key of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.1.encoder.value of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.2.encoder.key of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.2.encoder.value of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.3.encoder.key of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.3.encoder.value of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.4.encoder.key of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.4.encoder.value of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.5.encoder.key of shape [0, 8, 1, 64] in model2.
Addind a constant output for present.5.encoder.value of shape [0, 8, 1, 64] in model2.

Then, run this code

const { pipeline, env } = require('./src/transformers.js')

env.localURL = './models';
env.remoteModels = false;

async function go() {
  let pipe = await pipeline('translation', 'onnx/quantized/Helsinki-NLP/opus-mt-ru-en');
  let summary = await pipe("Впервые в истории США экс-президент становится фигурантом уголовного дела. Большое жюри Манхэттена в четверг вечером проголосовало за предъявление Дональду Трампу обвинения по делу о предполагаемых выплатах порнозвезде Сторми Дэниелс. Решение жюри стало неожиданностью как для экс-президента, так и для его команды, считавшей, что прокурор откажется от этого дела из-за его крайней запутанности. Теперь бывший президент должен явиться в суд для дачи показаний.");
  console.log(summary);
}

go();

Crazy output :)

[
  {
    translation_text: 'For the sake of the law, the law of the Republic of Korea states that the law of the Republic of Korea is applicable to the law of the Republic of Korea. The law of the Republic of Korea is applicable to the law of the Republic of Korea. The law of the Republic of Korea is applicable to the law of the Republic of Korea. The law of the Republic of Korea is applicable to the law of the Republic of Korea. The law of the Republic of Korea. The law of the Republic of Korea is applicable to the law of the Republic of Korea. The law of the Republic of Korea is applicable to the law of the Republic of Korea. The law of the Republic of Korea is applicable to the law of the Republic of Korea and the State of the Republic of Korea.'
  }
]

If a run the node code twice, it crash

Error: Failed to load model because protobuf parsing failed.
    at new OnnxruntimeSessionHandler (/home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-node/dist/backend.js:27:92)
    at /home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-node/dist/backend.js:64:29
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. 
/home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6
[ ..... ]
Error: no available backend found. ERR: 
    at /home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6:11822
    at m.create (/home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6:11918)
    at constructSession (/home/franck/code/transformers.js/src/models.js:44:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 1)
    at async seq2seqLoadModel (/home/franck/code/transformers.js/src/models.js:102:16)
    at async AutoModelForSeq2SeqLM.from_pretrained (/home/franck/code/transformers.js/src/models.js:1397:20)
    at async Promise.all (index 1)
    at async pipeline (/home/franck/code/transformers.js/src/pipelines.js:1005:17)
    at async go (/home/franck/code/transformers.js/run.js:17:14)

Any suggestion ?

xenova · 2023-04-02T17:37:17Z

Hey 👋 Yeah I'm still running more tests before pushing a release. I did recently fix an issue where it wouldn't use the correct default sampling settings (I will push this to main shortly). In the meantime, you can set the sampling settings using this:

let output = await pipe("Впервые в истории США экс-президент становится фигурантом уголовного дела. Большое жюри Манхэттена в четверг вечером проголосовало за предъявление Дональду Трампу обвинения по делу о предполагаемых выплатах порнозвезде Сторми Дэниелс. Решение жюри стало неожиданностью как для экс-президента, так и для его команды, считавшей, что прокурор откажется от этого дела из-за его крайней запутанности. Теперь бывший президент должен явиться в суд для дачи показаний.", {
  top_k: 0,
  do_sample: false,
  num_beams: 1
});

Running your code with these new changes produces the following text:

"For the first time in U.S. history, the ex-president is becoming a criminal figure, and the grand jury of Manhattan voted on Thursday night to charge Donald Trump with the alleged payment of a porn star, Stormy Daniels, and the jury's decision was a surprise to both the ex-president and his team, who thought that the prosecutor would drop the case because of his extreme confusion, and now the former president has to appear before the court to testify."

Although the English isn't perfect, it definitely is better than what you are seeing haha. One potential reason for the slight inaccuracies is the quantization, so, if you are able to avoid quantizing the model, you would probably get better results (you can do this by removing --quantize)

EDIT: Could you sent the input text for the output of the message which contains "Republic of Korea"? You seem to have sent the original text twice.

eviltik · 2023-04-02T18:02:56Z

I removed the --quantize, then convert again using

python3 -m scripts.convert --model_id Helsinki-NLP/opus-mt-ru-en --from_hub --task seq2seq-lm-with-past

let pipe = await pipeline('translation', 'onnx/unquantized/Helsinki-NLP/opus-mt-ru-en', { top_k: 0, do_sample: false, num_beams: 1 });

Got this. Better than quantized one, but your result is more accurate

[
  {
    translation_text: 'For the first time, the United Nations has been able to make a difference in the quality of its work, and it is the case that the United Nations has been able to do so in the form of a number of measures, such as the adoption of'
  }
]

And then, i can't run twice the nodejs code:

Error: no available backend found. ERR: 
    at /home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6:11822
    at m.create (/home/franck/code/transformers.js/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6:11918)
    at constructSession (/home/franck/code/transformers.js/src/models.js:44:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 1)
    at async seq2seqLoadModel (/home/franck/code/transformers.js/src/models.js:102:16)
    at async AutoModelForSeq2SeqLM.from_pretrained (/home/franck/code/transformers.js/src/models.js:1397:20)
    at async Promise.all (index 1)
    at async pipeline (/home/franck/code/transformers.js/src/pipelines.js:1005:17)
    at async go (/home/franck/code/transformers.js/run.js:19:14)

I check differences of model's files just after the first run of the code, then after : no diff. I need to convert hugging model each time.

I'm always using the same text for testing, i.e

Впервые в истории США экс-президент становится фигурантом уголовного дела. Большое жюри Манхэттена в четверг вечером проголосовало за предъявление Дональду Трампу обвинения по делу о предполагаемых выплатах порнозвезде Сторми Дэниелс. Решение жюри стало неожиданностью как для экс-президента, так и для его команды, считавшей, что прокурор откажется от этого дела из-за его крайней запутанности. Теперь бывший президент должен явиться в суд для дачи показаний.

I don't know why Republic of Korea appears in the translation. In fact i started to be really afraid about that, i don't wanna Kim Jong-un to be on my computer right now huhu :)

eviltik · 2023-04-02T18:24:29Z

OK, forget the error "Error: no available backend found. ERR: ". It was related to my #62 cache implementation.

xenova · 2023-04-02T18:24:53Z

I don't know why Republic of Korea appears in the translation. In fact i started to be really afraid about that, i don't wanna Kim Jong-un to be on my computer right now huhu :)

Oh goodness haha! Then, there is definitely something else wrong lol. I've pushed the latest changes so you can test on the same codebase as me. So, can you do the following things (in this order):

Update the codebase (particularly due to this bug which was fixed) and run your code to see if it fixed it
If not, can you use the model files which I already processed: https://huggingface.co/Xenova/transformers.js/tree/main/quantized/Helsinki-NLP/opus-mt-ru-en/seq2seq-lm-with-past

OK, forget the error "Error: no available backend found. ERR: ". It was related to my #62 cache implementation.

👍

eviltik · 2023-04-02T18:33:44Z

Done.

The result is ... let's say ... I died laughing

[
  {
    translation_text: "For the first time, I think it's possible to have a little bit of a chance to do something like that."
  }
]

Let me try with your remote model files

xenova · 2023-04-02T18:51:30Z

👀👀👀 Well... that is... unfortunate 🤣 I have received sponsorships before (see under my profile, it lists 3 names), so, it must be a GitHub bug 👀

eviltik · 2023-04-02T19:00:10Z

👀👀👀 Well... that is... unfortunate 🤣 I have received sponsorships before (see under my profile, it lists 3 names), so, it must be a GitHub bug 👀

Yep. But you accept koffee so ... done !

And it's just perfect with remote model files.

[
  {
    translation_text: "For the first time in U.S. history, the ex-president became a criminal figure, and the grand jury of Manhattan voted on Thursday night to charge Donald Trump with the alleged payment of a porn star, Stormy Daniels, and the jury's decision was a surprise to both the ex-president and his team, who thought that the prosecutor would drop the case because of his extreme confusion, and now the former president must appear before the court to testify."
  }
]

Because i plan to use lot's of Helsinki-NLP models for other languages, i really like to know how did you convert Helsinki-NLP files to transformers.js files exactly. You succeed with quantized files. Let me test again : python convert with --quantize and local models.

xenova · 2023-04-02T19:05:27Z

Yep. But you accept koffee so ... done !

Haha thank you so much! I really appreciate it 😊

And it's just perfect with remote model files.

That is very strange... The only real difference I can see between my output and yours is that you're using pytorch v2.0.0, while I am still using v1.13. If that is the only difference, you might have found a bug with optimum and/or transformers 👀

I actually set up a google colab document for doing the conversions, and I have already converted quite a few models in the meantime:

Let me share that colab notebook quickly.

xenova · 2023-04-02T19:32:17Z

Sorry for the delay 🤣 ... I've just been downloading the models to my local folder to test before I share the script.

xenova · 2023-04-02T19:52:59Z

Hmmm... Looks like like the model from Google Colab is different to mine 😅 ... I'll see what the problem is and fix it!

xenova · 2023-04-03T00:11:58Z

Hmm... after debugging for the past few hours, I'm unable to find the issue 😓 ... It's got something to do with the latest update(s) to optimum, which completely broke the conversion process...

I'll look into it more tomorrow :/

xenova · 2023-04-04T11:29:56Z

After a ton of debugging, the fix was actually very simple! 4868918

Let me know if that works for you :)

I will also release the other model files for other translations.

xenova · 2023-04-04T12:59:38Z

I'm uploading a few of the 1440 models (https://huggingface.co/Xenova/transformers.js/tree/main/quantized/Helsinki-NLP). I just chose the most popular ones, so there will be some missing, but you can just convert your own models. If you want, you can also make a PR to add those models to the base repo too :)

Here is some example usage code:

let pipe1 = await pipeline('translation', 'Helsinki-NLP/opus-mt-ru-en');
let output1 = await pipe1("Привет, как дела?", {
    top_k: 0,
    do_sample: false,
    num_beams: 1,
});
console.log(output1);
// Outputs: [{translation_text: "Hey, how's it going?"}]

let pipe2 = await pipeline('translation', 'Helsinki-NLP/opus-mt-de-en');
let output2 = await pipe2("Hallo, wie geht's dir?", {
    top_k: 0,
    do_sample: false,
    num_beams: 1
});
console.log(output2);
// Outputs: [{translation_text: 'Hello, how are you?'}]

I will close this issue once I publish the next release (v1.4.0) 👍

xenova · 2023-04-04T21:33:56Z

Marian models are now officially supported in v1.4.0 🥳
https://www.npmjs.com/package/@xenova/transformers/v/1.4.0

Closing the issue! 😄

eviltik · 2023-04-05T10:45:52Z

Huge thanks man. I'll test as soon as possible.

eviltik · 2023-04-16T10:55:18Z

Hi @xenova ! Hope you are well.

I'm testing right now, but i can't.

Error:

franck@debian:~/code/nlpd$ node src/test.js 
Error: Failed to load model because protobuf parsing failed.
    at new OnnxruntimeSessionHandler (/home/franck/code/nlpd/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-node/dist/backend.js:27:92)
    at /home/franck/code/nlpd/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-node/dist/backend.js:64:29
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)
Something went wrong during model construction (most likely a missing operation). Using `wasm` as a fallback. 
/home/franck/code/nlpd/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6
[......]

Error: no available backend found. ERR: 
    at /home/franck/code/nlpd/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6:11822
    at m.create (/home/franck/code/nlpd/node_modules/.pnpm/[email protected]/node_modules/onnxruntime-common/dist/ort-common.node.js:6:11918)
    at constructSession (/home/franck/code/nlpd/node_modules/.pnpm/@[email protected]/node_modules/@xenova/transformers/src/models.js:55:39)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 1)
    at async seq2seqLoadModel (/home/franck/code/nlpd/node_modules/.pnpm/@[email protected]/node_modules/@xenova/transformers/src/models.js:146:16)
    at async AutoModelForSeq2SeqLM.from_pretrained (/home/franck/code/nlpd/node_modules/.pnpm/@[email protected]/node_modules/@xenova/transformers/src/models.js:2327:20)
    at async Promise.all (index 1)
    at async pipeline (/home/franck/code/nlpd/node_modules/.pnpm/@[email protected]/node_modules/@xenova/transformers/src/pipelines.js:1441:17)

Code:

const { pipeline, env } = require('@xenova/transformers')

env.localURL = './models/quantized/'; // fresh downloaded from https://huggingface.co/Xenova/transformers.js/tree/main/quantized/Helsinki-NLP
env.remoteModels = false;

const model = 'Helsinki-NLP/opus-mt-fr-en';

pipeline('translation', model)
  .then(() => { console.log('ready'); })

package.json

"dependencies": {
    "@xenova/transformers": "1.4.2",
    "onnxruntime-node": "1.14.0",
  }

Node version: v18.16.0 (debian 11.6) in a virtualbox 7.

I tried things below, same error:

with/without onnxruntime-node
with/without custom wasm path
with another Helsinki model
transformer.js from 1.3.5 to 1.4.2 version

Are you able to reproduce ?

eviltik · 2023-04-16T11:16:53Z

It's fine when using remote model

eviltik added the question Further information is requested label Mar 31, 2023

eviltik closed this as completed Mar 31, 2023

xenova reopened this Mar 31, 2023

xenova changed the title ~~How to~~ [Model request] Helsinki-NLP/opus-mt-ru-en (marian) Mar 31, 2023

xenova added the enhancement New feature or request label Mar 31, 2023

xenova mentioned this issue Apr 1, 2023

[Bug] Converter doesn't convert Whisper model to ONNX #65

Closed

xenova added a commit that referenced this issue Apr 1, 2023

Add support for MarianMT models (#63)

42f91d9

xenova mentioned this issue Apr 4, 2023

Typegen #28

Merged

xenova added a commit that referenced this issue Apr 4, 2023

Add documented support for MarianMT models for translation (#63)

0e4079d

xenova closed this as completed Apr 4, 2023

arnabtarwani mentioned this issue Sep 11, 2023

[Question] Issue with deploying model to Vercel using NextJS and tRPC #295

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Model request] Helsinki-NLP/opus-mt-ru-en (marian) #63

[Model request] Helsinki-NLP/opus-mt-ru-en (marian) #63

eviltik commented Mar 31, 2023

eviltik commented Mar 31, 2023

xenova commented Mar 31, 2023

eviltik commented Mar 31, 2023 •

edited

Loading

xenova commented Mar 31, 2023

xenova commented Mar 31, 2023

eviltik commented Mar 31, 2023

xenova commented Mar 31, 2023

eviltik commented Mar 31, 2023

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023

xenova commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 2, 2023

xenova commented Apr 2, 2023

xenova commented Apr 2, 2023

xenova commented Apr 3, 2023 •

edited

Loading

xenova commented Apr 4, 2023 •

edited

Loading

xenova commented Apr 4, 2023 •

edited

Loading

xenova commented Apr 4, 2023

eviltik commented Apr 5, 2023

eviltik commented Apr 16, 2023 •

edited

Loading

eviltik commented Apr 16, 2023

[Model request] Helsinki-NLP/opus-mt-ru-en (marian) #63

[Model request] Helsinki-NLP/opus-mt-ru-en (marian) #63

Comments

eviltik commented Mar 31, 2023

eviltik commented Mar 31, 2023

xenova commented Mar 31, 2023

eviltik commented Mar 31, 2023 • edited Loading

xenova commented Mar 31, 2023

xenova commented Mar 31, 2023

eviltik commented Mar 31, 2023

xenova commented Mar 31, 2023

eviltik commented Mar 31, 2023

eviltik commented Apr 2, 2023 • edited Loading

xenova commented Apr 2, 2023 • edited Loading

eviltik commented Apr 2, 2023 • edited Loading

eviltik commented Apr 2, 2023

xenova commented Apr 2, 2023 • edited Loading

eviltik commented Apr 2, 2023 • edited Loading

xenova commented Apr 2, 2023 • edited Loading

eviltik commented Apr 2, 2023 • edited Loading

xenova commented Apr 2, 2023

xenova commented Apr 2, 2023

xenova commented Apr 2, 2023

xenova commented Apr 3, 2023 • edited Loading

xenova commented Apr 4, 2023 • edited Loading

xenova commented Apr 4, 2023 • edited Loading

xenova commented Apr 4, 2023

eviltik commented Apr 5, 2023

eviltik commented Apr 16, 2023 • edited Loading

eviltik commented Apr 16, 2023

eviltik commented Mar 31, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 2, 2023 •

edited

Loading

eviltik commented Apr 2, 2023 •

edited

Loading

xenova commented Apr 3, 2023 •

edited

Loading

xenova commented Apr 4, 2023 •

edited

Loading

xenova commented Apr 4, 2023 •

edited

Loading

eviltik commented Apr 16, 2023 •

edited

Loading