[Documentation] Add examples to Rust bindings

[alfredodeza]

Describe the documentation issue

Now that the Rust bindings for the ONNX runtime (#12606), it would be useful to have examples. I know there are a couple of files like this one but there are none for other models or other types of inference like for summarization.

Although I'm not very proficient in Rust, I've tried crafting a working example for summarization that is not quite right. My suspicion is that it is because the model I'm using (GPT-2) doesn't have the correct vocab.json and merges.txt files.

The full example with instructions for setup is here https://github.com/alfredodeza/rust-onnx-summarization

I'm happy to volunteer to help out and draft the wording and any additions if we can get at least the summarizer example working.

Page / URL

No response

[alfredodeza]

@boydjohnson any chance you can hint as to what we can do here to make this example work?

[boydjohnson]

@alfredodeza I think with summarization tasks you loop until the EOS (end of sentence) token is reached. I would also expect that before you do the argmax you do a softmax (provided by the rust bindings) on the logits to get probabilities.

I also noticed you used the 0.0.14 release of the crate onnxruntime. Those were under the old maintainer. We haven't released to crates yet. You can get the new onnxruntime bindings via a cargo git dependency until we release to crates.

cargo.toml

[dependencies]
onnxruntime = { git = "https://github.com/microsoft/onnxruntime" }

[natke]

@alfredodeza Did you try with the latest RUST API and the suggestion above? Let us know whether this solved your problem.

[alfredodeza]

@natke @boydjohnson I tried with the latest version of the runtime but I that introduced other errors I wasn't able to get past by the compiler.

Again, I think the ask is to try to have examples on the usage. I would be happy to produce them but I'm not that proficient in Rust. Any examples, including the porting of what I shared here would be useful to others and to increase the usage and awareness.

[alfredodeza]

@natke I think it would still be valuable to have documented examples as we can't find a way to use the Rust API for the runtime. Even if the suggestion of using the git directive in Cargo.toml would work correctly, it should still be a documented recommendation