ONNX Function Calling Support

[RogerBarreto]

I might be missing something but I didn't find any support for function calling on the APIs.

This capability arriving strong for SLM's (including Llama3+) there's a very big ask from community to start supporting function calling on SLMs.

As we seen this happening and arriving for Model Engines like Ollama would be very nice to have an abstraction that allow us to consume models with this capability with the Generator.

[natke]

Hi @RogerBarreto, we are working on this - via the constrained decoding feature. Do you have specific use-cases you can share to help us validate the feature? I'll move this into a discussion

[RogerBarreto]

I have plenty of use cases, but most of them are related to the current OpenAI contracts on tools behavior, if we can have some examples (if possible following similar abstractions) from the ONNX GenAI API perspective that would be really helpful.

• Request.Tools[] (advertisement - Available Fuctions)
• Request.Tool Choice (Auto/Required/None) (How the model will behave about the functions advertised)
• Response.Message.Assistant + ToolCalls[] (Response from the model when it wants to call one or many functions)
• Request.Message.Tool + FunctionResult (giving back to the model the function response)

[madroidmaq]

When I tried to construct a Function Calling Prompt myself, I encountered some issues, see #994.

[Taka152]

Hi, we just create #1038 about constrained decoding that can limit generation with json schema and regex, and should support function call too. If you are interested and want to try, it may needs to install rust toolchain and build from source for now.