.Net: Add prompt execution settings to AutoFunctionInvocationContext #10551
Conversation
markwallace-microsoft
added
.NET
SergeyMenshykh
temporarily deployed
to
integration
GitHub Actions
Inactive
| /// The execution settings associated with the operation. | ||
| /// </summary> | ||
| [Experimental("SKEXP0001")] | ||
| public PromptExecutionSettings? ExecutionSettings { get; init; } |
There was a problem hiding this comment.
Do we need a separate property in AutoFunctionInvocationContext for execution settings? Can we re-use AutoFunctionInvocationContext.Arguments for such scenario? Since KernelArguments type already contains a collection of execution settings.
There was a problem hiding this comment.
@dmytrostruk Not fully sure. Arguments today is only filled if PromptExecutionSettings was literally passed as an Argument right?
There was a problem hiding this comment.
@RogerBarreto Yes, but that's the only way how to pass PromptExecutionSettings if you use kernel.InvokePromptAsync instead of IChatCompletionService.
There was a problem hiding this comment.
@dmytrostruk I see, any changes to Arguments in this manner would change the behavior, could be a good thing anyways, but we would need to reconsider how Arguments will be reflected, because from this perspective if we reuse arguments for actual selected settings, would be also important to add knowledge of which of the arguments in the execution settings dictionary is the one being used by the function.
| @@ -129,6 +130,7 @@ public FunctionCallsProcessor(ILogger? logger = null) | |||
| /// <returns>Last chat history message if function invocation filter requested processing termination, otherwise null.</returns> | |||
| public async Task<ChatMessageContent?> ProcessFunctionCallsAsync( | |||
There was a problem hiding this comment.
Should this be public (is already a internal class and utility) ?
| public async Task<ChatMessageContent?> ProcessFunctionCallsAsync( | |
| internal async Task<ChatMessageContent?> ProcessFunctionCallsAsync( |
eavanvalkenburg
added
the
needs_port_to_python
Motivation, Context and Description
Today, the prompt execution settings supplied to the chat completion service are not available in
AutoFunctionInvocationContext. This PR adds the newAutoFunctionInvocationContext.ExecutionSettingsand assigns it to the settings with which the chat completion service was invoked.Closes: #10475