Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Bug: How do I disable parallel function calling in Semantic Kernel (Python)? #9478

Closed
jordanbean-msft opened this issue Oct 30, 2024 · 0 comments · Fixed by #9479
Closed
Assignees
Labels
ai connector Anything related to AI connectors bug Something isn't working python Pull requests for the Python Semantic Kernel

Comments

@jordanbean-msft
Copy link
Contributor

Describe the bug
We are dealing with this issue here : https://community.openai.com/t/the-multi-tool-use-parallel-bug-and-how-to-fix-it/880771. We tried some of the solution proposed on many threads, with no success.

We found many threads open in OpenAI about this, and a quick suggestion we are trying to implement is to disable parallel calling completely.
Also when reviewing the OpenAI documentation we found that “model outputs may not match strict schemas supplied in tools” when parallel calls are enabled (https://platform.openai.com/docs/guides/function-calling/parallel-function-calling-and-structured-outputs), an issue that also happens to us from time to time.

The problem we have is we are unable to find a setting in Semantic Kernel (Python) to disable parallel function calls at model level, there is a configuration on the Kernel class, but it seems to be internal to Semantic Kernel (meaning it will disable after the model responded with the call(s) instructions).

I can see how I would pass this value in if I directly called the underlying openai SDK, but how do I set this same value in Semantic Kernel?

To Reproduce
Steps to reproduce the behavior:
I was able to dig on the code flow and found the flag is present in OpenAI underlying dependency, but not on the pydantic representation in Semantic Kernel. We are able to change this to test, but I wasn’t able to figure out where in the code is Semantic Kernel doing the parallel calls when they are enabled.

Starting from /semantic_kernel/connectors/ai/open_ai/prompt_execution_settings/azure_chat_prompt_execution_settings.py

Image

I can get to the parent class which is in /semantic_kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py

Image

Then when doing a request I can verify the parameter is sent in /semantic_kernel/connectors/ai/open_ai/services/open_ai_handler.py

Image

And in turn that gets into OpenAI package: /openai/resources/chat/completions.py

Image

Which seems to be working as expected, but I cannot really confirm without checking how the tools are handled on the response by semantic kernel itself.

Expected behavior
A way to disable parallel function calling until the underlying bug from OpenAI is resolved.

Screenshots
Image

Platform

  • OS: Any
  • IDE: Any
  • Language: Python
  • Source: Any recent version of Semantic Kernel

Additional context
Add any other context about the problem here.

@jordanbean-msft jordanbean-msft added the bug Something isn't working label Oct 30, 2024
@markwallace-microsoft markwallace-microsoft added python Pull requests for the Python Semantic Kernel triage labels Oct 30, 2024
@github-actions github-actions bot changed the title Bug: How do I disable parallel function calling in Semantic Kernel (Python)? Python: Bug: How do I disable parallel function calling in Semantic Kernel (Python)? Oct 30, 2024
@moonbox3 moonbox3 self-assigned this Oct 30, 2024
@moonbox3 moonbox3 added ai connector Anything related to AI connectors and removed triage labels Oct 30, 2024
@moonbox3 moonbox3 moved this to Sprint: In Progress in Semantic Kernel Oct 30, 2024
@moonbox3 moonbox3 moved this from Sprint: In Progress to Sprint: In Review in Semantic Kernel Oct 30, 2024
github-merge-queue bot pushed a commit that referenced this issue Oct 30, 2024
…tion settings (#9479)

### Motivation and Context

OpenAI / Azure OpenAI released the ability to specify the
`parallel_tool_calls` = True | False boolean attribute on the prompt
execution settings in July. This was never brought into SK Python.

Further, there appears to be a pesky bug related to function calling
where enabling parallel tool calls can cause 500s. The only way to get
around this is to disable parallel function calling. This has been both
tested on the Azure Chat Completion and OpenAI Chat Completion code and
works well -- the chat history shows the synchronous function calls if
multiple tool calls are required.

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

Add the `parallel_tool_calls` attribute on the
OpenAIChatPromptExecutionExecution settings, which are also used by the
AzureChatCompletionClass.
- Closes #9478
- Adds unit tests for this prompt execution setting

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [X] The code builds clean without any errors or warnings
- [X] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [X] All unit tests pass, and I have added new tests where possible
- [X] I didn't break anyone 😄
@github-project-automation github-project-automation bot moved this from Sprint: In Review to Sprint: Done in Semantic Kernel Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ai connector Anything related to AI connectors bug Something isn't working python Pull requests for the Python Semantic Kernel
Projects
Archived in project
3 participants