Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JS] Gemini does not handle multiple parallel calls of the same tool correctly #2241

Closed
mbleigh opened this issue Mar 4, 2025 · 0 comments
Closed
Labels
bug Something isn't working js P1 important bugs that need to be addressed

Comments

@mbleigh
Copy link
Collaborator

mbleigh commented Mar 4, 2025

Describe the bug
The Genkit mechanism for matching up tool requests with their responses is by providing a "ref" to disambiguate. However, the current Gemini implementation does not do this. If two calls are made in parallel to the same tool, they do not have refs.

This causes problems such as #2218 when trying to run logic to match up requests and responses.

To Reproduce

const getWeather = ai.defineInterrupt({
  name: "getWeather",
  description: "get the weather for a location",
  inputSchema: z.object({ location: z.string() }),
});

const { messages } = await ai.generate({
  prompt: "Get the weather for New York and San Francisco",
  tools: [getWeather],
});

Expected behavior
Each tool request in the returned messages has a ref on it. Instead, none.

Additional context
Gemini does not return any kind of identifiers on its tool requests - but they do appear in an ordered list. What I think we ought to do is translate the array index of the tool requests into the ref (e.g. "0", "1" etc) so that Genkit's logic can handle it.

Then, when translating a request to Gemini, pass the responses in ref order by parsing them back into integers and sorting them. If the responses don't have a ref...well, I guess just put those at the end after the ones that do?

@mbleigh mbleigh added bug Something isn't working js labels Mar 4, 2025
@mbleigh mbleigh marked this as a duplicate of #2218 Mar 4, 2025
@mbleigh mbleigh added the P1 important bugs that need to be addressed label Mar 4, 2025
@mbleigh mbleigh closed this as completed in 41bb5b3 Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working js P1 important bugs that need to be addressed
Projects
Status: Done
Development

No branches or pull requests

1 participant