ChatGoogleGenerativeAI: Invalid JSON payload received

[netwrkx]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

javascript ...

class QuestionGenTool extends StructuredTool {
  questionSchema = z.object({
    question: z.string().describe("Question text"),
    type: z.enum(['singleChoice', 'multiChoice']).describe("Question type"),
    options: z.array(z.string()).describe("List of possible answers"),
    correctAnswer: z.string().optional().describe("correct answer from the possible answers"),
  });
  
  schema = z.array(questionSchema).describe("Array of question objects")

  name = "question_gen_tool";

  description =
    "useful for when you need to generate questions on a subject.";

  async _call(questions) {

    try {
      const validatedQuestions = schema.safeParse(questions);
      if (!validatedQuestions.success) {
        throw new Error("Invalid question schema: " + validatedQuestions.error.issues.join(", "));
      }

      return "question_gen_tool";
    } catch (error) {
      console.error("Error in QuestionGenTool._call:", error);
      throw error; // Re-throw the error for handling
    }
  }
}

const modelWithTools = googleModel.withStructuredOutput(tool.schema, {
   name: tool.name, 
});

const response = async () => await modelWithTools.invoke([
  [
    "human",
    "Create an array of 8 questions with 5 options(4 incorrect answer and 1 correct answer) for an interview with a science fiction author.",
  ],
]);

Error Message and Stack Trace (if applicable)

Error: [400 ] Invalid JSON payload received. Unknown name "additionalProperties" at 'tools[0].function_declarations[0].parameters.items': Cannot find field. [{"@type":"type.googleapis.com/google.rpc.BadRequest","fieldViolations":[{"field":"tools[0].function_declarations[0].parameters.items","description":"Invalid JSON payload received. Unknown name \"additionalProperties\" at 'tools[0].function_declarations[0].parameters.items': Cannot find field."}]}]
    at _makeRequestInternal (@langchain_google-genai.js?v=0a40b38c:184:13)
    at async generateContent (@langchain_google-genai.js?v=0a40b38c:468:20)
    at async @langchain_google-genai.js?v=0a40b38c:1568:18
    at async RetryOperation._fn (chunk-WXLEKSSA.js?v=59289361:279:19)

Description

i'm trying to use langchain library to generate questions in json array format
Followed the example in this link

Generating json object works but json array returns error.

System Info

"@langchain/community": "^0.2.11",
"@langchain/google-genai": "^0.0.17",
"langchain": "^0.2.5",
"zod": "^3.23.8"

Node v18.3.0

[dosubot]

Hello @netwrkx! I'm here to assist you with any bugs, questions, or contribution inquiries you may have.

The error you're encountering is due to the additionalProperties field in your schema, which is not supported by Google's function calling. To resolve this, you need to ensure that your schema does not include additionalProperties.

Here's how you can modify your QuestionGenTool class to avoid this issue:

1. Define the schema without additionalProperties.
2. Ensure that the schema is correctly passed to googleModel.withStructuredOutput.

Here's the updated code:

class QuestionGenTool extends StructuredTool {
  questionSchema = z.object({
    question: z.string().describe("Question text"),
    type: z.enum(['singleChoice', 'multiChoice']).describe("Question type"),
    options: z.array(z.string()).describe("List of possible answers"),
    correctAnswer: z.string().optional().describe("correct answer from the possible answers"),
  });

  schema = z.array(this.questionSchema).describe("Array of question objects");

  name = "question_gen_tool";

  description = "useful for when you need to generate questions on a subject.";

  async _call(questions) {
    try {
      const validatedQuestions = this.schema.safeParse(questions);
      if (!validatedQuestions.success) {
        throw new Error("Invalid question schema: " + validatedQuestions.error.issues.join(", "));
      }

      return "question_gen_tool";
    } catch (error) {
      console.error("Error in QuestionGenTool._call:", error);
      throw error; // Re-throw the error for handling
    }
  }
}

const tool = new QuestionGenTool();

const modelWithTools = googleModel.withStructuredOutput(tool.schema, {
  name: tool.name,
});

const response = async () => await modelWithTools.invoke([
  [
    "human",
    "Create an array of 8 questions with 5 options(4 incorrect answer and 1 correct answer) for an interview with a science fiction author.",
  ],
]);

This should resolve the Invalid JSON payload received error by ensuring that the schema does not include unsupported fields like additionalProperties ^[1].

To continue talking to Dosu, mention @dosu.

[jacoblee93]

CC @bracesproul

[bracesproul]

@netwrkx version 0.0.19 just released with a fix for this. Thank you for flagging this!

[netwrkx]

@bracesproul Thanks. version 0.0.19 fixed the error

[talperetz]

@bracesproul I'm still getting it when I run with Gemini.

const model = new ChatGoogleGenerativeAI({
        model: "gemini-pro",
    });

    const llmGraphTransformer = new LLMGraphTransformer({
        llm: model,
    });

    const result = await llmGraphTransformer.convertToGraphDocuments(
    ...

Error: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent: [400 ] Invalid JSON payload received. Unknown name "additionalProperties" at 'tools[0].function_declarations[0].parameters.properties[0].value.items': Cannot find field. Invalid JSON payload received. Unknown name "additionalProperties" at 'tools[0].function_declarations[0].parameters.properties[1].value.items': Cannot find field. Invalid JSON payload received. Unknown name "additionalProperties" at 'tools[0].function_declarations[0].parameters': Cannot find field. Invalid JSON payload received. Unknown name "$schema" at 'tools[0].function_declarations[0].parameters': Cannot find field. [{"@type":"type.googleapis.com/google.rpc.BadRequest","fieldViolations":[{"field":"tools[0].function_declarations[0].parameters.properties[0].value.items","description":"Invalid JSON payload received. Unknown name \"additionalProperties\" at 'tools[0].function_declarations[0].parameters.properties[0].value.items': Cannot find field."},{"field":"tools[0].function_declarations[0].parameters.properties[1].value.items","description":"Invalid JSON payload received. Unknown name \"additionalProperties\" at 'tools[0].function_declarations[0].parameters.properties[1].value.items': Cannot find field."},{"field":"tools[0].function_declarations[0].parameters","description":"Invalid JSON payload received. Unknown name \"additionalProperties\" at 'tools[0].function_declarations[0].parameters': Cannot find field."},{"field":"tools[0].function_declarations[0].parameters","description":"Invalid JSON payload received. Unknown name \"$schema\" at 'tools[0].function_declarations[0].parameters': Cannot find field."}]}]

"@langchain/community": "^0.3.12",
"@langchain/core": "^0.3.17",
"@langchain/google-genai": "^0.1.3",
"langchain": "^0.3.5",

Same thing works with openai