Issue with GraphCypherQAChain Using LangChain & Neo4j Knowledge Graph Builder

[PaulDeFi]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import { Neo4jGraph } from "@langchain/community/graphs/neo4j_graph";
import { ChatOpenAI } from "@langchain/openai";
import { GraphCypherQAChain } from "@langchain/community/chains/graph_qa/cypher";
import { FewShotPromptTemplate, PromptTemplate } from "@langchain/core/prompts";
import { neo4jConfigAura } from '../config/neo4j-config-aura.js';
import { OpenAIEmbeddings } from "@langchain/openai";
import { SemanticSimilarityExampleSelector } from "@langchain/core/example_selectors";
import { Neo4jVectorStore } from "@langchain/community/vectorstores/neo4j_vector";

// Examples of questions and their corresponding Cypher queries
const examples = [
    {
        question: "What are all the whitepapers connected to the Knowledge node?",
        query: "MATCH (k:Knowledge)<-[:PART_OF]-(wp) RETURN wp.name"
    },
    {
        question: "Show me all relationships of the Uniswap whitepaper",
        query: "MATCH (wp:Document {name: 'Uniswap'})-[r]-() RETURN wp.name, type(r)"
    },
    {
        question: "What are the technical features described in the Uniswap whitepaper?", 
        query: "MATCH (d:Document {name: 'Uniswap'})-[:CONTAINS]->(f) RETURN f.name, f.description"
    },
    {
        question: "Who are the authors of the Uniswap whitepaper?",
        query: "MATCH (d:Document {name: 'Uniswap'})-[:AUTHORED_BY]->(p:PERSON) RETURN p.name, p.role"
    },
    {
        question: "What are the different protocol versions mentioned?",
        query: "MATCH (d:Document)-[:INTRODUCES]->(v:VERSION) RETURN d.name, v.version, v.description"
    },
    {
        question: "What components require other components in the architecture?",
        query: "MATCH (c1:COMPONENT)-[:REQUIRES]->(c2:COMPONENT) RETURN c1.name, c2.name"
    },
    {
        question: "What features were improved between protocol versions?",
        query: "MATCH (v1:VERSION)-[:IMPROVES]->(f:FEATURE)<-[:INTRODUCES]-(v2:VERSION) RETURN v1.version, v2.version, f.name"
    },
    {
        question: "List all components that are part of the core protocol",
        query: "MATCH (c:COMPONENT)-[:PART_OF]->(p:PROTOCOL) RETURN c.name, p.name"
    }
];

async function setupChain() {
    try {
        const graph = await Neo4jGraph.initialize(neo4jConfigAura);
        await graph.refreshSchema();
        const schema = graph.getSchema();

        const examplePrompt = PromptTemplate.fromTemplate(
            "User input: {question}\nCypher query: {query}"
        );

        const exampleSelector = await SemanticSimilarityExampleSelector.fromExamples(
            examples,
            new OpenAIEmbeddings(),
            Neo4jVectorStore,
            {
                k: 5,
                inputKeys: ["question"],
                preDeleteCollection: true,
                url: neo4jConfigAura.url,
                username: neo4jConfigAura.username,
                password: neo4jConfigAura.password
            }
        );

        const promptWithExampleSelector = new FewShotPromptTemplate({
            exampleSelector,
            examplePrompt,
            prefix: `You are a Neo4j expert. Given an input question, create a syntactically correct Cypher query to run.
                    Here is the schema information:\n${schema}\n
                    Below are a number of examples of questions and their corresponding Cypher queries.`,
            suffix: "User input: {question}\nCypher query:",
            inputVariables: ["question"]
        });

        const chain = GraphCypherQAChain.fromLLM({
            graph,
            llm: new ChatOpenAI({
                model: "gpt-3.5-turbo",
                temperature: 0,
            }),
            cypherPrompt: promptWithExampleSelector
        });

        return chain;
    } catch (error) {
        console.error("Initialization error:", error);
        throw error;
    }
}

async function queryGraph(question) {
    try {
        const chain = await setupChain();
        if (!chain) throw new Error("Failed to initialize chain");
        
        const response = await chain.invoke({ query: question });
        console.log("\nAnswer:", response.result);
    } catch (error) {
        console.error("Query error:", error);
        if (error.message) console.error("Error details:", error.message);
    }
}

// Usage example
await queryGraph("What are the core features of AAVE V1?");
// await queryGraph("Show me the technical features of Uniswap V4");
// await queryGraph("What are the similarities between Aave and Uniswap?");

Description

Hello,

I'm building a DeFi AI application and want to use the Neo4j Graph Platform to store knowledge in a graph-based system.

So far, I have used the langchain.js framework along with the Neo4j LLM Knowledge Graph Builder to experiment with different ways of creating the best graph structure based on WhitePaper inputs.

I decided to use the Neo4j Knowledge Graph Builder, which generated a graph based on 10 DeFi app WhitePapers.

The issue I’m facing is related to the GraphRAG Query I am trying to run on this graph using the @langchain/community framework. Specifically, I’m using the langchain/chains/graph_qa/cypher tutorials from langchain.

When running the following code, I consistently get the INVALID_PROMPT_INPUT error (error details here), and I am unable to modify anything to resolve it.

Or if I'm changing a lot parts the code, I'm still unable to have a good output from the FewShotPromptTemplate and the GraphCypherQAChain.fromLLMas well.

System Info

@langchain/community": "^0.3.28",
"@langchain/core": "^0.3.37",
"@langchain/openai": "^0.4.2",
"dotenv": "^16.4.7",
"fs": "0.0.1-security",
"langchain": "^0.3.15",
"neo4j-driver": "^5.28.0"

My docker and graph are running properly on Neo4j

[dosubot[bot]]

Hey @PaulDeFi! How's it going? I'm here to help you with any bugs, questions, or even becoming a contributor. Let's tackle this together!

I found some relevant information that might help you resolve the INVALID_PROMPT_INPUT error you're encountering.

1. Common Causes of INVALID_PROMPT_INPUT Error: This error can occur due to missing or invalid input variables, unescaped curly braces in JSON objects, incorrect use of MessagesPlaceholder, or issues with shorthand tuples. Ensure that your prompt template is correctly set up, with all variables properly defined and escaped ^[1].

2. Similar Threads: There is a similar closed issue where the solution involved ensuring that the inputVariables property is correctly set when creating a FewShotPromptTemplate instance. Make sure your FewShotPromptTemplate includes the inputVariables array with the correct variable names used in your prompt templates ^[2].

3. GraphCypherQAChain.fromLLM Method: Ensure that the input to the GraphCypherQAChain.fromLLM method is correctly formatted. You need to provide at least one of llm, cypherLLM, or qaLLM, and you can specify up to two of these, but not all three simultaneously ^[3].

Review these points and adjust your code accordingly to see if it resolves the issue. If the problem persists, double-check the prompt template and the input variables to ensure they align with the expected format.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[PaulDeFi]

@dosu

Thanks for the answers but still not working.

Even if I'm modifying the PromptTemplate.fromTemplate, the issue isn't solved. I followed this guide step by step but with a different graph I've built on Neo4j.

I've also tried to pass my query in the inputVariables but doesn't work as well and yes my GraphCypherQAChain.fromLLMmethod has an llm.

Do you have any other idea?

[dosubot[bot]]

To resolve the INVALID_PROMPT_INPUT error, ensure the following:

1. Input Variables: Double-check that all input variables specified in your PromptTemplate are correctly passed when you run the chain. For example, if your template uses schema and question, ensure these are provided.

2. Template Structure: Make sure your template is correctly structured. Here's an example of a template for generating Cypher statements:

   const cypherTemplate = `Task: Generate Cypher statement to query a graph database.
   Instructions:
   Use only the provided relationship types and properties in the schema.
   Schema:
   {schema}
   Note: Do not include any explanations or apologies in your responses.
   Follow these Cypher examples:
   # How many actors played in Top Gun?
   MATCH (m:Movie {title:"Top Gun"})<-[:ACTED_IN]-()
   RETURN count(*) AS result 
   
   The question is:
   {question}`;
   
   const cypherPrompt = new PromptTemplate({
     template: cypherTemplate,
     inputVariables: ["schema", "question"],
   });

3. LLM and Graph Initialization: Ensure that the llm and graph are properly initialized. The GraphCypherQAChain.fromLLM method should have the llm parameter correctly set.

4. Check LLM Configuration: Ensure that either llm or cypherLLM and qaLLM are provided, but not all three simultaneously ^[1].

5. Logging Inputs: Consider logging the inputs to the prompt template to ensure they are as expected. This can help identify any discrepancies in the data being passed.

By following these steps, you should be able to resolve the INVALID_PROMPT_INPUT error. If the issue persists, ensure that your Neo4j graph schema and the input question are correctly formatted and aligned with the template requirements ^[2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[Simox21]

thx!)