ValueError(f"Could not parse LLM output: {llm_output}")

[chun1617]

File "C:\Program Files\Python\Python310\lib\site-packages\langchain\chains\base.py", line 268, in run return self(kwargs)[self.output_keys[0]] File "C:\Program Files\Python\Python310\lib\site-packages\langchain\chains\base.py", line 168, in __call__ raise e File "C:\Program Files\Python\Python310\lib\site-packages\langchain\chains\base.py", line 165, in __call__ outputs = self._call(inputs) File "C:\Program Files\Python\Python310\lib\site-packages\langchain\agents\agent.py", line 503, in _call next_step_output = self._take_next_step( File "C:\Program Files\Python\Python310\lib\site-packages\langchain\agents\agent.py", line 406, in _take_next_step output = self.agent.plan(intermediate_steps, **inputs) File "C:\Program Files\Python\Python310\lib\site-packages\langchain\agents\agent.py", line 102, in plan action = self._get_next_action(full_inputs) File "C:\Program Files\Python\Python310\lib\site-packages\langchain\agents\agent.py", line 64, in _get_next_action parsed_output = self._extract_tool_and_input(full_output) File "C:\Program Files\Python\Python310\lib\site-packages\langchain\agents\conversational\base.py", line 84, in _extract_tool_and_input raise ValueError(f"Could not parse LLM output:{llm_output}") ValueError: Could not parse LLM output: Thought: Do I need to use a tool? Yes
Action: Use the requests library to write a Python code to do a post request
Action Input:

import requests

url = 'https://example.com/api'
data = {'key': 'value'}

response = requests.post(url, data=data)

print(response.text)

``

[mrbende]

I am receiving a very similar response...

127.0.0.1 - - [07/Mar/2023 21:54:37] "OPTIONS /chat HTTP/1.1" 200 -
Message received: Please start a conversation with me!


> Entering new AgentExecutor chain...
127.0.0.1 - - [07/Mar/2023 21:54:39] "POST /chat HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask/app.py", line 2548, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.handle_exception(e)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask/app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask/app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/Users/reedbender/Repositories/Praxis/pria/app.py", line 78, in chat
    response = agent_executor.run(message)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/chains/base.py", line 213, in run
    return self(args[0])[self.output_keys[0]]
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/chains/base.py", line 116, in __call__
    raise e
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/chains/base.py", line 113, in __call__
    outputs = self._call(inputs)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/agents/agent.py", line 503, in _call
    next_step_output = self._take_next_step(
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/agents/agent.py", line 406, in _take_next_step
    output = self.agent.plan(intermediate_steps, **inputs)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/agents/agent.py", line 102, in plan
    action = self._get_next_action(full_inputs)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/agents/agent.py", line 64, in _get_next_action
    parsed_output = self._extract_tool_and_input(full_output)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/agents/mrkl/base.py", line 138, in _extract_tool_and_input
    return get_action_and_input(text)
  File "/Users/reedbender/.pyenv/versions/3.9.15/lib/python3.9/site-packages/langchain/agents/mrkl/base.py", line 46, in get_action_and_input
    raise ValueError(f"Could not parse LLM output: `{llm_output}`")
ValueError: Could not parse LLM output: `Sure, I'd be happy to chat with you! Is there anything specific you'd like to talk about or any questions you have?`

It seems that the ChatGPT api is handling the request, but something in the langchain parsing of the output is breaking. Any ideas? Thanks for these developments!

[RoySadaka]

It is possible that this is caused due to the nature of the current implementation, which puts all the prompts into the user role in ChatGPT.
I ran into the same issue when using the SQL agent.
To debug it, i took the text that is being sent to the LLM, but manually broke it down to ChatGPT messages style,
meaning, to the system role I've added the agent definition, followed by the user role with the "begin! and question" part, followed by the assistant role with the "Thought" part,
this resulted in a proper output that can be parsed.

It is plausible to assume that if/when the code will be updated to support chat format with system/user/assistant roles, in these agents, it will work again, until then, we can still use the (really good) davinchi model.

Thanks team!

[sidedwards]

Super hacky, but I appended " The response should include the prefix 'AI: <response>'." and that resolved some of my prompts that were breaking. 🤷

[jnorris441]

Sometimes the LLM ignores the instructions on the format which breaks the entire thing...like it clearly says the last line should be "Final Answer: " but if it comes back without that you are in trouble.

I hope the new chat messages format will eliminate the need to parse the reply text

[jphme]

I have the same problem, amplified by a SystemMessage in German. The model often forgets the correct tokens for Tool usage or the final answer, resulting in the parsing error.

Has anyone an example where all the prompts (at least the descriptions, not necessarily the tokens) for an Agent and the Tools are translated, so the model can't get confused between multiple languages?

[skylacking04]

It is possible that this is caused due to the nature of the current implementation, which puts all the prompts into the user role in ChatGPT. I ran into the same issue when using the SQL agent. To debug it, i took the text that is being sent to the LLM, but manually broke it down to ChatGPT messages style, meaning, to the system role I've added the agent definition, followed by the user role with the "begin! and question" part, followed by the assistant role with the "Thought" part, this resulted in a proper output that can be parsed.

It is plausible to assume that if/when the code will be updated to support chat format with system/user/assistant roles, in these agents, it will work again, until then, we can still use the (really good) davinchi model.

Thanks team!

can you show the code for that?

[claust]

This can happen if the model does not follow the given instructions. In most of the cases I have seen, it would proabably be better to simply return the output instead of throwing a ValueError when the output cannot be parsed according to the instructions. I will test this strategy further and maybe make a PR.

[libratiger]

It seem that because the LLM response with text start with "Thought", but the regex match code in langchain does not consider this.

[FJGEODEV]

Is there any solution to this? Thanks.

[alexprice12]

I tried to dig into a solution for this. It can be for a myriad for reasons it looks like. Basically, the model is not obeying the prompt.

At the minimum, I think agent code should ask the model for a well-structure response that adhere to the prompt.

[libratiger]

but the model may do not adopt your request. @alexprice12

this is about standard

[alexprice12]

When the model asks for a tool that doesn't exist, langchain tells the model that the tool they chose doesn't exist.

I'm open to more than one solution. What are you thinking is the best approach here?

[tomsib2001]

I hope it's not too silly an idea, and looking at the code I'm not sure where such an intervention would take place, but:
How about, instead of raising the error

        raise ValueError(f"Could not parse LLM output: `{llm_output}`")

, the answer given by the llm was sent back to it complaining that it does not have the right format, and asking it to comply with it?

[tiagoefreitas]

Duplicate of #1358

[rishibh7]

any solution yet?

[ion-storm]

This format_instructions prompt helped a bit to get the correct response format, open for suggestions on improvement:
format_instructions = """You must use the following format for all responses or your response will be considered incorrect:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do based on the tool capabilities available to you\nAction: the action to take, should only be one of [{tool_names}]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat 5 times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\nIf you can't find the answer, say 'I am unable to find the answer.'"""

[Willis75]

check this, worked for me:
https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/handle_parsing_errors.html

[gprabhuv4me]

I encountered a similar issue while using Azure OpenAI with the Pandas agent and attempted the solution mentioned above, but unfortunately, it did not resolve the problem.

To address this issue, I suggest creating a custom output parser specifically tailored to handle the responses from the Azure OpenAI Pandas Agent. With a custom output parser, we can ensure that the agent's output is handled appropriately, which may lead to resolving the problem more effectively.

from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser
from typing import List, Union
from langchain.schema import AgentAction, AgentFinish, HumanMessage, SystemMessage
import re

class CustomOutputParser(AgentOutputParser):

def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
    
    #print('step1')
    # Check if agent should finish
    if "Final Answer:" in llm_output:
        return AgentFinish(
            # Return values is generally always a dictionary with a single `output` key
            # It is not recommended to try anything else at the moment :)
            return_values={"output": llm_output.split("Final Answer:")[-1].strip()},
            log=llm_output,
        )
    
    # Check for specific message about missing values
    if "There are no missing values in the dataset." in llm_output:
        return AgentFinish(
            return_values={"output": "No missing values in the dataset."},
            log=llm_output,
        )
    # Parse out the action and action input - v1
    #regex = r"Action: (.*?)[\n]*Action Input:[\s]*(.*)"
    #match = re.search(regex, llm_output, re.DOTALL)
    
    # Parse out the action and action input - v2
    #regex = r"Action: (.*?)[\n]*Action Input:([\s\S]*)"
    #match = re.search(regex, llm_output, re.DOTALL)
    
    # Parse out the action and action input - v3
    regex = r"Action: (.*?)[\n]*Action Input:[\s]*(.*)"
    match = re.search(regex, llm_output, re.DOTALL)
    
    #print('step2')
    # If it can't parse the output it raises an error
    # You can add your own logic here to handle errors in a different way i.e. pass to a human, give a canned response
    if not match:
        raise ValueError(f"Could not parse LLM output: `{llm_output}`")
    
    action = match.group(1).strip()
    #action_input = match.group(2)
    action_input = match.group(2).strip(" ").strip('"')
    #print(action_input)
    # Return the action and action input
    #return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
    # Return the action and action input
    return AgentAction(tool=action, tool_input=action_input, log=llm_output)

from langchain.llms import AzureOpenAI
from langchain.agents.agent_types import AgentType
agent = create_pandas_dataframe_agent(llm, df,
verbose=True,
agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
output_parser=output_parser,
)

[dosubot]

Hi, @chun1617! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you raised titled "ValueError(f"Could not parse LLM output: {llm_output}")" is related to a ValueError that occurs while trying to parse LLM output in a Python code. There have been several comments from users suggesting possible solutions and workarounds. One user, tomsib2001, has proposed a resolution by suggesting to send the incorrect output back to the LLM and asking it to comply with the correct format.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain repository!

[zaldivards]

Surrounding the tool's output with quotes worked for me:

def my_calc(expr: str) -> str:
    """
    Some custom tool

    Evaluate the provided arithmetic expression in Python syntax.

    Example: my_calc("2+2")
    Result: 4
    """
    expr = re.sub(r"[^0-9+\-*/().]", "", expr)
    return f'"{eval(expr)}"' # surrounding the output

[dosubot]

@baskaryan Could you please help @chun1617 with the issue titled "ValueError(f"Could not parse LLM output: {llm_output}")"? They are still experiencing the problem and have provided a potential solution in their latest comment. Thank you!

[nameless0704]

I'm still getting this error on version 0.1.4 using open-source LLM Qwen-14B-Chat model. Is it still a thing?

[dosubot]

🤖

Based on the traceback you've provided, it seems like the error is originating from the _extract_tool_and_input method in the ConversationalAgent class. This method is responsible for parsing the output of the language model and extracting the tool and input for the next step in the chain.

The error message "Could not parse LLM output" suggests that the method was unable to parse the output of the language model. This could be due to the structure or format of the output not being what the method expects.

Here is the relevant code:

class ConversationalAgent(Agent):
    ...
    def _extract_tool_and_input(self, llm_output: str) -> Tuple[str, Dict[str, Any]]:
        ...
        raise ValueError(f"Could not parse LLM output: `{llm_output}`")

In this method, if the parsing fails for any reason, it raises a ValueError with the message "Could not parse LLM output".

To resolve this issue, you might need to check the output of your language model and ensure it's in a format that the _extract_tool_and_input method can parse. If it's not, you might need to modify the output of your language model or override the _extract_tool_and_input method in your own subclass of ConversationalAgent to handle your specific output format.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources:

• libs/langchain/langchain/agents/conversational/base.py

Sources

• libs/langchain/langchain/chains/llm.py
• libs/core/langchain_core/output_parsers/string.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.