Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add removing any text before json to parse_json_markdown #5334

Closed
wants to merge 1 commit into from

Conversation

mohsend
Copy link

@mohsend mohsend commented May 27, 2023

Add removing any text before the json string to parse_json_markdown (Issue #1358)

Fixes #1358 (ValueError: Could not parse LLM output: )

Sometimes the agent adds a little sentence before the thought JSON it's supposed to give. I causes an error. This little function removes that part before the main JSON response before trying to parse it. Here is an example error I got before this fix:

Traceback (most recent call last):
  File ".../langchain/agents/conversational_chat/output_parser.py", line 17, in parse
    response = parse_json_markdown(text)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../langchain/output_parsers/json.py", line 17, in parse_json_markdown
    parsed = json.loads(json_string)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
...
  File ".../langchain/chains/base.py", line 239, in run
    return self(kwargs, callbacks=callbacks)[self.output_keys[0]]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../langchain/chains/base.py", line 140, in __call__
    raise e
  File ".../langchain/chains/base.py", line 134, in __call__
    self._call(inputs, run_manager=run_manager)
  File ".../langchain/agents/agent.py", line 951, in _call
    next_step_output = self._take_next_step(
                       ^^^^^^^^^^^^^^^^^^^^^
  File ".../langchain/agents/agent.py", line 773, in _take_next_step
    raise e
  File ".../langchain/agents/agent.py", line 762, in _take_next_step
    output = self.agent.plan(
             ^^^^^^^^^^^^^^^^
  File ".../langchain/agents/agent.py", line 444, in plan
    return self.output_parser.parse(full_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../langchain/agents/conversational_chat/output_parser.py", line 24, in parse
    raise OutputParserException(f"Could not parse LLM output: {text}") from e
langchain.schema.OutputParserException: Could not parse LLM output: Sure, here's a sentence-long description of the first tool in the list:

```json
{
    "action": "Final Answer",
    "action_input": "The 'Search the internet' tool is useful for finding information about current events or the current state of the world. You can input a single search term to get started."
}
```

In this PR, in the example above parse_json_markdown will remove "Sure, here's a sentence-long description of the first tool in the list:" before trying to parse the string as a json.

Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
@vowelparrot

Copy link
Contributor

@hwchase17 hwchase17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets add tests!

@hwchase17
Copy link
Contributor

should be fixed in #5371

@hwchase17 hwchase17 closed this May 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ValueError: Could not parse LLM output:
2 participants