FIX [FAILOVER - SEMANTIC PARSE] - Could not parse LLM output [AGENTS]

[jimilp7]

My approach to fix LLM Output Parsing here is to have a Fail-Over approach of Semantic Searching according to Parsing Format Instructions from the Prompt and building JSON from that.

This has proved to be much better so far from my local testing, but really happy to hear everyone's input and how this works for you. @hwchase17 @agola11

I tried to break it by injecting wrong format instructions, but it still was able to parse it, for example:

Question: is xray covered

> Entering new AgentExecutor chain...
Using embedded DuckDB without persistence: data will be transient
{
Here is your response:
    "action": "User's Summary of Benefits and Coverage Document Search Tool",
    "action_input": "x-ray"
Please let me know if that works for you, i am here to help
}

Observation: For a diagnostic test (x-ray), you will pay 0% coinsurance for in-network providers and 30% coinsurance for non-network providers.
Thought:
{
Here is your response:
    "action": "Final Answer",
    "action_input": "For a diagnostic test (x-ray), you will pay 0% coinsurance for in-network providers and 30% coinsurance for non-network providers."
Please let me know if that works for you, i am here to help
}

> Finished chain.
For a diagnostic test (x-ray), you will pay 0% coinsurance for in-network providers and 30% coinsurance for non-network providers.

To-Do: Needs cleanup. Looking for feedback.

[yummydum]

FYI #2656 and many other PR discussing the same issue. It seems like automatic recovery is partly implemented already though. Any plans on this automatic recovery feature? @hwchase17

[jimilp7]

This work has been partly been picked up on #2987. Closing for now. Thank you @hwchase17!!!