Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SQLDatabaseToolkit doesn't work well with Postgresql, it will truncate the last double quotation marks in the SQL #5423

Closed
3 of 14 tasks
xiaohui-hiwintech opened this issue May 30, 2023 · 4 comments · Fixed by #5432
Closed
3 of 14 tasks

SQLDatabaseToolkit doesn't work well with Postgresql, it will truncate the last double quotation marks in the SQL #5423

xiaohui-hiwintech opened this issue May 30, 2023 · 4 comments · Fixed by #5432

Comments

@xiaohui-hiwintech
Copy link

System Info

Langchain: 0.0.184
Python: 3.10.9
Platform: Windows 10 with Jupyter lab

Who can help?

@vowelparrot

Information

  • The official example notebooks/scripts
  • My own modified scripts

Related Components

  • LLMs/Chat Models
  • Embedding Models
  • Prompts / Prompt Templates / Prompt Selectors
  • Output Parsers
  • Document Loaders
  • Vector Stores / Retrievers
  • Memory
  • Agents / Agent Executors
  • Tools / Toolkits
  • Chains
  • Callbacks/Tracing
  • Async

Reproduction

SQLDatabaseToolkit works well if the SQL doesn't include the double quotation marks at the end, if there is, it will truncate the last double quotation marks, resulting in an endless loop.
Below is the initial code snapshot.
image
And when I executed it.
image
The LLM generates the correct SQL, but the toolkit truncats the last double quotation marks.

Expected behavior

Won't truncate the last double quotation marks for PostgreSql.
@ninjapenguin
Copy link
Contributor

Could you include the full prefix and query you're using to generate this error please, I'm having a hard time recreating the issue locally? 🙇

ninjapenguin pushed a commit to ninjapenguin/langchain that referenced this issue May 30, 2023
MRKL output parser no longer breaks well formed queries
f41fa91 
/ref langchain-ai#5423

Fixes langchain-ai#5423
@ninjapenguin ninjapenguin mentioned this issue May 30, 2023
Merged
eyurtsev pushed a commit that referenced this issue May 30, 2023
MRKL output parser no longer breaks well formed queries (#5432)
1d861dc 
# Handles the edge scenario in which the action input is a well formed
SQL query which ends with a quoted column

There may be a cleaner option here (or indeed other edge scenarios) but
this seems to robustly determine if the action input is likely to be a
well formed SQL query in which we don't want to arbitrarily trim off `"`
characters

Fixes #5423

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Agents / Tools / Toolkits
  - @vowelparrot
@xiaohui-hiwintech
Copy link
Author

xiaohui-hiwintech commented May 31, 2023 •
edited
Loading

Could you include the full prefix and query you're using to generate this error please, I'm having a hard time recreating the issue locally? 🙇

Oh, just saw your message, below is the prompt, I just changed the original a little for more suitable to Postgresql.
You are an agent designed to interact with PostgreSQL database.\nGiven an input question, create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer. In the query, please enclose table names and column names with double quotation marks.\nUnless the user specifies a specific number of examples they wish to obtain, always limit your query to at most {top_k} results.\nYou can order the results by a relevant column to return the most interesting examples in the database.\nNever query for all the columns from a specific table, only ask for the relevant columns given the question.\nYou have access to tools for interacting with the database.\nOnly use the below tools. Only use the information returned by the below tools to construct your final answer.\nYou MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.\n\nDO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.\n\nIf the question does not seem related to the database, just return "I don\'t know" as the answer.\n

@ninjapenguin
Copy link
Contributor

Thanks! I managed to replicate and the fix is now merged, hopefully that resolves your issue for you

vowelparrot pushed a commit that referenced this issue May 31, 2023
@vowelparrot
MRKL output parser no longer breaks well formed queries (#5432)
7054826 
# Handles the edge scenario in which the action input is a well formed
SQL query which ends with a quoted column

There may be a cleaner option here (or indeed other edge scenarios) but
this seems to robustly determine if the action input is likely to be a
well formed SQL query in which we don't want to arbitrarily trim off `"`
characters

Fixes #5423

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Agents / Tools / Toolkits
  - @vowelparrot
@xiaohui-hiwintech
Copy link
Author

Thanks! I managed to replicate and the fix is now merged, hopefully that resolves your issue for you

That's cool, thank you so much.

Undertone0809 pushed a commit to Undertone0809/langchain that referenced this issue Jun 19, 2023
@Undertone0809
MRKL output parser no longer breaks well formed queries (langchain-ai…
184244a 
…#5432)

# Handles the edge scenario in which the action input is a well formed
SQL query which ends with a quoted column

There may be a cleaner option here (or indeed other edge scenarios) but
this seems to robustly determine if the action input is likely to be a
well formed SQL query in which we don't want to arbitrarily trim off `"`
characters

Fixes langchain-ai#5423

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Agents / Tools / Toolkits
  - @vowelparrot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

MRKL output parser no longer breaks well formed queries ninjapenguin/langchain
2 participants
@ninjapenguin @xiaohui-hiwintech