Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate LM Studio API calls from chat to completions #595

Closed
cpacker opened this issue Dec 9, 2023 · 1 comment · Fixed by #628
Closed

Migrate LM Studio API calls from chat to completions #595

cpacker opened this issue Dec 9, 2023 · 1 comment · Fixed by #628
Assignees
@cpacker
Labels
local-llm

Comments

@cpacker
Copy link
Collaborator

cpacker commented Dec 9, 2023 •
edited
Loading

  • LM Studio API integration currently routes requests via /chat/completions instead of /completions due to a bug in /completions
  • This bug is fixed in the latest LM Studio beta
  • Once the bug is fixed in main, should move current monkeypatch into a backend type lmstudio-legacy, and make lmstudio route to /completions
@cpacker cpacker self-assigned this Dec 9, 2023
@cpacker
Copy link
Collaborator Author

cpacker commented Dec 15, 2023

completions endpoint is now working in build 0.2.9, should migrate since it seems like chat/completions is now working worse than before (for our monkeypatch scheme)

@cpacker cpacker mentioned this issue Dec 15, 2023
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cpacker cpacker

Labels
local-llm
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

migrate to using completions endpoint by default letta-ai/letta
1 participant
@cpacker