Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Structured Output Interface #644

Merged
merged 12 commits into from
Oct 16, 2024

Conversation

GirinMan
Copy link
Contributor

@GirinMan GirinMan commented Oct 16, 2024

What does this PR do?

This PR enhances the /v1/chat/completions API in LoRAX to improve compatibility with OpenAI’s structured output interface. The changes include:

  • Adding support for text, json_object, and json_schema as valid type options in response_format.
  • Making the schema (or json_schema) field optional, allowing the generation of arbitrary JSON objects when using json_object without requiring a predefined schema.
  • Retaining the original LoRAX interface behavior for backward compatibility.

This update brings LoRAX's API closer in line with OpenAI’s, allowing developers to more easily switch between or integrate both systems.

Fixes #643

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Was this discussed/approved via a Github issue or the discord / slack channel? Please add a link
    to it if that's the case.
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@tgaddair
@donggyukimc

@GirinMan GirinMan marked this pull request as draft October 16, 2024 04:53
@GirinMan GirinMan marked this pull request as ready for review October 16, 2024 11:08
@tgaddair
Copy link
Contributor

Love this! Will review today :)

Copy link
Contributor

@tgaddair tgaddair left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I think the one thing I might do as a follow-up is allow specifying a JSON schema even when the format is json_object, that would ensure backwards compatibility for existing LoRAX users. Do you have any objections to making that change?

@tgaddair tgaddair merged commit 4fb4d69 into predibase:main Oct 16, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhance LoRAX /v1/chat/completions API to Align with OpenAI Structured Output Interface
2 participants