Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor cortex API with new model.yaml structure #656

Merged
merged 4 commits into from
Jun 5, 2024

Conversation

louis-jan
Copy link
Contributor

@louis-jan louis-jan commented Jun 3, 2024

Describe Your Changes

This PR is to revamp model persistence and its properties:

  • Settings & Params are flattened
  • No decoration metadata such as description, format, tags, metadata ...
  • Some of the properties are updated:
    • sources -> files (string [] or multimodal structure)
    • id -> model
  • Models are loaded from a YAML file instead of a SQLite database.
  • model_id with colon ":" will be replaced by dash "-"
  • Removed hard-coded chat inference parameters.
  • Support non streaming response
  • Removed the duplicated model pull function that returns a weird model ID with the syntax cortex pull llama3. It would return cortex pull llama3:branch after selection.

Model YAML file generated by a pull command

Screenshot 2024-06-05 at 14 37 15

Fixes Issues

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@louis-jan louis-jan marked this pull request as draft June 3, 2024 13:17
@louis-jan louis-jan force-pushed the feat/refactor-model-yaml branch 5 times, most recently from 494a155 to 9bec926 Compare June 4, 2024 15:11
@louis-jan louis-jan marked this pull request as ready for review June 4, 2024 15:18
@louis-jan louis-jan force-pushed the feat/refactor-model-yaml branch 2 times, most recently from faf5de8 to a1df7ba Compare June 5, 2024 06:02
@louis-jan louis-jan requested review from namchuai, irfanpena and marknguyen1302 and removed request for irfanpena June 5, 2024 06:08
@louis-jan louis-jan force-pushed the feat/refactor-model-yaml branch 5 times, most recently from 6cf320b to 6b3df94 Compare June 5, 2024 07:34
@louis-jan louis-jan force-pushed the feat/refactor-model-yaml branch from 6b3df94 to f7284c5 Compare June 5, 2024 07:36
@louis-jan louis-jan force-pushed the feat/refactor-model-yaml branch from b6c05d8 to 2adee26 Compare June 5, 2024 07:54
@louis-jan louis-jan force-pushed the feat/refactor-model-yaml branch from f5d8688 to 9c68747 Compare June 5, 2024 11:04
@louis-jan louis-jan merged commit cdf9e04 into dev Jun 5, 2024
@louis-jan louis-jan deleted the feat/refactor-model-yaml branch June 5, 2024 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants