Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support HF model handle and quant selection #1239

Merged
merged 3 commits into from
Sep 19, 2024
Merged

Conversation

namchuai
Copy link
Collaborator

@namchuai namchuai commented Sep 17, 2024

Describe Your Changes

  • Fix some issues with model download via CLI.
  • Allow user to select model variant to pull.

Screenshots

  1. When pull from cortex but user input an incompatible model repository. E.g. safetensor
Screenshot 2024-09-19 at 00 42 58
  1. When pull a model, display its variants
Screenshot 2024-09-19 at 00 44 45 Screenshot 2024-09-19 at 00 45 50

Fixes Issues

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@namchuai namchuai changed the title feat: add support HF model handle and quant selection [WIP] feat: add support HF model handle and quant selection Sep 17, 2024
@dan-menlo
Copy link
Contributor

@namchuai Quick check, which issue does this address: #1006 or #1145?

I will add a bit to the original Github issues - I'd like to ensure we approach this correctly:

  • API Design (for use to power Jan's Huggingface model download)
  • CLI Design

@namchuai
Copy link
Collaborator Author

@namchuai Quick check, which issue does this address: #1006 or #1145?

I will add a bit to the original Github issues - I'd like to ensure we approach this correctly:

  • API Design (for use to power Jan's Huggingface model download)
  • CLI Design

@dan-homebrew , this PR will address #1006 and will adding support for user to enter hugging face handle like author/model_name. #1145 is already supported in another PR.

@namchuai namchuai force-pushed the j/handle-hf-handle branch 4 times, most recently from 090b083 to b885fdd Compare September 18, 2024 17:16
@namchuai namchuai changed the title [WIP] feat: add support HF model handle and quant selection feat: add support HF model handle and quant selection Sep 18, 2024
@namchuai namchuai marked this pull request as ready for review September 18, 2024 17:17
@nguyenhoangthuan99
Copy link
Contributor

TODO:

@namchuai namchuai merged commit 6989aa4 into dev Sep 19, 2024
4 checks passed
@namchuai namchuai deleted the j/handle-hf-handle branch September 19, 2024 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants