Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add engine install cli #1034

Merged
merged 1 commit into from
Aug 28, 2024
Merged

Conversation

namchuai
Copy link
Collaborator

@namchuai namchuai commented Aug 27, 2024

Describe Your Changes

  • Add engine install with hardware detection.

Screenshots

Engine init cortex.llamacpp on windows. For this machine, AMD Ryzen 5950x, it choose AVX2, cuda version 12.0

image

Engine init cortex.onnx

image

Engine init cortex.tensorrt-llm

image

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@vansangpfiev
Copy link
Contributor

I think we should change from init to install
The command will be something like:

cortex engines tensorrt-llm install

@namchuai namchuai force-pushed the feat/add-engine-init branch from 0145f79 to 44a0247 Compare August 28, 2024 02:38
@namchuai
Copy link
Collaborator Author

I think we should change from init to install The command will be something like:

cortex engines tensorrt-llm install

Hmm, this will need to change a little bit because it goes against the design of cli application. Might have to add multiple subcommand though.

@namchuai namchuai force-pushed the feat/add-engine-init branch from 44a0247 to 287c750 Compare August 28, 2024 03:39
@vansangpfiev vansangpfiev changed the title feat: add engine init cli feat: add engine install cli Aug 28, 2024
@namchuai namchuai merged commit de5716c into feat/cortex-engine Aug 28, 2024
4 checks passed
@namchuai namchuai deleted the feat/add-engine-init branch August 28, 2024 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants