Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Request: CUDA/OpenCL Builds #45

Open
LLukas22 opened this issue Jun 22, 2023 · 2 comments · Fixed by #56
Open

Feature Request: CUDA/OpenCL Builds #45

LLukas22 opened this issue Jun 22, 2023 · 2 comments · Fixed by #56
Assignees
@louisgv

Comments

@LLukas22
Copy link
Collaborator

LLukas22 commented Jun 22, 2023 •
edited
Loading

Just a heads up, llm will soon have support for metal/cuda and opencl accelerators (see rustformers/llm#325).
These features are mutual exclusive, meaning you can only pack one backend at a time into a binary. E.g. CPU + Metal or CPU + CUDA.

It would be nice to have ready made binaries for those backends, that can be downloaded and just work.
@louisgv
Copy link
Owner

louisgv commented Jun 22, 2023

Yeet!

@louisgv louisgv pinned this issue Jun 23, 2023
@louisgv louisgv self-assigned this Jun 23, 2023
@louisgv louisgv mentioned this issue Jun 30, 2023
Merged
@louisgv louisgv linked a pull request Jun 30, 2023 that will close this issue
feat: cuda, cublast, metal dedicated builds #56
Merged
@louisgv louisgv unpinned this issue Jul 2, 2023
@louisgv louisgv reopened this Jul 2, 2023
@louisgv
Copy link
Owner

louisgv commented Jul 2, 2023

ref: #61

@louisgv louisgv pinned this issue Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@louisgv louisgv

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: cuda, cublast, metal dedicated builds louisgv/local.ai
2 participants
@louisgv @LLukas22