Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: CI packaging of llama.cpp dependencies into the binary file by default for Cortex's integration into Jan #1369

Open
1 task
hiento09 opened this issue Oct 1, 2024 · 4 comments
Assignees

Comments

@hiento09
Copy link
Contributor

hiento09 commented Oct 1, 2024

Linked to #1217
Given that: The current CI is publishing the binary file and installer to the release, but the binary does not include the llamacpp engine.
Expectation: Jan will pull the Cortex binary that has the llamacpp engine avx2 pre-integrated.

Bugs:

@hiento09 hiento09 added this to Menlo Oct 1, 2024
@hiento09 hiento09 self-assigned this Oct 1, 2024
@hiento09 hiento09 converted this from a draft issue Oct 1, 2024
@hiento09 hiento09 changed the title ci: The Cortex CI needs to package the cortex.llamacpp avx2 dependencies into the binary file by default for Jan's integration into Cortex ci: The Cortex CI needs to package the cortex.llamacpp dependencies into the binary file by default for Jan's integration into Cortex Oct 1, 2024
@hiento09
Copy link
Contributor Author

hiento09 commented Oct 1, 2024

Approach:

  1. Update Cortex CI: Similar to the old approach like Nitro, ship multiple Cortex binaries including different llamacpp variants.
    Image
  2. Update Jan CI: Modify Jan's CI to pull llamacpp engines from the cortex.llamacpp repo and the Cortex binary file from the cortex repo.

@hiento09
Copy link
Contributor Author

hiento09 commented Oct 1, 2024

cc @dan-homebrew @louis-jan @vansangpfiev

@hiento09 hiento09 changed the title ci: The Cortex CI needs to package the cortex.llamacpp dependencies into the binary file by default for Jan's integration into Cortex ci: The Cortex CI needs to package the cortex.llamacpp dependencies into the binary file by default for Cortex's integration into Jan Oct 1, 2024
@hiento09 hiento09 changed the title ci: The Cortex CI needs to package the cortex.llamacpp dependencies into the binary file by default for Cortex's integration into Jan ci: The CI needs to package the cortex.llamacpp dependencies into the binary file by default for Cortex's integration into Jan Oct 1, 2024
@gabrielle-ong
Copy link
Contributor

gabrielle-ong commented Oct 3, 2024

@hiento09 is this a Sprint21 issue and is there a corresponding PR? Else we can shift it to Sprint22
cc @dan-homebrew

@hiento09 hiento09 moved this from Review + QA to Scheduled in Menlo Oct 3, 2024
@hiento09
Copy link
Contributor Author

hiento09 commented Oct 3, 2024

@hiento09 is this a Sprint21 issue and is there a corresponding PR? Else we can shift it to Sprint22 cc @dan-homebrew

Let shift this issue to sprint 22 @gabrielle-ong , we have not finalized the approach yet

@dan-menlo dan-menlo changed the title ci: The CI needs to package the cortex.llamacpp dependencies into the binary file by default for Cortex's integration into Jan ci: CI packaging of llama.cpp dependencies into the binary file by default for Cortex's integration into Jan Oct 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Icebox
Development

No branches or pull requests

2 participants