feat: Cortex Docker Image

[tikikun]

Goal

• Cortex can be deployed via Docker
• We should maintain a Cortex Docker Hub
• Should support both GPU and CPU

Tasklist

• Eng Spec
• Docs

[ikraduya]

Hi, I am interested in the project and would like to work on this one :)

[tikikun]

awesome plz go ahead with PR

[ikraduya]

Hi, I noticed that the title has been changed. Is there any other task that needs to be done?

[dan-menlo]

Original post (outdated/deprecated)

Problem
Alpine can build llama cpp, probably will be helpful to build, alpine is very secure in nature and can be used in security tight environment

https://github.com/3x3cut0r/docker/blob/main/llama-cpp-python/Dockerfile

Success Criteria
Good alpine image for nitro

[gabrielle-ong]

Hi @hiento09, we're testing with the instructions in https://github.com/janhq/cortex.cpp/tree/dev/docker
(on Mac and Windows docker desktop)
but not able to get a response for docker hub image, or build from source
Are we missing something - is there a way to test docker nightly image?

@TC117 will test on Linux

[hiento09]

Hi @gabrielle-ong you can check the docker image tag on our nightly notification here https://discord.com/channels/1107178041848909847/1283654073488379904/1306528533530804285

Or on https://hub.docker.com/r/menloltd/cortex/tags

[gabrielle-ong]

Thanks @hiento09!
QAed by Tin for Windows and Linux
Creating a separate task for supporting Docker on Mac Silicon / multi-architecture.
Currently the Linux/amd64 image is not compatible with Mac Silicon with Rosetta or QEMU emulation.