epic: Fix Local Engine issues (llama.cpp)

[dan-menlo]

Goal

• Fix Local Engine issues
• Focus on llama.cpp first

Out-of-scope

• Other engines: TensorRT-LLM, ONNX

Tasklist

Our issues boil down to 5-6 main cortex issues

Architecture

• Document Jan -> Cortex integration and communication method (e.g. in Architecture Spec)
• Rename extension? (semantic naming)

Major Cortex Issues

• epic: cortex models pull <ID_or_URL> cortex.cpp#1077
• feat: cortex run model(:gguf) cortex.cpp#1076
  • bug: Mistral 7b results in Unexpected end of JSON input #3361
• epic: cortex engines commands cortex.cpp#1072
• epic: llama.cpp params are settable via API call or model.yaml cortex.cpp#1151
  • roadmap: Jan Context Length issues #2320
• epic: Implement new Model Folder and model.yaml cortex.cpp#1154
• epic: Model Converter Pipeline models#22

[freelerobot]

Outdated. And closing as it is a dupe of #3825