bug: Model failed to load with status code: 500, could not load engine llamacpp

[dan-menlo]

Goal

• A lot of useful error messages are currently not being bubbled up to CLI
• We should bubble up as many error messages, to allow users to file more informative bugs or know what is going on

Current State

Running llama3.1 failed with model status code 500:

The real error message is in the logs:

[gabrielle-ong]

v154 - also getting this error

Workaround

cortex engines install llama-cpp

which should have been pre-installed.

cc @hiento09 - discord discussion

[gabrielle-ong]

should be solved by #1369?

[gabrielle-ong]

Copying's Hien's post from #1396

I just tried a scenario as follows and encountered a similar issue to what you mentioned:
My machine is a Mac Silicon
I downloaded version 152, but the mac amd64 version – it still installed because my machine has Rosetta, but at this point, the llamacpp engine installed is the amd64 variant, which causes it to not run on Mac Silicon.
I ran cortex update to download version 154 universal and tried running cortex chat tinyllama:1b-gguf.

[gabrielle-ong]

Proposal: Investigating -> Feature (high pain) -> Eng Planning

• original bug should be fixed, possibly same error as bug: cortex models start: Error HTTP error: Failed to read connection / Model failed to load 500 #1474
• Task remains as a eng task: Useful error messages eg 402, 500 are not being bubbled up to CLI

[gabrielle-ong]

Decision from Dan / Nicole:

• new Epic: bubble up error messages from Engines, highlight important error messages
• Status: planning, Eng Spec: Planning, Sprints: 24/25

[gabrielle-ong]

Closing this bug as resolved,
creating a separate discussion for bubbling error logs #1514