bug: API /models/start does not work - weird response message

[louis-menlo]

Cortex version

v1.0.0-168

Describe the Bug

I'm checking out the latest nightly version, started the server successfully, however, when I request /models/start, it returns a 400 error with a very peculiar message response: "om ". Please see the attached screenshot. I can run the model via CLI tho.

Steps to Reproduce

1. Pull a model
2. Start server
3. Request a /models/start request, with correct model name or alias

Screenshots / Logs

What is your OS?

• MacOS
• Windows
• Linux

What engine are you running?

• cortex.llamacpp (default)
• cortex.tensorrt-llm (Nvidia GPUs)
• cortex.onnx (NPUs, DirectML)

[louis-menlo]

After stopping the server and restarting it, I was able to make it work, but sending the 2nd request resulted in the same issue. It might be a zombie process or something from the past. Now I received a 409 from the CLI but a 400 with this strange message from the API.

[gabrielle-ong]

@louis-jan verifying if this is fixed for you?
from v172, its fixed for me - I'm able to send /models/start request even if the model is already loaded.

[louis-menlo]

It's weird here. Checking engines

[louis-menlo]

Oh, it's another issue then; 500 when there's no engine setup. I installed the engine, but it's still 500. Restarting the server doesn't seem to work.

[gabrielle-ong]

Since om issue not happening anymore, shall close this
500 error for engine setup is captured in #1422,
and should be handled in #1514 to bubble up this common error