epic: cortex ps and Process Status

[dan-menlo]

Goal

• cortex ps lists running models and free/used memory
• We should ideally be able to show VRAM and RAM use (free vs. total)

Key Questions

• How do we do this in a stateless manner? (or persist a *.lock to Cortex data folder?)

Linked PRs

• chore: some format for cortex ps #1363
• feat: cortex ps #1345

DX

cortex start

• should stdout a link for api playground
• should correctly handle if port is occupied

20240919 09:30:08.293674 UTC 3351741 INFO  Host: 127.0.0.1 Port: 3928
 - main.cc:32
Server started

Current logs a bit confusing

cortex stop

cortex ps

• if no active processes, should return a msg to that effect

[gabrielle-ong]

@vansangpfiev @namchuai
v124, cortex ps on Windows seems broken
VM: test-windows-app-1-rdp

Works well on Mac and Ubuntu

[vansangpfiev]

This is not cortex ps command issue.
The root cause is the SIGINT signal (Ctrl + C) terminated the child process (server process).
It works well when we use exit() to exit chat.
Let me try to handle the SIGINT signal in this case.

[vansangpfiev]

@gabrielle-ong Please update cortex.llamacpp engine to latest (v0.1.34) to use the cortex ps command

cortex engines install cortex.llamacpp

[freelerobot]

What attributes are we showing when there are models running?

For reference this is docker:

[gabrielle-ong]

CLI UX request: cortex ps should show model alias

• since that is expected for cortex models stop commands
  

Models list