About Mistral-Small-Instruct-2409

[showgood163]

Hi there,

Will you test Mistral-Small-Instruct-2409?

22B so not too expensive for a test.

[official-elinas]

I'm trying to, but keep getting ModuleNotFoundError: No module named 'nemo'

Working on trying to fix this.

[official-elinas]

Fixed it but I'm not sure about the engine type / not loading the model anymore.

[hsiehjackson]

You are available to use vLLM or HF to run inference. BTW, I just found they change their claimed context length from 128K to 32K.