Does mistral support multiple gpus?

[NiuBlibing]

Describe the bug
I build mistral with cargo build --release --features "cuda flash-attn" and run model with ./target/release/mistralrs-server --port 1234 -n 8 plain -m ./Qwen/Qwen2-72B-Instruct/ -a qwen2 on 8*a100 device, the nvitop shown only one gpu' memory is growing and then oom.

Latest commit
3a79137

[b0xtch]

#375

I don't think so

[NiuBlibing]

#395

[EricLBuehler]

Hi @NiuBlibing! Multiple GPUs is not supported yet, but I will add cross-GPU mapping support first before NCCL support.