- Support for CPU inference (GGML models)
- GPU Required for GPTQ models
12 GB VRAM recommended (GPTQ models 13B models)
8 GB VRAM recommended (GPTQ models 7B models)
The installer was generated with Inno Setup
Install only for your user (the software create model files in the installation directory, so do not install it at program files)