Cortex.cpp: Hardware & OS Compatibility Matrix [WIP]

[dan-menlo]

Goal

• Clear articulation of what we support and don't support

Status: In progress

Current issues:

• hardware: Linux ARM CPU Support #527
• feat: Cortex Docker Image #365
• roadmap: Jan & Cortex support for AMD GPUs  jan#3529
• https://github.com/janhq/jan/issues/2832 (Bug related to old Ubuntu version)
• hardware: Windows on ARM support (with Snapdragon X series NPU) #928

OS

OS	Version	Llama.cpp	Tensorrt-LLM	DirectML	Fallback (Comments)
macOS Sonoma	14.6.1	FALSE	FALSE	FALSE
macOS Ventura	13.6.9	FALSE	FALSE	FALSE
macOS Monterey	12.7.6	FALSE	FALSE	FALSE
macOS Big Sur	11.7.10	FALSE	FALSE	FALSE
macOS Catalina	10.15.7	FALSE	FALSE	FALSE
macOS Mojave	10.14.6	FALSE	FALSE	FALSE
macOS High Sierra	10.13.6	FALSE	FALSE	FALSE
Windows
Linux

CPU

CPU	Description	Llama.cpp	Tensorrt-LLM	DirectML	Fallback (Comments)
x64	AMD (aka AMD64)	FALSE	FALSE	FALSE
x86	Intel older instruction set, aka IA-32	FALSE	FALSE	FALSE
x86_64	AMD for most laptops (2013)	FALSE	FALSE	FALSE
AVX	Intel 256-bit instruction set (2011)	FALSE	FALSE	FALSE
AVX2	56-bit instruction set (2013)	FALSE	FALSE	FALSE
AVX-512	512-bit instruction set (2017)	FALSE	FALSE	FALSE
ARMv7	RISC 32-bit instruction set	FALSE	FALSE	FALSE
ARMv8	64-bit instruction set (aka AArch64)	FALSE	FALSE	FALSE
ARMv9	64-bit instruction set (Apple M series)	FALSE	FALSE	FALSE

Not supported (yet)

• PowerPC 32/64
• MIPS32/64
• xTCA: (x86-based Tightly Coupled Architecture): a 64-bit instruction set developed by Intel for its Itanium processors
• PA-RISC: (Precision Architecture): a 32-bit and 64-bit RISC architecture developed by HP
• IA-64: (Intel Itanium Architecture): a 64-bit instruction set developed by Intel for its Itanium processors
• SPARC: 64-bit instruction set

GPU

AI Acceleration	Drivers	Llama.cpp	Llama.cpp w Vulkan	Tensorrt-LLM	DirectML	Fallback
Nvidia	CUDA 11	TRUE	FALSE	FALSE	FALSE
	CUDA 12	TRUE	FALSE	TRUE	FALSE
	Jetson	FALSE	FALSE	FALSE	FALSE
AMD		FALSE	TRUE	FALSE	FALSE
Qualcomm NPUs		FALSE	FALSE	FALSE	TRUE