epic: enabling GPU

[axel7083]

Epic domain

We need an issue to keep track of the requirements to support the GPUs in AI Lab

Detecting

• Detecting the podman VMType #1338
• Detecting GPU(s) on Windows (Name, VRAM) #1144
• Detecting GPU(s) on Mac (Name, VRAM) #1330
• Detecting GPU(s) on Linux (Name, VRAM) #2162

Usage

• LlamaCppPython inference provider should support GPU on Windows WSL #1279

Additional context

No response

[axel7083]

List of issues encountered

Windows

ulimit problems

warning: failed to mlock 74489856-byte buffer (after previously locking 0 bytes): Cannot allocate memory
Try increasing RLIMIT_MEMLOCK ('ulimit -l' as root).

Solution: prefix export use_mlock=False before starting the llamacpp server

ref: abetlen/llama-cpp-python#708

Root requirement with the WSL

Need to run the container with --user 0 to let it access properly the GPU