Make nvml access safer & more compatible across devices

[joeyballentine]

To prevent stuff like this if someone has old asss drivers or semi-unsupported GPUs: https://discord.com/channels/930865462852591648/930875396277280788/1228445525540339842

Since these bindings link to drivers, old drivers may or may not have any of these functions. so we can't be sure about any of them.

Yes, its ugly. But it prevents crashing.

[RunDevelopment]

Oh yeah, absolutely not. You're just making up garbage data.

That user even gave you the exact function that failed, so why are you even try-catching anything that is called before that function? All you needed was a wrapper around nv.nvmlDeviceGetArchitecture to return nv.NVML_DEVICE_ARCH_UNKNOWN and maybe something for get_current_vram_usage and supports_fp16(self, ...). That would have been enough.

[joeyballentine]

I just wanted to be extra safe. Imagine if Nvidia change all their functions tomorrow in a driver update. Suddenly nobody would be able to use the pytorch nodes. None of these functions are critical to the usability of pytorch and yet if they don't work, our pytorch nodes won't either.

[RunDevelopment]

Imagine if Nvidia change all their functions tomorrow in a driver update.

That's an unreasonable expectation. Nvml is an Nvidia library. If Nvidia does change their drivers, it's their job to make sure nvml keeps working, not ours.

None of these functions are critical to the usability of pytorch and yet if they don't work, our pytorch nodes won't either.

My problem isn't that you catch errors, but how. You essentially moved the error handling logic as far down the call stack as you could. The result is that you added 10 catches and that PyTorch node still won't show up if there's a bug in NvidiaHelper elsewhere.

Move the error handling up. All you had to do to make PyTorch nodes work as this:

# backend\src\packages\chaiNNer_pytorch\settings.py

should_fp16 = False
try:
    nv = get_nvidia_helper()
    if nv is not None:
        should_fp16 = nv.supports_fp16()
except Exception as e:
    logger.warn(e)
if is_arm_mac:
    should_fp16 = True

And in addition to that, I would still use nv.NVML_DEVICE_ARCH_UNKNOWN as the default value for nv.nvmlDeviceGetArchitecture in case it fails, because this default value makes sense.

[RunDevelopment]

This doesn't fix the error the user had. The error was

chaiNNer\python\python\Lib\site-packages\pynvml\nvml.py", line 4163, in nvmlDeviceGetArchitecture

pynvml.nvml.NVMLError_FunctionNotFound: Function Not Found

The error occurs inside nvmlDeviceGetArchitecture and here's the code of that function:

def nvmlDeviceGetArchitecture(device):
    arch = _nvmlDeviceArchitecture_t()
    fn = _nvmlGetFunctionPointer("nvmlDeviceGetArchitecture")
    ret = fn(device, byref(arch))
    _nvmlCheckReturn(ret)
    return arch.value

The problem isn't that the python function nvmlDeviceGetArchitecture doesn't exist, but that the C API can't find the function nvmlDeviceGetArchitecture is whatever DLL from the driver nvml loads.

[joeyballentine]

oh

[joeyballentine]

@RunDevelopment would you mind just doing this the way you want it? I'm going to close this PR.