voicevox_core python api - dll loading - windows

[Leikoe]

質問の内容

sorry for posting in english (I don't speak japanese yet)

The problem:
The python api loads onnxruntime.dll from current directory. I got it working after copying onnxruntime_providers_shared.dll and onnxruntime.dll in the example/python/ directory.

But, only CPU mode works, GPU mode fails with the following error

Traceback (most recent call last):
  File "C:\Users\chach\git\voicevox_core\example\python\run.py", line 78, in <module>
    main()
  File "C:\Users\chach\git\voicevox_core\example\python\run.py", line 33, in main
    core.load_model(SPEAKER_ID)
voicevox_core.VoicevoxError: modelデータ読み込みに失敗しました (C:\Users\chach\AppData\Local\Programs\Python\Python310\lib\site-packages\voicevox_core\model\d0.bin): Failed to create session options: Error calling ONNX Runtime C function: D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1106 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\Users\chach\git\voicevox_core\example\python\onnxruntime_providers_cuda.dll"

copying onnxruntime_providers_cuda.dll in the run.py directory doesn't help either:

Traceback (most recent call last):
  File "C:\Users\chach\git\voicevox_core\example\python\run.py", line 8, in <module>
    import voicevox_core
  File "C:\Users\chach\AppData\Local\Programs\Python\Python310\lib\site-packages\voicevox_core\__init__.py", line 1, in <module>
    from . import _load_dlls  # noqa: F401
  File "C:\Users\chach\AppData\Local\Programs\Python\Python310\lib\site-packages\voicevox_core\_load_dlls.py", line 16, in <module>
    CDLL(str(Path(dll).resolve(strict=True)))
  File "C:\Users\chach\AppData\Local\Programs\Python\Python310\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\chach\git\voicevox_core\example\python\onnxruntime_providers_cuda.dll' (or one of its dependencies). Try using the full path with constructor syntax.

• would it be possible to load the required dlls from PATH ?
• would it be possible to bundle required dlls in the .whl like pytorch does ?

VOICEVOXのバージョン

voicevox_core cloned 3/19/2023

OSの種類/ディストリ/バージョン

• Windows
• macOS
• Linux

なるべく詳しく書いてください 記述例:

• Windows 10 Home x86_64 (21H2 | 19044.2728)
• python 3.10
• onnxruntime 1.14.1 cuda

その他

[qryxip]

voicevox_core depends on ONNX Runtime 1.13.1. Could you download this and use the onnxruntime DLLs in it?

voicevox_core-windows-x64-cuda-0.14.1.zip

And one more thing, We also provide DirectML version of voicevox_core. This is primarily supported version for Windows.

voicevox_core-0.14.1+directml-cp38-abi3-win_amd64.whl
voicevox_core-windows-x64-directml-0.14.1.zip

• would it be possible to load the required dlls from PATH ?

Well, AFAIR, DLLs in %PATH% are automatically resolved and loaded on Windows. Aren't they loaded?

• would it be possible to bundle required dlls in the .whl like pytorch does ?

Unfortunately, We decided (didn't we? @Hiroshiba) not to bundle ORT DLLs at least directly because they are too large to upload to PyPI.

[Leikoe]

I got everything working by modifying the _load_dlls.py file.

The dlls in the path were NOT loaded by the provided file.

this is the one I'm now using :

import glob
import platform
from ctypes import CDLL
from pathlib import Path

import os
from ctypes.util import find_library
from pathlib import Path
from os import getenv

PATH = getenv("PATH")
PATHS = [path for path in PATH.split(
    ";") if path != "" and os.path.exists(path)]

for path in PATHS:
    if os.path.exists(f"{path}\\onnxruntime.dll") and "system32" not in path:
        print(f"{path=}")
        CDLL(f"{path}\\onnxruntime.dll")
    else:
        os.add_dll_directory(path)

# if platform.system() == "Windows":
#     pathname = "*.dll"
# elif platform.system() == "Darwin":
#     pathname = "*.dylib"
# elif platform.system() == "Linux":
#     pathname = "*.so.*"
# else:
#     raise RuntimeError("Unsupported platform")

# for dll in glob.glob(pathname):
    # CDLL(str(Path(dll).resolve(strict=True)))

lib_file_names = [
    "torch_cpu.dll",
    "torch_cuda.dll",
    "DirectML.dll",
]


for dll in lib_file_names:
    CDLL(find_library(dll))

the following _load_dlls.py requires the required .dll files to be in path