Python Bindings

[dnhkng]

Currently, all the listed Python bindings on the main ReadMe are broken and do not work on the current API.

This PR removes the links to the non-functional Python-binding repositories, and adds a simple binding system that works on Ubuntu x86 with CUDA.

This still needs testing on Windows, and it's not working on OSX yet, but better than what's currently available.

[bobqianic]

If possible, I think a Python binding test should be added in the CI.

[ggerganov]

If possible, I think a Python binding test should be added in the CI.

Yup, would be useful to have a CI.

This still needs testing on Windows, and it's not working on OSX yet, but better than what's currently available.

What are the issues on OSX related to?

[vivekuppal]

I was able to execute example.py file in python bindings folder with a few small tweaks on windows

Copy below dlls to current working directory so whisper.dll could be correctly loaded

        KernelBase.dll
        advapi32.dll
        cublas64_12.dll
        cublasLt64_12.dll
        cudart64_12.dll
        kernel32.dll
        msvcp140.dll
        ntdll.dll
        ucrtbase.dll
        vcruntime140.dll
        vcruntime140_1.dll
        whisper.dll

Execution log

(venv) C:\git\dnhkng\whisper.cpp\bindings\python>python example.py
whisper_init_from_file_with_params_no_state: loading model from './models/ggml-medium.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head  = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1024
whisper_model_load: n_text_head   = 16
whisper_model_load: n_text_layer  = 24
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 4 (medium)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 CUDA devices:
  Device 0: NVIDIA GeForce GTX 1660 SUPER, compute capability 7.5
whisper_backend_init: using CUDA backend
whisper_model_load:     CUDA buffer size =  1551.93 MB
whisper_model_load: model size    = 1551.56 MB
whisper_backend_init: using CUDA backend
whisper_init_state: kv self size  =  132.12 MB
whisper_init_state: kv cross size =  147.46 MB
whisper_init_state: compute buffer (conv)   =   25.54 MB
whisper_init_state: compute buffer (encode) =  170.21 MB
whisper_init_state: compute buffer (cross)  =    7.78 MB
whisper_init_state: compute buffer (decode) =   98.25 MB
 And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

[dnhkng]

@vivekuppal Thanks for testing on Windows!

[dnhkng]

@ggerganov

The autogenerated wrapper is just not loading the library:

>>> import whisper_cpp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/david/Projects/whisper.cpp/whisper_cpp.py", line 4, in <module>
    import whisper_cpp_wrapper
  File "/Users/david/Projects/whisper.cpp/whisper_cpp_wrapper.py", line 863, in <module>
    _libs["whisper"] = load_library("whisper")
  File "/Users/david/Projects/whisper.cpp/whisper_cpp_wrapper.py", line 551, in __call__
    raise ImportError("Could not load %s." % libname)
ImportError: Could not load whisper.

Trying to rebuild the wrapper with ctypesgen has the following errors:

% ctypesgen whisper.h -l whisper -o whisper_cpp_wrapper.py
INFO: Status: Preprocessing /var/folders/sh/c07q498j5k51jd0g5j7vcz400000gn/T/tmposgdmzqi.h
INFO: Status: gcc -E -U __GNUC__ -dD -U __BLOCKS__ "-D__extension__=" "-D__const=const" "-D__asm__(x)=" "-D__asm(x)=" "-DCTYPESGEN=1" "-D_Nullable=" "-D_Nonnull=" "/var/folders/sh/c07q498j5k51jd0g5j7vcz400000gn/T/tmposgdmzqi.h"
ERROR: gcc -E: In file included from /var/folders/sh/c07q498j5k51jd0g5j7vcz400000gn/T/tmposgdmzqi.h:1:
ERROR: gcc -E: In file included from /Users/david/Documents/Projects/whisper.cpp/whisper.h:4:
ERROR: gcc -E: In file included from /Users/david/Documents/Projects/whisper.cpp/ggml.h:207:
ERROR: gcc -E: In file included from /Library/Developer/CommandLineTools/usr/lib/clang/15.0.0/include/stdint.h:52:
ERROR: gcc -E: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdint.h:52:
ERROR: gcc -E: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:32:
ERROR: gcc -E: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:81:2: warning: "Unsupported compiler detected" [-W#warnings]
ERROR: gcc -E: #warning "Unsupported compiler detected"
ERROR: gcc -E:  ^
ERROR: gcc -E: 1 warning generated.
INFO: Status: Parsing /var/folders/sh/c07q498j5k51jd0g5j7vcz400000gn/T/tmposgdmzqi.h
ERROR: /Users/david/Documents/Projects/whisper.cpp/ggml.h:294: Syntax error at 'ggml_fp16_t'
ERROR: /Users/david/Documents/Projects/whisper.cpp/ggml.h:300: Syntax error at 'x'
ERROR: /Users/david/Documents/Projects/whisper.cpp/ggml.h:303: Syntax error at '*'
ERROR: /Users/david/Documents/Projects/whisper.cpp/ggml.h:303: Syntax error at 'int'
ERROR: /Users/david/Documents/Projects/whisper.cpp/ggml.h:304: Syntax error at 'ggml_fp16_t'
ERROR: <built-in>:26: Syntax error at '\n'
ERROR: <built-in>:405: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:162: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:163: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:164: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:169: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:174: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:180: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:197: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:211: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:232: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:287: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:299: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:310: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:858: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:933: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:934: Syntax error at '\n'
ERROR: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:941: Syntax error at '\n'
INFO: Status: Processing description list.
WARNING: Could not load library "whisper". Okay, I'll try to load it at runtime instead. 
INFO: Status: Writing to whisper_cpp_wrapper.py.
INFO: Status: Wrapping complete.

Not sure what the issue is! Maybe some developer stuff for C++ is not installed correctly? I'm a Python programmer, and can't assist here. But success so far on both Windows and Linux (Ubuntu).

[chrisspen]

The sample code in the README does not work. The wrapper builds fine for me, but is inaccessible.

>>> from whisper_cpp import WhisperCpp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'whisper_cpp'

[vonstring]

FWIW this runs fine for me on MacOS. I get the same errors as @dnhkng when running ctypesgen, but the generated wrapper works fine anyway, and the example runs fine too, with GPU support and everything.

[vonstring]

The Pybind11 binding from the readme also works for me, as long as I update the whisper.cpp submodule. But no GPU support, sadly.

[benniekiss]

What's the status of this PR? Have people been using it successfully?

[jechaviz]

I was able to execute example.py file in python bindings folder with a few small tweaks on windows
So sorry, how to did you do that? What binding you choose? Any steps to follow? Thank you so much