k2 package conflict with NeMo

[hamjam]

Hi,

I was examining examples/asr/experimental/k2/align_speech_parallel.py script as I wish to get speech alignments from my trained CTC-based ASR model. Somewhere, I was told that I have to install k2 package to use this script and its recommended method is by using scripts/speech_recognition/k2/setup.sh. However, when I executed the setup.sh script, its output is like this:
ImportError: k2 was built using PyTorch 1.7.1+cu101 But you are using PyTorch 1.11.0+cu102 to run it k2 installed with errors! Please check installation manually.
As far as I know, NeMo needs Pytorch 1.10.0 or above to function but it seems that k2 has been built by PyTorch 1.7.1.
I know the k2 based scripts in NeMo are experimental for now, but is there any chance to bypass this?

[GNroy]

Hi @hamjam,

The version check that throws the error was introduced in k2 release 1.5.0, namely in this PR: k2-fsa/k2#949
The error is triggered by the following after-install check in the setup.sh:

NeMo/scripts/speech_recognition/k2/setup.sh

Line 24 in 1d3eb3d

python3 -m k2.version > /dev/null || (echo "k2 installed with errors! Please check installation manually."; exit 1) && echo "k2 installed successfully!"

I just rebuilt k2 with the setup.sh and the check went just fine.

I suspect that in your case k2 wasn't built from source for some reason.

Could you please check if k2 is actually built from source when you run the script? It is expected to occupy all the CPU kernels and last for several minutes.

You can also try downgrading k2 by explicitly specifying the version, like running this block instead of the setup.sh:

K2_MAKE_ARGS="-j" pip install git+https://github.com/k2-fsa/[email protected]#egg=k2
python3 -m k2.version

Side note: the whole k2 info can be obtained by calling k2.__dev_version__ from python.