Pyannote.audio 3.1.1 and speaker-diarization 3.1 slower than 3.0 on CPU

[askiefer]

Tested versions

• Reproducible in pyannote.audio=3.1.1 and pyannote/speaker-diarization-3.1

System information

AWS instance c5.2xlarge, 8 vCPUs, 16.0 GiB of memory - pyannote.audio 3.1.1 - pyannote/speaker-diarization-3.1

Issue description

I ran two pyannote diarization pipelines, one with the previous version of pyannote / speaker-diarization and one with the updated one for the same 45min video. Here is the output running locally on a Mac using GPU/MPS:

# pyannote/speaker-diarization-3.0 model with pyannote 3.0.1
Diarized with pyannote: video.wav, took 24.07m
Found 108 speakers for video

# pyannote/speaker-diarization-3.1 model with pyannote 3.1.1
Diarized with pyannote: video.wav, took 3.28m
Found 108 speakers for video

Then, I ran the same video on the AWS instance (8 vCPUs, 16.0 GiB of memory) and received the following results:

# pyannote/speaker-diarization-3.0 model with pyannote 3.0.1
      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:29
speaker_counting     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
embeddings           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:13:18
discrete_diarization ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Diarized with pyannote: video.wav, took 14.34595715602239m
Found 108 speakers for videosegmentation   

# pyannote/speaker-diarization-3.1 model with pyannote 3.1.1
segmentation         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:49
speaker_counting     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
embeddings           ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:35:11
discrete_diarization ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Diarized with pyannote: /home/ec2-user/dd-VideoDownloader/bd3ded311505d363bdb52ce5ad699ba7.wav, took 36.54m
Found 108 speakers for video
Diarized with pyannote: video.wav, took 36.54m
Found 108 speakers for video

It appears the embeddings takes much longer in the 3.1 version.

Is there something in the updated library / model that would actually slow down processing in CPU, but speed it up on GPU?

Issue is related to #1481, I wanted to improve diarization runtime and upgraded my libraries. Was very excited at the improvement but realized I was not seeing the same improvements on GPU vs CPU

Minimal reproduction example (MRE)

I did not upload the video file to the google drive but the one I used is 45min

https://colab.research.google.com/drive/1W001tHy774lRFIf3M_oF_oVKNHOA7XTo#scrollTo=2kPb2SHMJGaa

[hbredin]

Closing as duplicate of #1621.
Feel free to continue the discussion there (or re-open that one if you think those are two different issues).