Speaker reidentification

[Vermeille]

Hello,

The project I am currently working on needs to identify speakers. Having the audio segments labeled "Speaker 0" to Speaker N" is cool, but labels "Yann LeCun", "Yoshua Bengio" etc are much better.

I am willing to contribute the feature and was thinking of implementing it the following way:

1. Have the user have a directory with speaker samples (speakers/Yann_LeCun.wav, speakers/Yoshua_Bengio.wav, etc)
2. Get a speaker embedding for each file (Time avg pool? Max pool? Attention is out of the equation since it needs)
3. Optional: store the embeddings on disk (safetensors)
4. When doing the speaker clustering, assign each speaker embedding to a centroid (hungarian algorithm?) and use the speaker labels.

What do you think?

Related to: #1310 . Could be an implementation starting point and have two birds with one stone.

[github-actions]

We found the following entry in the FAQ which you may find helpful:

• Does pyannote support streaming speaker diarization?

Feel free to close this issue if you found an answer in the FAQ. Otherwise, please give us a little time to review.

This is an automated reply, generated by FAQtory

[hbredin]

Thanks. This is definitely a recurrent request and would make a nice addition to pyannote.

Related ongoing PR by @flyingleafe would make a good starting point.
It makes the speaker diarization pipeline output one speaker embedding per speaker.

diarization, embedding = pipeline("audio.wav", return_embedding=True)

You could maybe start by having a look at this PR and the three of us (@flyingleafe, @Vermeille, and I) can continue the discussion there.

[hbredin]

To backup my "recurrent request" claim, here is a poll that I recently posted on Twitter (where speaker tracking ~= speaker reidentification)

https://twitter.com/hbredin/status/1647146368782270464

[flyingleafe]

@hbredin By the way, I think that I have applied all your comments to the mentioned PR a long time ago. If you do not have further comments, maybe we can merge it, so that people can start using this apparently desired feature?

[flyingleafe]

@Vermeille in the meanwhile, yes, you can use the PR branch and simply compare the embeddings returned by the pipeline with the known embeddings by cosine distance, then relabel the annotation accordingly.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.