online diarizer

[jeremy110]

Hi!

I was trying an online diarizer with reference to this PR(#5609). I am using the Nemo Frame VAD and modified it to directly output vad_timestamps (e.g., tensor([[0.0000, 1.5200, 1.5300]])). Then, I call diarize_step function every two seconds, as I'm not quite sure what one "step" (1 frame) represents. However, I only get one speaker output each time, despite calling diarize_step multiple times. Could you please help me identify what might be going wrong? I will provide some relevant code snippets and logs below. Thank you in advance.

MODEL_CONFIG = "/workspace/diar_infer.yaml" 
cfg = OmegaConf.load(MODEL_CONFIG)
online_diar = OnlineClusteringDiarizer(cfg)
filepath = "/workspace/test_60s.wav"
waveform, _ = torchaudio.load(filepath)
vad_model = fvad("/workspace/model/vad_multilingual_frame_marblenet.nemo", "/workspace/diar_infer.yaml")
print(f"waveform.size() {waveform.size())}")
online_diar.buffer_end = 60
scale = 2
for step in range(20):
    start = step * scale
    online_diar.frame_start  = start
    s =  start * 16000
    e = s + scale * 16000  #(step + 1) * scale * 16000
    cur_wav = waveform[:,: e]
    vad_timestamps = vad_model.infer(cur_wav[:, s: ]) + start
    print(f"vad_timestamps: {vad_timestamps}")
    diar_hyp = online_diar.diarize_step(cur_wav.squeeze(0), vad_timestamps)
    print(f"step: {step}, diar_hyp: {diar_hyp}")

log_online1.txt

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale.