Beam search algorithm implementation for TDT models #10903

lilithgrigoryan · 2024-10-16T13:58:23Z

What does this PR do ?

PR introduces default beam search and mAES (with Ngram LM fusion) decoding implementations for TDT models.

Collection: ASR

Changelog

[nemo/collections/asr/parts/submodules/tdt_beam_decoding.py]: Default Beam Search and mAES implementations for TDT.

Usage

python examples/asr/speech_to_text_eval.py \
  model_path="$NEMO_MODEL_PATH" \
  dataset_manifest="$DATASET_MANIFEST_PATH" \
  rnnt_decoding.strategy=maes \
  rnnt_decoding.beam.beam_size=8 \
  rnnt_decoding.beam.ngram_lm_model="$KENLM_NGRAM_MODEL_PATH"

GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: lilithgrigoryan <[email protected]>

…into beam_search

Signed-off-by: lilithgrigoryan <[email protected]>

nemo/collections/asr/parts/submodules/tdt_beam_decoding.py

Signed-off-by: lilithgrigoryan <[email protected]>

…A/NeMo into lgrigoryan/tdt_beam_search

Signed-off-by: lilithgrigoryan <[email protected]>

…/tdt_beam_search

…A/NeMo into lgrigoryan/tdt_beam_search

github-actions · 2024-11-13T00:02:52Z

[🤖]: Hi @lilithgrigoryan 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

Signed-off-by: lilithgrigoryan <[email protected]>

…/tdt_beam_search

Signed-off-by: lilithgrigoryan <[email protected]>

artbataev

Thank you so much! Cool work!

github-actions · 2024-11-13T09:21:51Z

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

Your code was analyzed with PyLint. The following annotations have been identified:


------------------------------------
Your code has been rated at 10.00/10

Thank you for improving NeMo's documentation!

github-actions · 2024-11-13T11:24:00Z

[🤖]: Hi @lilithgrigoryan 👋,

We wanted to let you know that a CICD pipeline for this PR just finished successfully

So it might be time to merge this PR or get some approvals

I'm just a bot so I'll leave it you what to do next.

//cc @pablo-garay @ko3n1g

* Timestamps to transcribe (#10950) * inital version Signed-off-by: Nithin Rao Koluguri <nithinraok> * Support for RNNT, TDT, Hybrid Models Signed-off-by: Nithin Rao Koluguri <nithinraok> * move change of decoder stratery from mixin to individual model class Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <[email protected]> * update transcribe_speech.py Signed-off-by: Nithin Rao Koluguri <nithinraok> * uncomment Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <[email protected]> * add docs Signed-off-by: Nithin Rao Koluguri <nithinraok> * fix docs Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <[email protected]> * codeql fixes Signed-off-by: Nithin Rao Koluguri <nithinraok> * unit tests Signed-off-by: Nithin Rao Koluguri <nithinraok> * minor rebase fix Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <[email protected]> * add None case to restore the state set outside using decoding_stratergy() Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <[email protected]> * remove ipdb traces Signed-off-by: Nithin Rao Koluguri <nithinraok> * updates doc for transcription.py Signed-off-by: Nithin Rao Koluguri <nithinraok> * remove preserve alignment for AED models as it doesn;t support it Signed-off-by: Nithin Rao Koluguri <nithinraok> * lint warnings Signed-off-by: Nithin Rao Koluguri <nithinraok> * Apply isort and black reformatting Signed-off-by: nithinraok <[email protected]> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <[email protected]> Co-authored-by: Nithin Rao Koluguri <nithinraok> Co-authored-by: nithinraok <[email protected]> * [🤠]: Howdy folks, let's bump `Dockerfile.ci` to 1b8fce7 ! (#11247) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * [🤠]: Howdy folks, let's bump `Dockerfile.ci` to 47ff44e ! (#11254) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Handling tokenizer in PTQ for Nemo 2.0 (#11237) * Handling tokenizer in PTQ for Nemo 2.0 Signed-off-by: Jan Lasek <[email protected]> * Print log msg and enable overriding Signed-off-by: Jan Lasek <[email protected]> * Warning for legacy tokenizer config Signed-off-by: Jan Lasek <[email protected]> * Save HF tokenizer to make tokenizer_config.yaml (almost) redundant Signed-off-by: Jan Lasek <[email protected]> * Handle tokenizer in a unified way Signed-off-by: Jan Lasek <[email protected]> * Move saving context within export Signed-off-by: Jan Lasek <[email protected]> * Fix typo in get_tokenzier Signed-off-by: Jan Lasek <[email protected]> * Reduce diff Signed-off-by: Jan Lasek <[email protected]> * Drop unused import Signed-off-by: Jan Lasek <[email protected]> --------- Signed-off-by: Jan Lasek <[email protected]> * Fix finetuning datamodule resume (#11187) * fix datamodule resume Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix subclass Signed-off-by: Chen Cui <[email protected]> * docstrings and formats Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> * ci: Move `bump mcore` to templates (#11229) * ci: Move `bump mcore` to templates Signed-off-by: Oliver Koenig <[email protected]> * fix Signed-off-by: Oliver Koenig <[email protected]> * fix Signed-off-by: Oliver Koenig <[email protected]> * fix Signed-off-by: Oliver Koenig <[email protected]> * final Signed-off-by: Oliver Koenig <[email protected]> --------- Signed-off-by: Oliver Koenig <[email protected]> * fix: Update baseline (#11205) Signed-off-by: Oliver Koenig <[email protected]> * Remove deprecated builder_opt param from build command (#11259) Signed-off-by: Jan Lasek <[email protected]> * chore(beep boop 🤖): Bump `MCORE_TAG=aded519...` (2024-11-12) (#11260) Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * [Doc fixes] update file names, installation instructions, bad links (#11045) * rename eval_beamsearch_ngram.py to eval_beamsearch_ngram_ctc.py in docs Signed-off-by: Elena Rastorgueva <[email protected]> * replace out of date installation instructions with pointer to NeMo README installation section Signed-off-by: Elena Rastorgueva <[email protected]> * point to user guide instead of readme Signed-off-by: Elena Rastorgueva <[email protected]> * some link updates Signed-off-by: Elena Rastorgueva <[email protected]> * update more links Signed-off-by: Elena Rastorgueva <[email protected]> --------- Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> * fix(export): GPT models w/ bias=False convert properly (#11255) Signed-off-by: Terry Kong <[email protected]> * ci: Run secrets detector on `pull_request_target` (#11263) Signed-off-by: Oliver Koenig <[email protected]> * fix(export): update API for disabling device reassignment in TRTLLM for Aligner (#10863) * fix(export): update API for disabling device reassignment in TRTLLM for Aligner [feat] Upgrade nemo-export path for aligner to TRTLLM-v12 and use python runtime Signed-off-by: Terry Kong <[email protected]> fix: forgot to always set _disable_torch_cuda_device_set Signed-off-by: Terry Kong <[email protected]> Signed-off-by: Terry Kong <[email protected]> Apply isort and black reformatting Signed-off-by: terrykong <[email protected]> invert torch device set Signed-off-by: Terry Kong <[email protected]> * remove comment Signed-off-by: Terry Kong <[email protected]> --------- Signed-off-by: Terry Kong <[email protected]> * new vfm training features (#11246) Signed-off-by: Zeeshan Patel <[email protected]> Co-authored-by: Zeeshan Patel <[email protected]> * Update pruning and distillation tutorial notebooks (#11091) * Update pruning and distillation tutorial notebooks Signed-off-by: Gomathy Venkata Krishnan <[email protected]> * Update README Signed-off-by: Gomathy Venkata Krishnan <[email protected]> * Update batch size in width pruning script Signed-off-by: Gomathy Venkata Krishnan <[email protected]> * Update README Signed-off-by: Gomathy Venkata Krishnan <[email protected]> --------- Signed-off-by: Gomathy Venkata Krishnan <[email protected]> * Beam search algorithm implementation for TDT models (#10903) * initial commit Signed-off-by: lilithgrigoryan <[email protected]> * add: default beam search implementation Signed-off-by: lilithgrigoryan <[email protected]> * fix: changed to removing duplicate hypothesis in separate function Signed-off-by: lilithgrigoryan <[email protected]> * fix: changed to cartesian product in choosing best hyp Signed-off-by: lilithgrigoryan <[email protected]> * fix: minor fixes in comments Signed-off-by: lilithgrigoryan <[email protected]> * add: maes decoding strategy Signed-off-by: lilithgrigoryan <[email protected]> * add: durations filtering in maes, lm fusion in progress Signed-off-by: lilithgrigoryan <[email protected]> * fix: refactored, added comments, command line args, finalized Signed-off-by: lilithgrigoryan <[email protected]> * fix: removed prints Signed-off-by: lilithgrigoryan <[email protected]> * add: docs Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix: minor fix Signed-off-by: lilithgrigoryan <[email protected]> * fix: rm beam_size=1 exception, rm duplicates check, fix error handling Signed-off-by: lilithgrigoryan <[email protected]> * fix: error handling Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix: removed evaluations file Signed-off-by: lilithgrigoryan <[email protected]> * rn: blank scoring Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * rm: blank scoring and duration beam size Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix: removed durations_beam_size from default beam search Signed-off-by: lilithgrigoryan <[email protected]> * add: logaddexp Signed-off-by: lilithgrigoryan <[email protected]> * rm: prefix search Signed-off-by: lilithgrigoryan <[email protected]> * rn: nested loop over extensions Signed-off-by: lilithgrigoryan <[email protected]> * fix: bug with caching Signed-off-by: lilithgrigoryan <[email protected]> * rm: topk on durations Signed-off-by: lilithgrigoryan <[email protected]> * add: restored prefix search Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * fix: fixed comments Signed-off-by: lilithgrigoryan <[email protected]> * refactored duplicate merging Signed-off-by: lilithgrigoryan <[email protected]> * changes batch scoring Signed-off-by: lilithgrigoryan <[email protected]> * refactored rnnt batch scoring Signed-off-by: lilithgrigoryan <[email protected]> * alsd first working Signed-off-by: lilithgrigoryan <[email protected]> * refactored Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * remove stacking operations Signed-off-by: lilithgrigoryan <[email protected]> * fixes im base class Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * remove potentially uninitialized local variable Signed-off-by: lilithgrigoryan <[email protected]> * default beam search minor fixes Signed-off-by: lilithgrigoryan <[email protected]> * add test, fix maes timesteps Signed-off-by: lilithgrigoryan <[email protected]> * rm file Signed-off-by: lilithgrigoryan <[email protected]> * rm file Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * fix comments Signed-off-by: lilithgrigoryan <[email protected]> * add ngram lm test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix maes_num_steps=1 Signed-off-by: lilithgrigoryan <[email protected]> * fix kenlm model path Signed-off-by: lilithgrigoryan <[email protected]> * fix kenlm model full path Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * made requested changes Signed-off-by: lilithgrigoryan <[email protected]> * merge after isort Signed-off-by: lilithgrigoryan <[email protected]> * add prints to test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add Kenlm to asr requirements Signed-off-by: lilithgrigoryan <[email protected]> * remove prints in tests Signed-off-by: lilithgrigoryan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add kenlm to test requirements Signed-off-by: lilithgrigoryan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rm kenlm from link, add package-name Signed-off-by: lilithgrigoryan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rm second kenlm installation Signed-off-by: lilithgrigoryan <[email protected]> * rm kenlm from dependencies make test optional Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix in test Signed-off-by: lilithgrigoryan <[email protected]> * fix in test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix comments Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * splitted docstrings Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * splitted docstrings Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fixes to python3 type annotations Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * merging Signed-off-by: lilithgrigoryan <[email protected]> * merging Signed-off-by: lilithgrigoryan <[email protected]> * fix in return type Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * rm time_idx Signed-off-by: lilithgrigoryan <[email protected]> * fix comments to python3 style Signed-off-by: lilithgrigoryan <[email protected]> --------- Signed-off-by: lilithgrigoryan <[email protected]> Signed-off-by: lilithgrigoryan <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * update nemo1->2 conversion according to changes in main (#11253) * update nemo1->2 conversion according to changes in main Signed-off-by: Huiying Li <[email protected]> * Apply isort and black reformatting Signed-off-by: HuiyingLi <[email protected]> * format fix Signed-off-by: Huiying Li <[email protected]> * add docstrings Signed-off-by: Huiying Li <[email protected]> --------- Signed-off-by: Huiying Li <[email protected]> Signed-off-by: HuiyingLi <[email protected]> Co-authored-by: HuiyingLi <[email protected]> * Add llama 3.1 recipes (#11273) * add llama 3.1 recipes Signed-off-by: Chen Cui <[email protected]> * Apply isort and black reformatting Signed-off-by: cuichenx <[email protected]> * fix pylint Signed-off-by: Chen Cui <[email protected]> * Fix llama3.1 wrong config in io.json --------- Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: Ao Tang <[email protected]> * Fix Finetune Recipe (#11267) * Fix Starcoder_15 SFT recipe * Fix PP type SFT recipe * Fix PP type SFT recipe * Fix Gemma2b SFT TP=1 * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * Fix more sft recipe * remove pp dtype * remove pp dtype * Configure no restart validation loop in nl.Trainer (#11029) * Configure no restart validation loop in nl.Trainer Signed-off-by: Hemil Desai <[email protected]> * fix Signed-off-by: Hemil Desai <[email protected]> * Skip validation whenever restarting=True Signed-off-by: Hemil Desai <[email protected]> * PR feedback Signed-off-by: Hemil Desai <[email protected]> * Apply isort and black reformatting Signed-off-by: hemildesai <[email protected]> --------- Signed-off-by: Hemil Desai <[email protected]> Signed-off-by: hemildesai <[email protected]> Co-authored-by: hemildesai <[email protected]> * Handle _io_unflatten_object when _thread_local.output_dir is not available (#11199) Signed-off-by: Hemil Desai <[email protected]> * change default ckpt name (#11277) Signed-off-by: Maanu Grover <[email protected]> * Use MegatronDataSampler in HfDatasetDataModule (#11274) * Use MegatronDataSampler in HfDataset Signed-off-by: Alexandros Koumparoulis <[email protected]> * Apply isort and black reformatting Signed-off-by: akoumpa <[email protected]> --------- Signed-off-by: Alexandros Koumparoulis <[email protected]> Signed-off-by: akoumpa <[email protected]> Co-authored-by: akoumpa <[email protected]> * Remove opencc upperbound (#10909) Signed-off-by: Dong Hyuk Chang <[email protected]> --------- Signed-off-by: Nithin Rao Koluguri <nithinraok> Signed-off-by: nithinraok <[email protected]> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Jan Lasek <[email protected]> Signed-off-by: Chen Cui <[email protected]> Signed-off-by: cuichenx <[email protected]> Signed-off-by: Oliver Koenig <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Elena Rastorgueva <[email protected]> Signed-off-by: Terry Kong <[email protected]> Signed-off-by: Zeeshan Patel <[email protected]> Signed-off-by: Gomathy Venkata Krishnan <[email protected]> Signed-off-by: lilithgrigoryan <[email protected]> Signed-off-by: lilithgrigoryan <[email protected]> Signed-off-by: Huiying Li <[email protected]> Signed-off-by: HuiyingLi <[email protected]> Signed-off-by: Hemil Desai <[email protected]> Signed-off-by: hemildesai <[email protected]> Signed-off-by: Maanu Grover <[email protected]> Signed-off-by: Alexandros Koumparoulis <[email protected]> Signed-off-by: akoumpa <[email protected]> Signed-off-by: Dong Hyuk Chang <[email protected]> Co-authored-by: Nithin Rao <[email protected]> Co-authored-by: nithinraok <[email protected]> Co-authored-by: oliver könig <[email protected]> Co-authored-by: Jan Lasek <[email protected]> Co-authored-by: Chen Cui <[email protected]> Co-authored-by: cuichenx <[email protected]> Co-authored-by: Elena Rastorgueva <[email protected]> Co-authored-by: Terry Kong <[email protected]> Co-authored-by: Zeeshan Patel <[email protected]> Co-authored-by: gvenkatakris <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Huiying <[email protected]> Co-authored-by: HuiyingLi <[email protected]> Co-authored-by: Ao Tang <[email protected]> Co-authored-by: Hemil Desai <[email protected]> Co-authored-by: hemildesai <[email protected]> Co-authored-by: Maanu Grover <[email protected]> Co-authored-by: Alexandros Koumparoulis <[email protected]> Co-authored-by: akoumpa <[email protected]> Co-authored-by: Dong Hyuk Chang <[email protected]>

* initial commit Signed-off-by: lilithgrigoryan <[email protected]> * add: default beam search implementation Signed-off-by: lilithgrigoryan <[email protected]> * fix: changed to removing duplicate hypothesis in separate function Signed-off-by: lilithgrigoryan <[email protected]> * fix: changed to cartesian product in choosing best hyp Signed-off-by: lilithgrigoryan <[email protected]> * fix: minor fixes in comments Signed-off-by: lilithgrigoryan <[email protected]> * add: maes decoding strategy Signed-off-by: lilithgrigoryan <[email protected]> * add: durations filtering in maes, lm fusion in progress Signed-off-by: lilithgrigoryan <[email protected]> * fix: refactored, added comments, command line args, finalized Signed-off-by: lilithgrigoryan <[email protected]> * fix: removed prints Signed-off-by: lilithgrigoryan <[email protected]> * add: docs Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix: minor fix Signed-off-by: lilithgrigoryan <[email protected]> * fix: rm beam_size=1 exception, rm duplicates check, fix error handling Signed-off-by: lilithgrigoryan <[email protected]> * fix: error handling Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix: removed evaluations file Signed-off-by: lilithgrigoryan <[email protected]> * rn: blank scoring Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * rm: blank scoring and duration beam size Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix: removed durations_beam_size from default beam search Signed-off-by: lilithgrigoryan <[email protected]> * add: logaddexp Signed-off-by: lilithgrigoryan <[email protected]> * rm: prefix search Signed-off-by: lilithgrigoryan <[email protected]> * rn: nested loop over extensions Signed-off-by: lilithgrigoryan <[email protected]> * fix: bug with caching Signed-off-by: lilithgrigoryan <[email protected]> * rm: topk on durations Signed-off-by: lilithgrigoryan <[email protected]> * add: restored prefix search Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * fix: fixed comments Signed-off-by: lilithgrigoryan <[email protected]> * refactored duplicate merging Signed-off-by: lilithgrigoryan <[email protected]> * changes batch scoring Signed-off-by: lilithgrigoryan <[email protected]> * refactored rnnt batch scoring Signed-off-by: lilithgrigoryan <[email protected]> * alsd first working Signed-off-by: lilithgrigoryan <[email protected]> * refactored Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * remove stacking operations Signed-off-by: lilithgrigoryan <[email protected]> * fixes im base class Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * remove potentially uninitialized local variable Signed-off-by: lilithgrigoryan <[email protected]> * default beam search minor fixes Signed-off-by: lilithgrigoryan <[email protected]> * add test, fix maes timesteps Signed-off-by: lilithgrigoryan <[email protected]> * rm file Signed-off-by: lilithgrigoryan <[email protected]> * rm file Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * clean up Signed-off-by: lilithgrigoryan <[email protected]> * fix comments Signed-off-by: lilithgrigoryan <[email protected]> * add ngram lm test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix maes_num_steps=1 Signed-off-by: lilithgrigoryan <[email protected]> * fix kenlm model path Signed-off-by: lilithgrigoryan <[email protected]> * fix kenlm model full path Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * made requested changes Signed-off-by: lilithgrigoryan <[email protected]> * merge after isort Signed-off-by: lilithgrigoryan <[email protected]> * add prints to test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add Kenlm to asr requirements Signed-off-by: lilithgrigoryan <[email protected]> * remove prints in tests Signed-off-by: lilithgrigoryan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add kenlm to test requirements Signed-off-by: lilithgrigoryan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rm kenlm from link, add package-name Signed-off-by: lilithgrigoryan <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rm second kenlm installation Signed-off-by: lilithgrigoryan <[email protected]> * rm kenlm from dependencies make test optional Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix in test Signed-off-by: lilithgrigoryan <[email protected]> * fix in test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix comments Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * splitted docstrings Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * splitted docstrings Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * add comments Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fixes to python3 type annotations Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * merging Signed-off-by: lilithgrigoryan <[email protected]> * merging Signed-off-by: lilithgrigoryan <[email protected]> * fix in return type Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * fix test Signed-off-by: lilithgrigoryan <[email protected]> * Apply isort and black reformatting Signed-off-by: lilithgrigoryan <[email protected]> * rm time_idx Signed-off-by: lilithgrigoryan <[email protected]> * fix comments to python3 style Signed-off-by: lilithgrigoryan <[email protected]> --------- Signed-off-by: lilithgrigoryan <[email protected]> Signed-off-by: lilithgrigoryan <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: lilithgrigoryan <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

lilithgrigoryan and others added 30 commits July 24, 2024 17:14

initial commit

0f0958b

Signed-off-by: lilithgrigoryan <[email protected]>

add: default beam search implementation

adb08dd

Signed-off-by: lilithgrigoryan <[email protected]>

fix: changed to removing duplicate hypothesis in separate function

30d0599

Signed-off-by: lilithgrigoryan <[email protected]>

fix: changed to cartesian product in choosing best hyp

796bbc7

Signed-off-by: lilithgrigoryan <[email protected]>

fix: minor fixes in comments

47bbffa

Signed-off-by: lilithgrigoryan <[email protected]>

add: maes decoding strategy

1b64767

Signed-off-by: lilithgrigoryan <[email protected]>

add: durations filtering in maes, lm fusion in progress

7551e32

Signed-off-by: lilithgrigoryan <[email protected]>

fix: refactored, added comments, command line args, finalized

417eb2d

Signed-off-by: lilithgrigoryan <[email protected]>

fix: removed prints

4b83af3

Signed-off-by: lilithgrigoryan <[email protected]>

add: docs

51ab336

Signed-off-by: lilithgrigoryan <[email protected]>

Merge branch 'main' into beam_search

f88dcca

Apply isort and black reformatting

1115529

Signed-off-by: lilithgrigoryan <[email protected]>

fix: minor fix

d5cad08

Signed-off-by: lilithgrigoryan <[email protected]>

fix: rm beam_size=1 exception, rm duplicates check, fix error handling

2d8b455

Signed-off-by: lilithgrigoryan <[email protected]>

fix: error handling

90e452c

Signed-off-by: lilithgrigoryan <[email protected]>

merge

3c1dd89

Signed-off-by: lilithgrigoryan <[email protected]>

Apply isort and black reformatting

98f4d53

Signed-off-by: lilithgrigoryan <[email protected]>

fix: removed evaluations file

20dfd07

Signed-off-by: lilithgrigoryan <[email protected]>

Merge branch 'beam_search' of https://github.com/lilithgrigoryan/NeMo …

1495f0d

…into beam_search

rn: blank scoring

0d472cf

Signed-off-by: lilithgrigoryan <[email protected]>

clean up

71f0607

Signed-off-by: lilithgrigoryan <[email protected]>

rm: blank scoring and duration beam size

2f1f495

Signed-off-by: lilithgrigoryan <[email protected]>

Apply isort and black reformatting

e748171

Signed-off-by: lilithgrigoryan <[email protected]>

fix: removed durations_beam_size from default beam search

b75ff04

Signed-off-by: lilithgrigoryan <[email protected]>

merge

5cfbd2d

Signed-off-by: lilithgrigoryan <[email protected]>

add: logaddexp

6b6fa1f

Signed-off-by: lilithgrigoryan <[email protected]>

rm: prefix search

3e95406

Signed-off-by: lilithgrigoryan <[email protected]>

rn: nested loop over extensions

9a386e2

Signed-off-by: lilithgrigoryan <[email protected]>

fix: bug with caching

9d8aeeb

Signed-off-by: lilithgrigoryan <[email protected]>

rm: topk on durations

08aecb7

Signed-off-by: lilithgrigoryan <[email protected]>

lilithgrigoryan and others added 2 commits November 13, 2024 00:35

fix in return type

0d544b5

Signed-off-by: lilithgrigoryan <[email protected]>

Apply isort and black reformatting

f1d8932

Signed-off-by: lilithgrigoryan <[email protected]>

github-advanced-security bot found potential problems Nov 12, 2024

View reviewed changes

nemo/collections/asr/parts/submodules/tdt_beam_decoding.py Fixed Show fixed Hide fixed

lilithgrigoryan and others added 3 commits November 13, 2024 01:42

fix test

74542a8

Signed-off-by: lilithgrigoryan <[email protected]>

Merge branch 'lgrigoryan/tdt_beam_search' of https://github.com/NVIDI…

0070039

…A/NeMo into lgrigoryan/tdt_beam_search

Apply isort and black reformatting

677b88d

Signed-off-by: lilithgrigoryan <[email protected]>

lilithgrigoryan added Run CICD and removed Run CICD labels Nov 12, 2024

lilithgrigoryan added 2 commits November 13, 2024 01:52

Merge branch 'main' of https://github.com/NVIDIA/NeMo into lgrigoryan…

0669d3d

…/tdt_beam_search

Merge branch 'lgrigoryan/tdt_beam_search' of https://github.com/NVIDI…

2169fd0

…A/NeMo into lgrigoryan/tdt_beam_search

lilithgrigoryan added Run CICD and removed Run CICD labels Nov 12, 2024

lilithgrigoryan added 2 commits November 13, 2024 12:54

rm time_idx

28dce00

Signed-off-by: lilithgrigoryan <[email protected]>

Merge branch 'main' of https://github.com/NVIDIA/NeMo into lgrigoryan…

ef99181

…/tdt_beam_search

lilithgrigoryan added Run CICD and removed Run CICD labels Nov 13, 2024

fix comments to python3 style

c61f01b

Signed-off-by: lilithgrigoryan <[email protected]>

lilithgrigoryan added Run CICD and removed Run CICD labels Nov 13, 2024

artbataev approved these changes Nov 13, 2024

View reviewed changes

lilithgrigoryan added Run CICD and removed Run CICD labels Nov 13, 2024

lilithgrigoryan merged commit a2572a7 into main Nov 13, 2024
176 of 321 checks passed

lilithgrigoryan deleted the lgrigoryan/tdt_beam_search branch November 13, 2024 11:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Beam search algorithm implementation for TDT models #10903

Beam search algorithm implementation for TDT models #10903

lilithgrigoryan commented Oct 16, 2024 •

edited by artbataev

Loading

github-actions bot commented Nov 13, 2024

artbataev left a comment

github-actions bot commented Nov 13, 2024

github-actions bot commented Nov 13, 2024

Beam search algorithm implementation for TDT models #10903

Beam search algorithm implementation for TDT models #10903

Conversation

lilithgrigoryan commented Oct 16, 2024 • edited by artbataev Loading

What does this PR do ?

Changelog

Usage

GitHub Actions CI

Before your PR is "Ready for review"

Who can review?

Additional Information

github-actions bot commented Nov 13, 2024

artbataev left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 13, 2024

github-actions bot commented Nov 13, 2024

lilithgrigoryan commented Oct 16, 2024 •

edited by artbataev

Loading